Thoughts on the Okcupid experiments from three reluctant guinea pigs

Happy Friday, readers!

When the story of Okcupid’s experiments broke a few weeks ago, the Stucu ladies were already in a bit of a dark place. Dating-wise, it’s been a wildly unsuccessful summer for the three of us, so it was total insult to injury when Okcupid plopped a big cherry on our shit sundae by announcing that we may have been unwitting, unwilling participants in some… experiments. If you haven’t read up on what happened, check out some of the coverage here, and here and here. Awesome. We appear to be dating in The Truman Show! Which come to think of it would explain a lot.

ts rain

Source: gifstumblr.com

Sure enough, soon after the news broke, LSD received this fun email from Okc’s evil masterminds:

okc

janice

Source: meangirlgifs.tumblr.com

If you’re scratching your head after reading that and saying, “What the fuck?” then that makes four of us (five if we include Janis Ian). First of all, I’d just like to point out that this company clearly has a fetish for dehumanizing its users by comparing us to gross, thoroughly un-cute animals. In January it was scorpions, now it’s large rodents. Okcupid, if you’re going to compare single people to animals, can you at least use cute ones? Or funny, nostalgic ones? These guys seem particularly appropriate:

Pues no hay nadie perfecto..

Source: ffffound.com

Anyway, next time you have something controversial to tell us, maybe don’t let a 17 year old unpaid summer intern put together the “artwork” in MS Paint and fire it off to your users. You fools.

Before my rage bubbles over too much, let’s get to the point of this post, which is to share our thoughts on the experiments themselves (SPOILER ALERT: we’re not thrilled). If you’re Bill Nye or something, and want to read the full, original text about the experiments before you take our incredibly unbiased words for what they are worth, check them out here. Three of them, three of us. #winning.

Experiment #1: LOVE IS BLIND, OR SHOULD BE

Quick summary: for a period of time one day back in January OKC declared it Love is Blind Day and hid all user profile pics. What they “found out” shouldn’t surprise anyone. At all.

First, while profile pics were hidden, there were WAY less first messages sent, compared to a “typical Tuesday.” It looks like an average of around 3,000 messages per hour for the entire time the experiment was running, when it’s normally around 20,000 – 30,000. Essentially, most people didn’t want anything to do with Love is Blind Day.

love is blind

Of the new conversations that were started in that period, people responded more often, and allegedly exchanged contact details more often. 

compared to a typical tuesday

I wasn’t surprised by this either. Far fewer conversations were started. So it makes sense that of the ones that were, they progressed further/faster. The people who participated in Love is Blind day were the very types who don’t place a really heavy emphasis on looks. Or at least the ones who don’t think they do. More on that in a little bit.

As an aside, while I had a depressing personal reaction to some of the results (see below), this is actually the only part of this experiment itself that bothers me. Are the people at OKC routinely monitoring the contents of all messages exchanged, like some creepy big brother matchmaker? Or was it just during this “experiment” to see what happened? I mean, I know I’m using their service, so I don’t expect that messages are totally private. But the idea that rather than just policing messages that people report as inappropriate, someone is over at OKC headquarters watching conversations progress and timing how long it takes for phone numbers to be exchanged, makes me never want to send another message.

Lastly, OKC kept track of how many of the conversations started during the Love is Blind period actually continued once profile pics were restored. The trend was overwhelmingly to abandon the conversation. No matter how deep into the conversation people were. Even after the exchange of 12 messages, there was still a more likely chance that the conversation would end rather than continue. 

convos in progress when pics came back 

This is one of those results that surprises me, but also doesn’t. I mean, on the one hand, I don’t routinely exchange 12 messages with someone who I don’t find interesting. And yet, even those conversations that were going seemingly well tended to end once profile pics were restored. But, on the other hand, we live in a world that places a high value on certain kinds of physical attributes. So basically, Love is Blind day proved that the majority of people who claim “personality is more important to me than looks” are big fat liars. Because like I said above, those are the people who were much more likely to give Love is Blind day a chance. OKC users are big fat liars? That is SHOCKING.

I’d like to think I’m open-minded enough that if we were having a good conversation, but the person turned out to not be my type physically, I’d continue the conversation. i’ve responded to great messages from people whose pictures didn’t strike my fancy right away. Because, despite my tendency to form extreme crushes on total strangers (just last week I fell in love with a stranger on the T), physical attraction is also something that can build for me. Obviously, there has to be some baseline attraction. But much like a hot guy can become hideous based on his personality, a normal guy can become really attractive once I get to know him. And similarly, I’d like to think that there are others out there with the same outlook. But then I read about this experiment, and quite frankly, it made me super sad about the future of my dating life. I’m not especially attractive. Now, I don’t think I’m hideous either. It’s just that I would categorize myself as fairly plain. At least as far as looks go. I’m far from plain personality wise. I’ve got that in spades, and I like who I am. But if looks really are as important to online dating as they appear to be, then I don’t know why I’m bothering with this in the first place. Lord knows I get fewer messages, and go on fewer dates, than L and S. That’s never really bothered me before. I mean, we’re all different people with different tastes, living in different cities. And I tend to be more attracted to country boys (read: hicks), which is not the most populous type of guy in Boston. But then this “experiment” came out, and all I could think was, “well now I know the real reason I’ve been so unsuccessful.” (Oh man, D, now I’m sad. I’m going to resist the urge to throw out a patronizing “Ra-ra you are so amazing I wish we were both into women so I could date you right now”, even though that’s what I genuinely think. Instead I’m going to agree that the dating game can be a serious self esteem crusher. Show me a person who online dates and doesn’t go to this place at some point or another:

and I will show you a liar. Or possibly just a straight man (badum-ching!). Anyway, D, I just wanted to say I love you. Publicly. On our blog. And totally derail this post. Sorry, readers…)

have a lot of feelings

Source: perezhilton.com

Okay, I think I’ve gotten a hold of myself. Carry on, ladies.

I mean, the take-aways here are essentially: 1) virtually no one is interested in pictureless online dating; and 2) the world is a shallow place.

Experiment #2: SO, WHAT’S A PICTURE WORTH?

Did someone say shallow? L here, LSD’s admittedly most shallow contributor, ready to take on experiment numero dos. Basically, OKC asked a sample of users to rate other users’ profiles and pictures separately, and here is what they found:

looks-v-personality

In case, unlike me and S, who had a high school statistics teacher who was so amazing that his last name actually RHYMED with a deity, you have trouble interpreting scatter-plots, let me break it down for you: OKC users, by and large, rated people the same rating for looks and for personality. Now, this means one of two things:

1. People are truly as nice, kind, funny, and cool as they are good looking. Naomi Campbell is a good example of this. She is not only beautiful, but a really nice person.

That’s why she just hit Oprah with her cellphone. OPRAH, PEOPLE!

And before Gandhi passed away, “People Magazine” was frequently putting him on their “50 Most Beautiful People” list. I believe that Paul Wellstone will be remembered not as one of the most compassionate, progressive legislators of our time, but as 2008’s “Sexiest Man Alive.”

OH WAIT…

2. What’s that you say? The above paragraphs reek of sarcasm? You’re right, dear readers, they do, because the much more plausible explanation for this graph is that OKC users are just really shallow and care WAYYY more about what you look like than what you write in your profile. Clearly, the mad scientists at OKC had this hypothesis as well, so they decided to take their experiment one step further: Remove profile text altogether and see if users’ rankings of people differed at all from when they could see profile text. And sure ’nuff, what did they find?

profile-text-experiment

Yup, same basic trend: profile text had very little to do with rankings.

Is this surprising to me? No. As the mastermind behind Tacosdelish, I’ve actually seen firsthand that many people could care less about what you put in your profile, or even what you say when you’re messaging with them. But does that mean that we should all give up unless we have really hot profile pics to post? Not at all, and let me tell you why:

  1. You can screen out a lot of crazy by looking at a profile. One of our mottos here at StuCu is “live every day like it will be your last because there is a good chance you’ll meet a murderer when you’re online dating and he’ll kill you.” Believe it or not, there are a staggering number of potential murderers (as well as womanizers, mentally unstable trainwrecks, perverts, and general fools) who actually just admit to being these things on their profiles. And no matter what a guy looks like in his pictures, if there are major crazy flags in his profile, I screen him out.
  2. A profile gives you something to message about. How many times have we complained on this blog about getting generic messages? Or messages just about our looks? Or messages that were actually intended for Angela Merkel? Your profile gives people the ability to start some sort of conversation with you, or, at the very least, make a witty joke before they ask you out for drinks. 
  3. Most people are just average looking. I’m not trying to pretend I’m better then the average OKC user. I didn’t message a single person or return messages on “love is blind” day because love is not blind.  I am actually legally blind in my left eye, so I know what it is to be blind, and LOVE IS NOT. I need to be attracted to the person I’m going out with, and he needs to be attracted to me, or otherwise the date is a huge waste of time (unless we end up starting a book club or write a plan for world domination or something). Now, I know that attraction can develop over time, but I do think this is only true up to a certain point. That is why, when I’m evaluating people’s profiles and deciding whether to message them or message them back, I put them into  one of three categories. (See, now do you understand why I’m the shallow blogger?) :
    • Category 1: I can’t get past his looks. The pics they posted are just not attractive to me. (15% of men)
    • Category 2: They look passable. This means they have at least one picture that I find attractive, and no pictures that scare me. (65% of men)
    • Category 3: They look super hot. Every picture they posted is attractive to me. (10% of men)

So here’s the thing. Most men fall into Category 2. (I actually think even MORE men could graduate from Category 1 to Category 2 if they stopped posting idiotic bathroom selfies or creepy pics of them in bed.) And, if you’re Cateogry 2, I read your profile to get more information.  Because I believe there is a chance we could have a decent date/you could be the love of my life. So, if there is nothing in your profile, or if it is full of stupid adjectives and not so subtle digs at women, then I will GTFO and move on (sadly, I’d say more than HALF of the Category 2 guys weed themselves out this way). But, if we have even a little bit in common and you are not a She-Man-Woman-Hater, I’ll message you back.

Now,  if you’re Category 1, I do just skip your profile and continue on my way. Sorry. I’m not as nice a person as Naomi Campbell, ok? And, Category 3 guys do get a VERY LENIENT read on the profile, and I’ve been known to go out with a couple Category 3 guys whose profiles have expressed thinly veiled chauvinism or some pretty egregious spelling errors. But this has almost NEVER turned out well for me, so honestly, I do this less and less.

The big takeaway here: Most of us look like average, Category 2 people, and so what people think of our looks is pretty subjective.  So keep on writing those profiles boys. Unless of course, you’re so hot, you look good even when you’re pumping gas

RYAN GOS PUMPING GAS

 

http://www.buzzfeed.com/mjs538/15-insanely-sexy-pictures-of-ryan-gosling-pumping-gas#3qqsag1

Then my answer is YES YES YES.

Experiment #3: THE POWER OF SUGGESTION

I actually asked L and D if I could take this one, because I think it’s the most brazen (and therefore, offensive) of the three experiments. While the first two covered looks and how much they matter (shocker: they matter a lot. I could have told you that, fools) this third one was about compatibility. Basically Okcupid attempted to figure out whether people’s behavior on the site could be influenced by being told they were “good” or “bad” matches when in reality they were the opposite. They did this by manipulating the match percentage of a sample of users and watching what happened. Here’s what the Okc nerds say about match percentage in their blog post:

“By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?”

burn

Source: www.goodreads.com

Um, let’s leave Jay-Z out of this, shall we, Okcupid?

Anyway, I’ll come back to my thoughts on match percentage in a sec. For part one of this experiment, Okcupid told people with low match percentages (30%)  that they were super compatible by making the percentages appear as 90%. Unsurprisingly, those people sent more first messages than those who saw the actual lower percentages.

okc1

Then they wondered, does this perceived compatibility cause people to actually hit it off and send more than just a first message? According to their data yes, it does:

okc2

Anyway, these results were stressing the Okc peeps out, because they basically suggested that their match percentage algorithm was worthless. So they flipped it and reversed it: this time around they told actual “good” matches (90%) that they were “bad” matches (30%) and saw the same general trend. Here are all the results displayed all together:

okc3

In summary, according to Okcupid, users can be influenced to message more often if they’re told they’re a good match, and less often if they’re told they’re a bad match. Unsurprisingly, people have the best chance of hitting it off if they appear to be a good match and actually are a good match (bottom right number).

My one real beef I have with the experiment itself is: for bad matches who are displayed as good, why is four messages considered some magic number of compatibility? Okcupid had this to say about it:

“The four-message threshold is our internal measure for a real conversation. And though the data is noisier, this same “higher display means more success” pattern seems to hold when you look at contact information exchanges, too.”

True, four messages technically constitutes a conversation, but you can’t then conclude that those people actually hit it off. I’ve exchanged four messages with plenty of dudes and decided that I actually hate/could never date them. Same goes for exchanging contact information… if I had a dollar for every time I seriously regretted giving someone my phone #, I would have enough to cover happy hour later today. Which would be awesome.

happy hour

Source: modernmagnolias.blogspot.com

 

Mmmmm happy hour. Focus, S.

Anyway, if Okcupid had instead polled the people involved in this experiment and asked what they actually thought of their match after four messages, or checked back with them a week later, then that data would be much more revealing and accurate. I suspect that for many people it just took that long for the other person to share some detail about themselves that horrified them (and that could have easily been discovered earlier if that person’s info. had been provided truthfully and accurately in the first place). Maybe five messages is how long it typically takes for people to casually reveal whichever of their answers are deal breakers, like that they hate gays or love nuclear war.

Also, not to get all ‘big brother is watching/NSA is listening’, but I’m with D on that last quote from Okc about “looking at contact information exchanges”. The idea that some 23 year old statistics nerd may be reading my messages to see whether I exchanged phone #s with a dude is gross. And creepy. And gross.

So, regarding experiment #3, my first takeaway is that dudes appear to actually look at match percentages, which given the two above experiments and L’s own experiments with Tacos Delish, I wasn’t entirely sure about. Hurrah! Second, I’m not surprised that people can be influenced to some extent (as you can see we’re not talking huge numbers here) by match percentages. Because, if I may remind the nerds of Okcupid, match percentage is not some random, arbitrary number. If you really want to get nerdy, check out their formula for calculating it here. Basically the number is representative of two people’s answers to questions and how important or unimportant they consider the questions to be. The Okcupid people themselves insist that some of these questions are a very reliable predictor of compatibility. Some questions are stupid and meaningless (I hate camping and horror movies, but they’re certainly not deal breakers), but others, likes these, are pretty big deal (you can literally mark them as ‘mandatory’, and I do):

dealbreaker

Source: modernmagnolias.blogspot.com

Yup. A guy’s answers to these matter to me just a smidge. It’s true, a 30% match could probably make me laugh, or we could have off the charts chemistry or even really like each other. But at the end of the day, if he answered his questions truthfully, we’d still disagree on some fundamental things, and odds are those things would become an issue somewhere down the line.

Now on the flip side, a high percentage is no guarantee that I’ll actually be into the person. I look at a guy’s stats (age, location, education level, height etc), then profile and pics. Then match percentage. Then if we message, he could be ass numbingly boring, or reveal that he has only a basic grasp of the English language, or be mean-spirited. Or we could meet and have zero chemistry. There are a million different reasons why a 99% match could still not be the guy for me, but the point is at least I know when I message a 99% match (or even an 80% match) that we’ve presumably cleared the hurdle of agreeing on things that are important to both of us. In short, it saves time. Unless, of course, you’re an Okcupid guinea pig, and like I said above, I have a sneaking suspicion that these poor souls came to the exact same conclusion, just not as quickly. Such is the life of a lab animal.

 

pinky

Source: www.thedailytouch.com

Conclusion

I’ll admit, there’s a part of me, the secretly nerdy psych major part, that finds this all fascinating. We spend so much of our time on this blog being utterly baffled by people’s behavior, so anything that may shed some light on why people do what they do is, on some level, really cool. People are cool! Human behavior is fascinating! Science! Discovery! Progress!

But then there’s the other part of me, the painfully single part who accidentally goes out with bros, cries in public after being dumped, and experiences fade aways on an alarmingly regular basis. That part of me is

not impressed

That part of me thinks that online dating is hard and confusing and ridiculous and soul crushing enough without some Harvard BA in applied mathematics fucking with my user experience for sport. Yes, I realize that I signed a “user agreement” and it basically says Okc can do whatever they want with me. And I think it makes total sense for Okcupid to study the data they already have (and they have plenty) to draw lots of interesting, informative and helpful conclusions. Experiment #1, while stupid, was at least clear cut. Everyone realized what was happening. Nobody was being deceived. Experiment #2 was more deceiving, for sure, but still an A/B test (two versions of something with info. included or left out).

But when these assholes start changing match percentages, now people are just straight up being lied to and that’s when I hop onto my soapbox. Because you, Okcupid, an online dating service that people use to meet their significant others, may have legitimately prevented people from crossing paths who would have otherwise seriously hit it off. Many of us, LSD included, are using your service to hopefully meet the person we want to spend the rest of our lives with, sparing a lot of time, money, misery, humiliation and heartache in the process. Maybe that sounds dramatic, but if you were looking for a blog free of drama and ended up here, you clearly took a wrong turn somewhere.

drama

Source: www.huffingtonpost.com

Believe it or not, Okcupid, we’re not thrilled to sign onto your site day after day and cruise for thoroughly undatable sociopaths and illiterate weirdos, while our friends get engaged and married around us. It sucks. It’s the worst. And your comical disregard for that fact is not only infuriating, it sort of makes me want to stop using your site and go elsewhere. Except as we’ve covered pretty well here…. there is. no. elsewhere.

Plus the site’s arrogant, flippant tone throughout this whole thing (go reread that email above), combined with the way they’ve been dressing it up as if 1. if we’re on the internet period we should expect to be lied to in the name of research on a daily basis and 2. it’s some benevolent way for them to help us have a better experience, is also totally gross to me. This WSJ interview with Okc co-founder Chris Rudder sums it up pretty well:

okc4

Source: http://blogs.wsj.com/digits/2014/08/12/qa-okcupids-cofounder-on-the-growing-pains-of-data-science/

Blech. Sure, Chris Rudder, we’re in control, but we base who we interact with on certain criteria. And if we’re given the wrong criteria or you withhold information from us, then that control is a fallacy. Don’t patronize us, buddy. Also, you’re lucky, so lucky, that a Stucu blogger didn’t wind up in one of these experiments, because you and our poor readers would probably never hear the end of it.

Have a great weekend, everyone!

Advertisements

4 thoughts on “Thoughts on the Okcupid experiments from three reluctant guinea pigs

  1. love you ladies and loved this analysis. what i want to know is how okc’s interest in matching people for longer term relationships – and thus presumably leaving their website – balances with a fear of people abandoning it for another option. is there some interest in them not being totally accurate with match % numbers etc. (beyond this experiment), or are they just relying on dating being hard enough without their help? i want to sick some harvard math nerds on that.
    you all are the best! your posts are my favorite distraction from/reward for doing my work, so thank you so much for your wonderful (and hard-earned) research and writing!

    • Thanks for the wonderful comment, E! Given my bitterness towards online dating sites and general paranoia just in life (haha), I really should have thought about this possibility, but I honestly didn’t until you mentioned it. It does make sense and actually L had mentioned when I wrote my eHarmony critique a few months ago that there was something comical/suspicious about an online dating service attempting to rope people in to 2 year memberships, when the entire point of their services is (allegedly) to match you with someone quickly and successfully. Obviously they all have a financial incentive to keep single people toiling away on their site, but you’re right, if people get TOO frustrated/discouraged they risk losing them to another (from my experience, equally shitty) site. So in theory there has to be a sweet spot between those two options, and I can’t imagine those evil nerds wouldn’t be researching it in some way or another. Is that what you were getting at?

      Thank YOU so much for reading and supporting us always! You’re the best! xo S (and L&D)

  2. Basically women can “date up” but they have to “marry down,” and men will “date down” but will only “marry up”. Men realize this from a young age because to get sex we have to drop our standards. It often takes women a lot longer to realize that in order to get married she will eventually have to “marry down” (settle) or stay single. The longer she waits, the older she gets and the more she will have to compromise.

    That’s why women find dating so difficult.

    If your parents were both 6s, your mother could have spent her 20s hooking up with 8s – but your father couldn’t have. Young women days often do that these days. However, when these women reach their late 20s / 30s they will have to drop back down to dating 6s if they want to secure commitment. This process is difficult to accept because after 10 years of hooking-up with 8s she will have to realize that in fact she was only a 6 the whole time (the same applies for 5s hooking up with 7s, 4s with 6s etc). Truly hypergamous women never realize/accept it and so they keep getting dumped, or stay single.

    “Settling” isn’t the best term to use because it sounds depressing. Perhaps we should say “realignment of priorities?”

    I also agree with you on the idea of a soul-mate. I don’t believe in “the one” as such, rather “the timing” – people who just happen to be looking for the same thing at the same time, and then they meet each other.
    And this is why women going for someone out of her league has much more impact on the sexual market than a man doing so. She’ll get laid and he won’t.
    Women generally get more dating options even after accounting for the degenerates, criminals and creeps (read unattractive men) and having more options moulds you into different people, whether you like to accept or not. Having more options makes you selective. At the very least it gives you some validation. It makes you less desperate. It gives you a better idea of what kind of men are best for you because you have the privilege to date so many people. Most men just get down on their knees and flip out a ring for the 1st or 2nd woman who show mild interest in them. Most guys simply arent selective and dont have the long checklists because they never get so many options.

    And after a break up or divorce, women are in most cases better off. They can start an active dating and sex life soon after divorce. Its so common to see single freshly divorced moms having sexual relationships with good looking men. Meanwhile most divorced men have no one but prostitutes to turn to. Its all about “options’ and women have more of them.

    Women rarely experience sexual rejection. So even when a man isnt interested in committing, it at least validates her as a sexual desirable being. Atleast he found her attractive enough to have sex and share intimacy with. This reminds me of the fact that women never get friend-zoned. Its always a Friend-with-Benefit zone. It is so much better than simply being limited and confined to a non sexual being by the object of your affection/crush.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s