Did Robert Scoble Join Mahalo’s Payroll?

That’s the only reason I can think for the pro-Mahalo and anti-Google videos Robert Scoble released this weekend. Because quite honestly most of the information inside the video makes little sense.

If you want to watch here are the videos

The first two parts are about 15 minutes each and the last is 6 minutes so be prepared to invest a little time watching. The only thing I learned from them is that Robert Scoble has little to no understanding of search engines, search users, and search marketers. That and he’s heavily biased against anyone who claims to be an SEO.

Rand’s written up a great counter post dissecting the videos point for point and where Scoble goes astray.

The main premise I get from Robert is that human powered search is less spammable than algorithmic search and thus provides more relevant results. This has to be one of the most naive points of view I’ve ever heard. Any hacker or security expert will tell you that the weak link in the chain is always human beings. Real people are much easier to manipulate than machines.

Mahalo is using 100 editors to build pages on the most searched queries. I don’t know what those 100 people are getting paid, but I doubt it’s very much. Is it really that unimaginable to think you couldn’t pay them more to skew pages in your favor?

From Mahalo’s blog here’s how they identified one site that was trying to spam them.

First, and most simply, it felt spammy. After a little research, we discovered it had a very low Alexa ranking.

Hmm? So a handful of people felt the site was spammy. Not enough Alexa? I guess low tech sites are out and we’ve finally discovered a use for Alexa data. Oh, and they linked to the site in the blog post. Their response to the manipulation attempt was to link to the site trying to manipulate them.

Human beings can be manipulated. Why do you think you get all those spam emails every day. You get them because human beings are falling for them.

Human beings also carry bias in what they think is relevant. What’s relevant to me isn’t necessarily relevant to you. We can both type the same words into a search engine and be looking for completely different things. 100 people at Mahalo can only present results they think are relevant to the majority. An algorithmic search engine makes guesses too, but at least there we can refine our queries. A site like Mahalo doesn’t provide that option. Your search will always be limited to what a small group of people think you should see.

Human Powered Search Does Not Scale

Robert seems to think that you can build a scalable trusted algorithm based on human interaction (This is all in part III). His idea is that if you have 100 friends you trust through a site like Facebook they can provide good sources of information for you. They can tell you what pages are the best ones for your query. Sounds ok in theory except that friends you connect with on a social networking site aren’t granted the same level of trust as friends in the real world. We make much looser connections online than offline. Trust is more than accepting an invite on a social networking site.

Scoble mentions his 5,000 Facebook friends. Do you know 5,000 people who you trust? I don’t either. There’s no way Robert actually knows all 5,000 of those people well enough to know if their judgment can be trusted to send him the best search results.

Robert continues by assuming that if you have 100 friends who can recommend you pages and they each have 100 friends who can further recommend pages and so on you can quickly be connected with experts on any subject who can recommend the most relevant pages for you. Again ok in theory, but not in the real world.

First chances are your 100 friends and you share a lot of friends in common so it doesn’t scale quite as fast as Robert thinks. Second and more importantly is the trust evaporates a lot with each level of separation. Just because I trust a friend’s recommendation doesn’t mean I trust the recommendation of a friend of a friend. Go one more friend out and I would view that person as a stranger. We’re connected through mutual friends, but we would know nothing about each other. That’s not a trusted source of information. That’s a stranger.

Algorithms aren’t perfect by any means. I think Google has quite a few problems at the moment such as returning Wikipedia for nearly every query and going overboard in fighting paid links. And let’s not pretend algorithms don’t have bias either. Algorithms carry the bias of the people who create them. Still an algorithm is less likely to carry as much direct bias as a single human being and they are more difficult to manipulate.

There is definitely room for social powered search. You can find great sources of information through real people. Many of the new web pages I encounter daily were recommendations in the form of a link. I read content on a trusted source and followed a link on the page. But I didn’t find those pages on command. I didn’t ask a question and get those pages returned. Social search can work well, especially in verticals where it can remain more focused. But it will always work best as an addition to algorithmic search, not as a replacement.

I have to ask again if Robert Scoble is now on the Mahalo payroll. The arguments in his video are nonsensical and highly biased and show a complete lack of understanding about search. I thought Scoble was smarter than this which is the reason for the question. He can’t be this clueless can he?

Download a free sample from my book, Design Fundamentals.

8 comments

  1. Wow Jason you found me fast. I think I published all of 5 minutes ago. I never would have thought you knew I existed, but I guess mentioning Mahalo gets your attention fast.

    I do realize Robert isn’t on the payroll of course.

    I didn’t mean to imply that your employees are easily bought, but I did want to point out that Robert is wrong to think that people can’t be manipulated. I’m glad to see that you have some checks in place, but I’ll still argue your employees can be manipulated even if not bought directly.

    I’ll also argue that your employees will always carry their own bias with them. I don’t think that makes them bad people at all. We all have biases. But what it does mean is that the results they provide will never be relevant to everyone. They will always be skewed in some direction. That bias can be manipulated. I think it’s good you have checks in place and that will help prevent the most obvious manipulation, but you can’t catch all manipulation attempts the same way Google can’t detect all paid links.

    You can probably guess I’m not someone who uses Mahalo much, but I assume there are those who will find the site useful. My main point here is that you’re not going to replace algorithmic search and I think that’s something you’d agree with. I think you would see Mahalo as a companion site instead of a replacement.

    I do disagree with you and Robert about SEOs being evil by default. I know some are snake oil salesmen, but I’d be willing to bet the percentage of good to bad is similar to the percentage in most other industries.

  2. Is Scoble on our payroll?! Ha! I wish! Seriously, I would hire Robert in a minute… he’s a great ombudsman for PodTech and was amazing for Microsoft. He is great at starting very interesting discussions… heck, look we’re talking about his videos right now.

    On your issue of could a Mahalo Guide get bought off the answer is, sadly, of course! I mean, if an NBA referrer, our politicians, and the CEOs of major corporations can get bought off why couldn’t one of our “Guides.” However we do have a number of items in place to protect against this:

    1. We hire folks really well.
    2. The relationship with Mahalo is worth more than any bribe someone would pay.
    3. Every SERP in Mahalo is looked at by three folks.
    4. We run statistics to see what Guides are doing what edits–and those stats are going to be available to the public!
    5. The public can see any Guides edit history, and the edit history of any SERP–it’s 100% transparent. Transparency means you can catch some quickly, and reverse their edits even quicker!

    So, it is a concern but we are hiring only the best and training our folks really well. Adding systems to that makes me very confident that we’re not going to have many problems–if any.

    Additionally, in order to survive as a company we’re going to have to be fair, transparent, and accountable for our actions. People are ALREADY holding our feet to the fire for tiny decisions… in that way we’re like the Wikpedia: we can make mistakes, but they’re not going to stick around for long.

    In fact, we’re going to be MUCH better than the Wikipedia in that regard because you have to use your real name, social security number, and addresses in order to edit the pages (and get paid!). So, we’re in really solid shape when it comes to keeping the index clean and fair.

    all the best,

    Jason

  3. I’m not so worried about bias because we put systems in place for that as well:

    a) we have a discussion board associated with every single SERP
    b) we will discuss every link on every page on Mahalo with anyone at any time
    c) we rotate folks around SERPs as time goes on… so, even if you have some bias you’re not going to “own” a SERP for ever.
    d) every single edit in the system is tracked. That means if we run a search of your edits and find that you link to a hate site, or a pro/anti INSERT SUBJECT HERE site, we’re gonna catch you and we can revert your edits instantly.

    Also, the entire team at Mahalo is committed to building a service free from bias…. our only bias is help the users find great site–a bias I think everyone can get behind! :-)

    all the best,

    Jason

  4. Jason when I say bias I don’t mean anything extreme. I’m not thinking about hate sites or anything like that.

    Everything has bias though and if you honestly believe you can create something bias free that’s naive. We all have our own unique experiences in life and so naturally have our own unique biases. You and I could both build a page for a particular search and both of us could build a very useful page. But they would probably be different. That’s the bias I’m talking about.

    It might not seem like such a big deal, but over time it does mean anyone using Mahalo is going to see more pages that the 100 or so of you working there think are the most relevant. And again that doesn’t mean they won’t be good pages. But whether you consciously realize it or not the results will always lean somewhat in a direction that aligns with all of you.

    All the checks you have in place are great and will help. The site shouldn’t be biased towards one person’s point of view, but it will still be biased toward the collective view of 100 which is a small group. And I would even say that your personal voice within that 100 will carry more weight.

    I don’t think that automatically makes Mahalo bad. It makes it not for everyone. I think people who lean in the same way as your 100 will probably find the results great. People who lean in a different direction will find them less relevant. It’s not a big deal since you’re obviously not trying to please everyone.

  5. So Jason – not only will you be able to create and curate all of your results, your 100 staff will also monitor every page for these messageboards? C’mon, geez.

  6. Funny Morton. Twice now I’ve written about Mahalo and both times within 5 minutes someone from the site was here to put their positive spin on things. They are certainly paying attention to what’s said about them.

  7. Steven: I’m obsessive about talking about the project… listening to feedback is VERY important when you’re in ALPHA/BETA mode since that’s when you can actually make changes.

    Morton: Actually, we will in all likelihood have thousands of full and part-time folks working on Mahalo so monitoring the messages boards will not be such a big deal no. We have some solid tools built for this already (i.e. when you post to a message board or submit a link the guide who manages that SERP gets an alert instantly). Would it work for 1M pages? Nope, but it will work just fine for 25,000.

    RE:

    >> The site shouldn’t be biased towards one person’s point
    >> of view, but it will still be biased toward the collective
    >> view of 100 which is a small group. And I would even say
    >> that your personal voice within that 100 will carry more
    >> weight.

    Actually, we don’t think our internal staff will carry more weight then the part-time staff or the members. We will discuss and debate every issue of bias in a public forum. So, if there is any bias–or heck, even perceived bias–everyone will know about it instantly. The cost of inserting bias is so great that our team is not going to risk their jobs for it (not to mention stock options!)…. and we’re certainly not going to stand for it.

    So, while I appreciate the deep thoughts you’ve put into this potentially being a problem I suspect the reality is that it won’t be. In fact, if you can find one example of bias I’ll buy you sushi. :-)

    all the best,

    Jason

  8. Jason you should be obsessive talking about the project and I agree listening to feedback is important.

    I think I’m not making myself clear about bias. I’ll give you an example. Every result on Mahalo has bias in it. The same way every result on Google. Yahoo, MSN, and Ask has bias. The simple fact that someone made a choice means that their is some bias.

    Everyone post I’ve ever written has bias. There’s bias in the choice of post, the choice of language used to write the post, where the posts links and where the post doesn’t link.

    I’m not necessarily talking about a conscious bias or even a negative bias. But everything a human being does has bias in it. You make choices based on the sum total of your experiences. I do too. Your experiences are not mine though and we make different choices.

    it’s got nothing to do with how many people look over a result. It’s simply how people are wired. All people have their own psychological makeup. That makeup influences the decisions they make. When people get together in groups the group has its own psychological makeup that influences the choice of that group.

    The group could be 2, 100, or 1000 it still forms a psychological makeup that influences decisions. When I say your personal bias probably influences things more it’s because Mahalo is yours. it’s your vision. Your vision comes with your personal bias about life and consequently Mahalo gets that bias. Again that’s not good or bad. It just is.

    If you won’t accept that all people have their own bias then you’re either incredibly naive or you’re just refusing to get out of spin mode.

    If you want a more specific example of bias in your results I’ve noticed that a Wikipedia page is almost always among the top 7 results. In fact it’s conspicuously #2 a lot. I don’t think the Wikipedia is the most relevant for much of anything. Most of the time I can find at last a dozen pages that are more relevant to me. Let me repeat that last part. Relevant to me. That’s one example where there’s a difference in bias. I don’t want to see Wikipedia pages in search results on Mahalo. I don’t want them on Google either. You and your editors have decided that the Wikipedia is relevant to most things.

Leave a Reply

Your email address will not be published.

css.php