February 14, 2008

Adult Films Vivid Asks Google, Yahoo to Protect Kids

Vivid CEO Steven Hirsch, speaking at Yale University, will call on Google and Yahoo to take measures to keep explicit material from children, according to a company press release.

Regarding his upcoming address to graduate business students during Sex Week at Yale, Hirsch stated: "Responsible companies in the adult industry such as ours have done a great deal to deter minors from accessing adult material."

"None of the search engines and portals, but particularly Yahoo and Google, has taken any significant steps in this direction. Vivid will work with any company that is ready to make it much more difficult for children to be exposed, even inadvertently, to material intended only for adults. This is not about First Amendment rights, it is about protecting children," according to Mr. Hirsch.

An interesting move given the failure of the .xxx extension over the last few years. But a way to effectively keep children away through some advanced filtering is something that needs to be developed with the growing reach of the web. How this develops will be worth tracking.

Posted by Frank Watson at 4:44 PM | Permalink

November 13, 2006

Google Ranks Google Video Questioning Iran's Authority Over Tabriz

Via Valleywag and The Guardian, Iran is upset with Google over ranking a Google Video too high in the Google results. The video titled Tabriz - Azerbaijan describes that the ancient city Tabriz is located in Southern Azerbaijan, suggestion it shouldn't be part of Iran. A Google search on [tabriz azerbaijan] shows the Google Video as (currently) number one with a snippet that reads; "This video shows Tabriz, a city in Southern Azerbaijan, currently in the territory of Iran." A search for [tabriz] shows this result at the bottom of the first page or top of second page. Iran is upset with Google over this and feels that it is a US centric method of undermining Iran.

Postscript From Danny: To be really clear about how absurd this is, it's not Google directly suggesting that Tabriz is not somehow part of Iran. It's a description in a video that someone uploaded to Google Video. Where does Google itself think Tabriz is located at? According to Google Maps -- which Google runs directly -- firmly within the boundaries of the country of Iran.

Posted by Barry Schwartz at 8:25 AM | Permalink

October 30, 2006

Judge To Rule By End Of Year On Kinderstart Case

Reuters reports that Judge Jeremy Fogel said he will take until the end of this year to rule on the Kinderstart case. The case was about how Kinderstart's ranking and PageRank fell and Kinderstart sued Google on numerous counts for the ranking drop. The judge recently said, "Assuming Google is saying that KinderStart's Web site isn't worth seeing. Why can't they say that? That's my question." So he will consider this and other questions in his ruling.

Posted by Barry Schwartz at 9:03 AM | Permalink

October 4, 2006

Eric Schmidt Warns Politicians That Elections Will Forever Change

The Financial Times reports that Eric Schmidt, CEO of Google, warned politicians at a Tory party conference in Bournemouth that the outcome of general elections will be changed "within five years" by what Eric Schmidt calls "truth predictor" software.

A politician may be making claims live during a debate and a voter will be searching the internet to validate those claims, in real time.

Schmidt said, “One of my messages to them is to think about having every one of your voters online all the time, then inputting ‘is this true or false?’ We [at Google] are not in charge of truth but we might be able to give a probability.”

The Register.com has a link to the speech here (for Windows Media users).

Posted by Barry Schwartz at 9:10 AM | Permalink

September 21, 2006

KinderStart Issues An Amended Complaint Against Google

Eric Goldman wrote that KinderStart has issued a 63 page second amended complaint against Google. KinderStart lost their first case against Google back in July of this year - that case was, in my opinion, ridiculous. This new complaint is even worse. The 43(B)log summarizes the complaints, calling many of them "incomprehensible." Eric Goldman says "I expect Google will file a motion to dismiss, which the judge will grant, at least in part (at minimum, to eliminate the Violation of Free Speech claim). I expect Google to go on the counter-offensive and renew its anti-SLAPP motions."

Posted by Barry Schwartz at 10:51 AM | Permalink

August 31, 2006

Google Uses AdWords Conversion Data To Fight AdSense Fraud?

ShoeMoney blogged that at the Arbitrage Issues session at SES San Jose, during Q&A time, Kim Malone of Google said that the AdSense team may use AdWords conversion data to determine which sites are trying to fraud advertisers and users (often named Made for AdSense sites - MFAs).

Jeremy said, "Kim explained that in many cases they are using Adwords conversion data to tell what MFA sites were actually converting for advertisers. She said using this data helps them easily identify arbitragers." To me, this doesn't sound a 100% right. The Google web search team said time and time again that they would not use this data for any organic ranking purposes. I know the AdSense team is separate from them and is even more closely related to the AdWords team, but to use that data for purposes of detecting MFAs? I am not so sure about this. Anyway, I trust Jeremy heard this but I suspect Kim Malone of Google didn't mean for this to come out. Either way, I would love to hear back from Google on this topic.

Postscript by Jennifer Slegg: Actually, this has been known for quite some time that AdSense uses AdWords conversion data to determine how well each publisher's clicks are converting for advertisers. I covered this nearly a year ago on JenSense in a detailed post about how smart pricing works, and what factors go into determining it.

Postscript by Barry: Ouch! I stand way corrected. Jen is the guru on this stuff. I should have looked at JenSense.com prior to putting myself out like this. Nice to have experts to correct you.

Posted by Barry Schwartz at 9:46 AM | Permalink

August 4, 2006

Searching For Other Search Engines At Search Engines

Yesterday, I thought I discovered Yahoo stealing searching from Google, but I did not, Rand discovered it. Basically, if you do a search on Yahoo for google, Yahoo puts up a Yahoo Shortcut asking you "Want to search the Web?" with Yahoo? Is this fair? Is this respectable? Who cares... But is this relevant?

Navigational searches are important. Does Google throw up a OneBox result doing the same? No, they do not. Yahoo doesn't do it for MSN or Ask.com, only for Google. When I asked GoodROI to ask Tim Converse of Yahoo on his WebmasterRadio show last night, Tim replied that he was unaware of that result but it is possible that it may be a joke. Meaning, sometimes the search engines play jokes with each other.

When I went over to Ask.com to do a search on Google, I got this Smart Answer that is incredibly useful and relevant (IMO) to this search.

When I went over to MSN yesterday I did not get anything special. But today, it seems MSN is playing the joke on Google & Yahoo but not on Ask. They ask "Want to search the Web? Try MSN Search" with the search box to MSN.

So now we have Yahoo and MSN both playing this game. Google doesn't do anything much special for these navigation searches. And Ask.com shows a detailed Smart Answer with details of each search company.

Why does Yahoo and MSN do this? Well, as I said it is most likely because Yahoo and MSN are portals. They attract less web savvy individuals and when they search at Yahoo or MSN, they may not understand that they are actually searching. Sounds kind stupid, but this is the case.

Matt Cutts of Google commented in the SEOMoz post saying;

Yahoo: "Want to search the Web?" User: "Yes, but not with you." Too funny. I wish I had a T-shirt with that on it. :)

So Chris Boggs decided to play artist and make a Did you mean? result in Google for good search engine, it does not really work, but he wanted to have fun.

I doubt this is just a joke between search engines. For a high volume keyword, trust me, it is high volume, like "Google," search engines typically play jokes with each other.

Posted by Barry Schwartz at 9:51 AM | Permalink

August 1, 2006

Google Paying News Sites For Google News Syndication?

Philipp Lenssen has some details on Google paying some news sources, such as the Associated Press, for syndicating their content in Google News. Philipp initially posted the story based on a Mercury News article that was foggy on the details of such relationships. But then Philipp received a statement from Google that read:

Google has always believed that content providers and publishers should be fairly compensated for their work so they can continue producing high quality information. We are always working on new ways to help users find the information they are looking for, and our business agreement with the Associated Press is one example of that.

Now, it is hard to know what this exactly means. Is Google paying the AP for allowing them to crawl and include their content in the Google News index? Will Google be creating a news portal, similar to Yahoo News (Yahoo has long paid to host some news content, though the story suggests Yahoo might be doing more of this). It is still unclear. How does Google determine who to pay and how much to pay? Can you pay Google to be included in Google News? Very interesting discovery by Philipp and I look forward to understanding this in more detail soon.

Posted by Barry Schwartz at 9:45 AM | Permalink

July 14, 2006

KinderStart Becomes KinderStopped In Ranking Lawsuit Against Google

Kinderstart has lost its case over lost rankings on Google, though the company will be allowed to amend defamation claims relating to its PageRank zero score. If it does by September 29, I suspect that reattempt will go down in flames as well. But the entire case exposes vulnerabilities Google has created for itself with mixed messages over how keyword ranking and Pagerank work.

Google Sued Over Site Penalty By KinderStart.com covers the case being filed back in March and provides a link to the actual suit. It was heard in court earlier this month, and you can review the transcript and analysis of that hearing.

Judge dismisses suit over Google ranking from News.com covers yesterday's ruling, where the claims against Google were dismissed. The judge gave leave for KinderStart to revise on some claims, apparently in particular on the idea that KinderStart was defamed by being dropped to a PageRank of zero as reported by the Google Toolbar.

KinderStart now apparently hopes it can enlist other PR0 sites to file a class action lawsuit against Google (info is supposed to be here, but site is currently down). The KinderStart attorney said:

"The decision suggests that, if properly alleged, Google may be defaming a whole class of Web sites sacked with a '0' PageRank," he wrote in a statement. "If plaintiffs show Google manually tampered with even a single Web site's PageRank, Google's entire claim of 'objectivity' of search results and rankings could collapse."

Sure. Fire away with that class action suit. Two class action suits over click fraud, where defendants have real monetary claims arising out of actual contacts with the major search engines, have netted around $60 million for advertisers for over four years worth of advertising activity. Assuming a somewhat nebulous defamation claim won, I can't imagine the settlement would be for much.

Keep in mind that by default, the PageRank meter is still not turned on, to my knowledge. Toolbar users have to specifically enable it. I've never seen any stats or breakdowns on who uses the PR meter, but that seems to be mainly site owners concerned about SEO, rather than typical web surfers.

Still, the case highlights a Google vulnerability. Google has argued in this case that ranking is subjective, an opinion that it offers about web sites. But go to its technology page, and you get this:

PageRank Technology: PageRank performs an objective measurement of the importance of web pages by solving an equation of more than 500 million variables and 2 billion terms. Instead of counting direct links, PageRank interprets a link from Page A to Page B as a vote for Page B by Page A. PageRank then assesses a page's importance by the number of votes it receives. PageRank also considers the importance of each page that casts a vote, as votes from some pages are considered to have greater value, thus giving the linked page greater value. Important pages receive a higher PageRank and appear at the top of the search results. Google's technology uses the collective intelligence of the web to determine a page's importance. There is no human involvement or manipulation of results, which is why users have come to trust Google as a source of objective information untainted by paid placement.

So what is it, objective or subjective, or argue what's most convenient, as John Battelle raised earlier. The answer to me gets confused by Google's outdated information online plus confusion between PageRank and ranking.

Ranking, or keyword ranking, is where a site appears in response to a keyword search. It's supposed to be an objective decision made by using a computer algorithm to sort through factors, though not said is how some of those factors might have subjective decisions made over them.

PageRank is a numeric score that counts how important a page is based on analyzing the links pointing to it. It is one of many factors that Google uses to decide where a page should appear when you do a keyword search. In other words, PageRank is part of what determines keyword ranking, but it's not the only factor, nor is it the same as keyword ranking.

But doesn't Google say that pages with a higher PageRank appear at the top of the search results. Yes, and it says this incorrectly. That's right, Google's statement on this is flat out wrong. Wrong, wrong, wrong. Wrong. WRONG.

Am I clear enough? But how can I say Google's official information is wrong? First, I can demonstrate it, as I've done before. Try this tool. Here's a search for cars. Notice how the movie Cars is ranked second. The home page for the site listed is a PR5, putting it above several pages ranking below it with a higher PR score. Got Firefox? Try Aaron Wall's new tool that makes seeing this type of thing even easier. End Of Demonstration.

Google has tons of things they've said publicly that get outdated like this or aren't explained properly by those charged to write up copy. In particular, Google has allowed PageRank to be a synonymous term to mean how a site ranks. You can see how this makes life confusing by the first paragraph in the News.com story about the case:

KinderStart, a directory and search engine for information related to children, sued Google in March after it fell to a "zero" ranking in the Google index.

Actually, I believe that two different things happened. KinderStart:

  • No longer had good keyword rankings, not in the first page of results, but perhaps still buried further down unless it was banned completely. And if it was banned completely, that's not a "zero" rank but instead just called a ban.  
  • Probably had a penalty put on it manually that produced a zero score in the PageRank meter.

The judge does not seem to be saying Google defamed the site through a lower keyword ranking. But he does seem to suggest that the PageRank score in the Google Toolbar meter might have that issue. From Eric Goldberg's nice write-up on the case (and he has a copy of the ruling there, as well):

Googles statement as to whether a particular website is worth your time necessarily reflects its subjective judgment as to what factors make a website important. Viewed in this way, a PageRank reflects Googles opinion. However, it is possible a PageRank reasonably could be interpreted as a factual statement insofar as it purports to tell a user how Googles algorithms assess the importance of the page youre viewing. This interpretation would be bolstered by evidence supporting Googles alleged representations that PageRank is objective, and that a reasonable person thus might understand Googles display of a 0 PageRank for Kinderstart.com to be a statement that 0 is the (unmodified) output of Googles algorithm. If it could be shown, as Kinderstart alleges, that Google is changing that output by manual intervention, then such a statement might be provably false.

I'm actually surprised the judge doesn't seem to know that Google does indeed change that output by manual intervention. That's what the entire SearchKing case was about. First some background on that:

The case involved another US District Court judge ruling that yes, Google had manipulated the PageRank score showing for SearchKing and that it had a constitutionally protected right to do so, to offer its opinion this way.

Of course, the ruling confuses PageRank and keyword ranking as I've explained above often happens:

PageRanks are opinions -- opinions of the significance of particular Web sites as they correspond to a search query.

Still, since the case was indeed focused about the PageRank meter, I suspect we're safe in knowing this was about PageRank scores getting protected status. And what the KinderStart case now tells us is that Google (and other search engines) also have the right to do keyword rankings however they like.

We'll see if the PageRank scores get challenged again. Certainly Google could short-circuit this by dropping the scores and the meter altogether (please do it). As explained, few people to my knowledge use them, and plenty of site owners are tired of newbie search marketers obsessing over them. PageRank was mainly a marketing tactic for Google that's long since been blowing up in its face.

If the meter doesn't go away, certainly Google needs to take a harder look at what it says about both the Google Toolbar and keyword rankings if it doesn't want to be vulnerable in future court cases (plus just be consistent with the public).

For example, what's a site owner told about a PR0 score:

A page may be assigned a rank of zero if Google crawls very few sites that link to it. Additionally, pages recently added to the Google index may also show a PageRank score of zero because they haven't been crawled by Googlebot yet and haven't been ranked. A page's PageRank score may increase naturally with subsequent crawls, so this shouldn't be a cause for concern. To learn more about PageRank, please see http://www.google.com/technology/index.html

There's no mention of the fact that you might have a PR0 score because Google has manually intervened to reduce it. And as for what it tells the general public:

Wondering whether a new website is worth your time? Use the Toolbar's PageRank display to tell you how Google's algorithms assess the importance of the page you're viewing.

Again, it's more than just the algorithms being involved. Human are making decisions that impact that score, as well.

In short, Google is continuing to make statements that PageRank is objective to the public, but in two court cases now, it has said the scores are subjective. One case as supported its right to make subjective cases. The other has supported a defendants right to challenge if those subjective opinions are fair or defamatory. We'll see what happens next.

Finally, the entire human intervention thing with PageRank scores brings back the issue of Google long saying there's no human intervention in keyword ranking, such as they used to say about censorship:

Google does not censor results for any search term. The order and content of our results are completely automated; we do not manipulate our search results by hand.

And similar to what they still say here:

Sites' positions in our search results are determined automatically based on a number of factors, which are explained in more detail at http://www.google.com/technology/index.html. We don't manually assign keywords to sites, nor do we manipulate the ranking of any site in our search results.

In general, webmasters can improve the rank of their sites by increasing the number of high-quality sites that link to their pages. For more information about improving your site's visibility in the Google search results, we recommend reviewing our webmaster guidelines. They outline core concepts for maintaining a Google-friendly website.

As I've written before, Google does indeed hand manipulate results, but not in the sense of trying to reorder them. Instead, it manually intervenes in terms of banning some sites or putting overall ranking penalties on them. There's even been updated attempts to help site owners know when they've been banned through the Google Sitemaps program.

Overall, Google's got plenty of mixed messages out there that don't help on the PR front and potentially leave it vulnerable on the legal front, as this case has shown.

Posted by Danny Sullivan at 8:48 AM | Permalink

July 6, 2006

Kinderstart Transcript Available

Eric Goldman posted the Kinderstart transcript and other case documents on his site. Recently, Kinderstart's case was heard in court and the judge requested Kinderstart to provide some more information. The full, 45 page, transcript of the June 30th hearing can be downloaded here.

Posted by Barry Schwartz at 9:15 AM | Permalink

July 3, 2006

KinderStart.com Case May Proceed To Court?

News.com reports that the KinderStart.com case might proceed to court, based on this past Friday's hearing. Kinderstart.com initially sued Google for a site penalty that downgraded the site's rankings in the Google search results. Kinderstart.com claims Google violated antitrust laws, "What Google is trying to do is take out the competition," Kinderstart.com's lawyer said. The judge gave KinderStart.com's lawyers until September 29th to make revisions to the complaint. The judge said, "You can't just file a blanket lawsuit and say, 'We think we're going to find some stuff.'" Also see news brief at ComputerWorld.

Posted by Barry Schwartz at 8:55 AM | Permalink

June 28, 2006

KinderStart.com To Be Heard In Court Friday On Google Penalizing Site

Webuser reports that the KinderStart.com lawsuit against Google for penalizing their site will be heard in court this Friday. It is not clear if there will be a ruling on Friday, the article does state that the judge "will rule at 9am in California (5pm BST) on nine counts, including whether or not Google should warn sites about changes in their ranking as well as on financial damages KinderStart is seeking." Can you imagine the buzz and controversy in the SEO community if Google loses the case?

Posted by Barry Schwartz at 12:32 PM | Permalink

June 22, 2006

Google Answers: Ask Whatever You Like, Except About Google

We wrote earlier about Google pulling a question at Google Answers about Google. Philipp Lenssen at Google Blogoscoped followed-up further and found that Google officially disallows people to ask questions about the company because the researchers at Google Answers aren't Google employees.

Got that? Freelance researchers are apparently qualified to answer questions about any other company in the world, but when it comes to Google, special treatment is required. Incredible.

In Google Answers Question Removed, Philipp says he was told:

Questions about Google, Google Search, and search engine optimization are not allowed because Google Answers researchers are not employees of Google. Researchers dont have access to any inside information. The information they do have access to is available for free on the Google help pages or by writing to Google support.

And Steve Hall, who started this all when his question was rejected earlier this week, was told:

We'd like to clarify the reason for removal of this question. Please note that Google Answers researchers are not employees of Google. They are independent contractors, and they only have access to information about Google and Google Search that is publicly available. Therefore, all users with questions about Google and/or Google Search are directed to these Google support pages.

In the comments to Steve's post, someone raises a good point that Google might have this policy to help keep those getting responses from thinking they are getting "official" information from Google. I can understand that. But that can also be dealt with differently than just removing questions wholescale. And no questions on search engine optimization, as Philipp was told? Please.

For the record, the Google Answers FAQ says this about questions that aren't allowed:

Google Answers discourages and may remove questions that:

  • request private information about individuals
  • want assistance in conducting illegal activities
  • are meant to sell or advertise products
  • refer or relate to adult content
  • are homework or exam questions
  • seek specific information about Google or Google Answers (email answers-support@google.com instead)

Fair to say, I think that last line should go. People should be able to ask about Google and Google Answers, at the very least because such restrictions make the entire system seem silly.

More important, the Google help pages and other information about Google do NOT have everything you'd want to know about Google. Consider:

  • How does Google technically perform censorship in China? (Answer -- see this good New York Times article, with info not on the Google web site that I know of)  
  • How did a web site recently get so many spam pages indexed so quickly in Google? (Answer -- our article is one of many that explains its both a glitch with Google's site: command plus probably just a big problem with Google's spam control pages, something not covered on the Google web site)  
  • Is it cloaking if the New York Times puts up a page in front of those clicking from Google to reach paid articles? (Answer -- it's a matter of debate, as going on in our Search Engine Watch Forums. Google has general guidelines, but there's no agreement on whether these apply. And there's nothing on the Google site giving a definitive answer).

Out of curiosity, I did a little searching at Google Answers to see if much was getting through about Google. Not much, that I could see. But this question caught my eye, Mod Rewrite code for the .htaccess file. It asks:

I have a website called www.greathostels.com it is written in php i need to know the code to put in the htaccess file to make it search engine friendly using mod_rewrite as at the moment its not effectivly spidered.

The answer was a list of pointers to other sites, all of which look pretty useful to me, someone who is not an expert. But the person asking also raised a good point:

If asking a Php code question i think the answer should be provided by someone who knows PhP code.

Which got him this over-the-top response:

Apparently you negelected to read the FAQs for the Google Answers service:

"Are Researchers experts in their field?"

"All Google Researchers are tested to ensure that they are expert searchers with excellent communication skills. Some of them also have expertise in various fields. Your question may be answered by an expert in a particular field or by an expert searcher. Either way, if you are unsatisfied with your answer for any reason, you may apply for a full refund." http://answers.google.com/answers/faq.html#experts

Our job is to provide an answer to your question. If we personally lack the expertise to do so, we seek out authoritative resources on the internet.

Therefore, I referred you to an authoritative site which provided information which was extremely specific to your question. To imply that the authors of The SEO Toolset website are not experts in their topic, when they authored precisely the information you requested, and created precisely the URL Rewriting Tool which you so badly need, is the heighth of insolence and absurdity.

I would request that my answer be removed by the editors, simply to remove the taint of my association with you, but they tend not to remove answers which have satisfactorily answered the question.

If you ever plan to use this service again, I suggest you register under a different username, given the fact that other researchers will be more than reluctant to deal with someone who doesn't bother to inform themselves about what to expect from the service.

Ouch. I can understand the researcher feeling slighted. But it's also a fair opinion to have, that it would be nice if an actual expert in the area answered the question. Which brings it back to Google's censorship of questions about itself. It's OK for people to research things like PHP and rewriting, even if they have no expertise in them -- but Google itself is too sensitive a topic?

Oh, but remember, people can write to Google Support to get real expert advice. You mean like I did when Gmail went down for me last week? You mean like Tom Foremski did over at Silicon Valley Watcher when Gmail went down for him yesterday? I don't think he got a response. I know I didn't -- and this is now a week after I had my problem.

Instead, I hunted and hunted through support areas and eventually guessed that a solution for an entirely different problem might work for me. It did. But go read my Getting Gmail To Resume POP Access With Captcha Unlock article, because it explains just how lame the Gmail support documentation is in terms of helping people with this problem. And yet, that's what Google Answers thinks is fine for people to use instead of being able to ask questions?

And as for company questions, while asking about Google is off limits, these are fine:

Apparently, having researchers answer questions about other companies without inside information is OK. It's only Google itself that needs special protection.

What do you think? Should the policy change? I'm going to ask in two places and will postscript links here. The first will be our Search Engine Watch Forums. The second will be Yahoo Answers, where there are no restrictions about asking about Google -- or Yahoo -- that I can see. Perhaps that's one of the reasons it's growing by leaps-and-bounds, as covered in my recent article, Look Out Wikipedia, Here Comes Yahoo Answers!

Want to comment? Come join:

Posted by Danny Sullivan at 8:15 AM | Permalink

June 20, 2006

Google Pulls Question About Google From Google Answers

Peter Da Vanzo reports on an individual who posted a question on Google Answers that was removed by Google. The question was, "What percentage of Google searches are contextual?" Specifically, the person wanted to know what percentage of Google searches give back results based on the content of a page someone is reading.

You can see the thread title in the cache or via this image capture, at this moment in time, where the poster was willing to pay $20 for the answer. A Google editor removed the question, stating:

Hello hallsteve11-ga,

Thank you for your question ID 739118, titled "Percentage of Google searches that are "contextual"." We've removed your question because you can find the answer on our main site, free of charge. All publicly available information about Google is available at: http://www.google.com/about.html.

For additional questions about Google, please visit: http://www.google.com/support

Thank you for your interest in Google Answers. Please visit us again.

Sincerely,

The Google Answers Team

Is this a case of Google censoring a question that they don't feel comfortable being answered?

Postscript From Danny: I don't think there's anything "uncomfortable" about the question, especially since Google doesn't really contextually create search results anywhere. It does put ads on pages that are based by analyzing the content/context of those pages -- but those ads are not searches. Still, pulling the question makes no sense. If someone wants to ask a question on the service, let them answer it. And pointing at the Google About page doesn't answer the question at all!

Posted by Barry Schwartz at 8:39 AM | Permalink

June 15, 2006

MSN's Hand Crafted Results (Fake? - Shame On Me!)

"MSN Hiring People to Hand Code SERPS" at SEO Blackhat is a nice catch from the MSN Search jobs page talking about needing people to help hand-craft results. Philipp Lenssen at Google Blogoscoped reacts with "Oh my." I react with "Hallelujah."

Note: As Threadwatch spots in comments, this page looks like a joke that MSN is hosting. Shame on me for not reading more closely -- type 150 words per minute! The page IS on the real MSN Search domain, but it's not linked from the real jobs area [OK, Pip at Google Blogoscoped found it connected from the jobs page]. Anyway, I'll drop a note and get confirmation. And the points below -- still valid :)

Let's look at the job post first:

When all else fails, and the ranking algorithms do not pass the confidence threshold, we fall back to delivering handcrafted results. Working on a team of approximately 132 other handcrafters in 26 worldwide markets, you will receive a user query, use all the available search engines to quickly scour the web for results, pick the top 10 results for this query, and send it on to the user. Successful handcrafters can typically find top 10 results for a real-time users query in less than 3.8 seconds. This is an opportunity to truly connect with customers, because the queries that get routed to you are precisely the ones that the engine cannot answer well. We will have adequate staffing to allow generous coffee and bathroom breaks. If you are an expert at using at least 3 different search engines, well versed with American English/colloquial usage, and can type at > 149 words/minute as measured by the Simia-Lico method come join us and delight users real-time!

I agree. Search engine algorithms are not perfect. I'm tired of seeing bad listings make it into the top results that any human reviewer would nix. The Google mantra has always been that they prefer to tailor their computer algorithms to figure out how a human would see and rate things and then get the algorithm to do the right thing. We've had that mantra for years. And yes, generally the algorithms do the right things. Still stuff gets through. So kill off the bad stuff with a human and sure, insert a good quality page you know you are missing.

As a reminder, MSN used to have human editors, as I've written before. That was actually one reason why years ago, they compared pretty well when we would do relevancy tests on popular queries. They had a very sophisticated editing suite that allowed a team of editors to constantly review -- AND FIX -- bad results.

Now I can buy into the "Oh My" idea if MSN is returning to hand crafted results because their automated technology is so bad they've got to fall back on humans. No, that's not good. But if it's to complement and better tune what the automation does? Bring it on. If you want more on the how and why this can help, see my past post, More On Query Refinement, The Human Scale Problem & Creating The Search Dialog.

I also have the "Oh My" reaction if hand handcrafting involves payment. This year, I've had one serious allegation that MSN has rigged one set of its results to favor a top advertiser. I just had another serious allegation like that levied against Yahoo. In the MSN case, the difficulty in pursuing the allegation is deciding whether they are true or an attempt to knock out a competitor that might be ranking well. In the Yahoo case, I'm awaiting that tipster to send me more information beyond the quick eye opening stuff I was shown at our recent London conference.

Yahoo, of course, does hand manipulate already, to my belief (I'm not saying for payment -- only that for whatever reason, they seem to hand craft some results). I wrote about this in 2004 but never got an answer about it from Yahoo, nor did I get an answer when I followed up at least one other time. It also came up on our forums last year and at here at Search Engine Roundtable.

Google has long denied "hand jobs," as wizened search marketers call them. Setting aside censorship cases, I believe that. I've never seen any solid evidence of results being hand selected by Google (and the quality raters we're written about before have not been shown to be manipulating results).

In fact, Google used to trumpet that it had no hand manipulation. That was true in crafting results, but it wasn't true in terms of removing them. As I wrote in 2004:

Of course, Google does indeed intervene manually in search results. It removes material because it may be deemed illegal, as was the case in the infamous chester guide search. The company also removes material in response to DMCA complaints and also because for spamming reasons, as this article explains further.

Such interventions make some marketers confused (or even livid) when they read Google's oft-repeated claims of no hand manipulation of search results. To them, such removals as I've described above are hand manipulation. You can get a flavor of such confusion in this recent WebmasterWorld forum thread.

These interventions are not specifically rank related. When they happen, Google doesn't try to reorder the ranking of how a page appears. Instead, it simply pulls the page from the index entirely. And if you aren't in the index, you naturally no longer rank number one. But to save confusion, it might be better for Google to be clearer in saying that they don't chose by hand which sites rank well.

By the way, I asked Google previously about the reference in a Wired article about wanting to "attach" better sites to queries to ensure it had good information available. I remember being disturbed by this, just as some in the aforementioned thread were, as it indeed suggested that Google was doing hand-ranking in some cases.

I was told by Google that this was a misinterpretation on the part of Wired. The Google engineer apparently meant that the Google search algorithm would be tweaked to produce better results, not that the results would be reordered by hand.

Overall, I'm fine with hand-crafting, hand manipulation, hand jobs or whatever you want to call it as long as:

  • It improves search quality
  • It's not done to favor an advertiser by rigging the editorial results

Posted by Danny Sullivan at 10:16 AM | Permalink

June 7, 2006

Brin Suggests Google Might Reverse Chinese Censorship In The Short Term; Meanwhile, China Ramps Up Google.com Blocks

"Brin says Google compromised principles" from the Associated Press covers Google cofounder Sergey Brin telling reporters yesterday that it's possible Google might reverse its policy of censoring on behalf of the Chinese government. That's the real news from his talk -- a possible reversal, perhaps soon -- not the admission of compromise which Google's made before. The news comes on the heels of China apparently ramping up blocks on the uncensored Google.com site.

Back in January, Brin already said the move wasn't "to the 100 percent that we ideally would like." That same month, Google CEO Eric Schmidt said that Google had created an "evil scale" to determine just how much a compromise of the company's "Don't Be Evil" mantra the China move would be. In February, Google communications chief Elliot Schrage explained to a US Congressional committee how in 2002, Google refused to compromise its principles:

We faced a choice at that point: hold fast to our commitment to free speech (and risk a long-term cut-off from our Chinese users), or compromise our principles by entering the Chinese market directly and subjecting ourselves to Chinese laws and regulations. We stood by our principles, which turned out to be a good choice, as access to Google.com was largely restored within about two weeks.

But then he also explained that compromise in 2006 was deemed necessary:

Our hope is that our mix of measures, though far from our ideal, would accomplish more for Chinese citizens access to information than the alternative. We dont pretend that this is the single right answer to the dilemma faced by information companies in China, but rather a reasonable approach that seems likely to bring our users greater access to more information than any other search engine in China. And by serving our users better, we hope it will be good for our business, too, over the long run.

So fast forward to yesterday and Brin's statement:

"We felt that perhaps we could compromise our principles but provide ultimately more information for the Chinese and be a more effective service and perhaps make more of a difference," Brin said.

Frankly, it's really nothing they haven't said before, albeit more directly to have "compromise" and "principles" coming up so directly and so close together. Far more interesting was this statement:

"Perhaps now the principled approach makes more sense," Brin said.

Change directions? That itself isn't necessarily news. Google's already said before they might reverse course. Going back to Schrage's statement from February:

Looking ahead, we will carefully monitor conditions in China, including new laws and other restrictions on our services. If we determine that we are unable to achieve the objectives Ive outlined above, we will not hesitate to reconsider our approach to China.

That statement sounds pretty long-term -- watch and see over time. Brin's statement sounds more immediate, as if suddenly things have changed that might make Google rethink its approach in the near term. What could those things be?

For one, the China issue isn't going away. Google continues to take flak for it, both outside China by those opposed to the move and even those inside China by those upset with the disclosure route they've followed.

More recently, just this week, journalists in the UK started organizing to boycott Yahoo over its actions in China. Google, which has done far less egregious things than Yahoo is accused of, could still come under fire for a similar move.

Also this week, reports have started coming in that Google.com is being more heavily blocked from China. China 'blocks' main Google site from the BBC (and see also Reuters) has more on this.

Google.com has never worked perfectly within China. That, after all, is the reason why Google caved to create an approved Chinese edition. But Google.com has remained held out by the company as a way for those in China to still get an uncensored look at search results (at least uncensored according to Chinese laws; US laws still get a far more limited amount of material removed).

If Google.com is now being more aggressively blocked -- not just occasionally for particular sensitive queries but in a heavy-handed fashion -- Google loses another relief valve for those upset with its actions.

It's also important to note that one of the ways Google decides what to censor is to look at the differences between Google.com and Google China from within China, as the New York Times explained in an excellent story back in April. If Google.com is being blocked more heavily, then it's harder for Google to maintain its censored version.

Then again, perhaps a reversal isn't really likely to happen soon. Later in his talk with reporters, Brin said:

"It's perfectly reasonable to do something different, to say, 'Look, we're going to stand by the principle against censorship and we won't actually operate there.' That's an alternate path," Brin said. "It's not where we chose to go right now, but I can sort of see how people came to different conclusions about doing the right thing."

The "not where we chose to go right now" sort of defused the idea that dropping censorship might be a short term thing.

It's also interesting that only two months ago, Google CEO Eric Schmidt talked about the Chinese action as "absolutely the right move." I didn't agree with that, and now we've got one of the cofounders clarifying it wasn't absolutely the right choice but rather simply one of at least two directions Google decided to go.

If you're interested in more about these issues and past developments, be sure to check out the Legal: Censorship category we maintain for Search Engine Watch members.

Want to comment or discuss? Visit our Search Engine Watch Forums thread, Brin says Google compromised principles.

Postscript: Google.cn: more 'Google Speak' at ZDNet from Donna Bogatin notes that only last week, Google CEO Eric Schmidt said he doesn't "see a need to change what we have done." So a split among the Google leadership? Or a recent change in China making them all rethink things?