- Sponsored by:
June 23, 2008
Google Trends for Websites Adds to Comparison Sites ConfusionIf you want to see how your site is doing compared to the competition, then the new Google Trends for Websites is yet another option to conduct your research. Following the lead of sites like Compete.com and Alexa, Google Trends for Websites allows you to see a graph of traffic to the sites you designate.
But Google Trends for Websites is only adding to the confusion caused by Compete and Alexa. I used all three to compare SearchEngineWatch.com, SearchEngineLand.com, and SEOmoz.org, three of the leading search blogs. I got three very different graphs. Check out these screenshots.
With all three, there are definite seasonal dips. But these graphs may speak more to the popularity of Google, Alexa and Compete than they do of the websites you may search. Alexa makes the sites look like they've seen traffic decline, and Compete makes the sites look like the traffic has increased, beginning with a big jump last June. Incidentally, almost any site related to internet marketing seems to have a big jump last June on Compete, which was discussed at search marketing conferences and made known to a wider audience.
Furthermore, Google Trends for Websites does not offer numerical values to give you a ballpark figure of how a site is performing. Alexa and Compete do. Still, Google Trends will likely become the most authoritative source for comparison data since Google has access to far more data than Compete or Alexa.
Posted by Nathania Johnson at 9:14 AM | Permalink | Comments (5)
December 18, 2006
Keyword Research Best PracticesBarry pointed out a great response to thread at Search Engine Watch forums by Paid Search Guru Ian McAnerin. A member had asked Forum visitors which industries they "would not touch with a 10 foot SEO pole?"
Ian answers led to some additional excellent discussion at SEW and a couple of gems in the SER comments.
Keyword research is a topic that is considered to be very basic by many in the SEO and Paid Search fields. This is likely due to the fact that it has been one of the few constants since the early days of SEO, when tools began to appear that were geared towards finding the right keywords. Since, many writing about the subject have indicated the same core needs: relevancy and popularity, including Danny from way back when, Kevin Lee, Shari Thurow, and most recently Christine Churchill.
Ian's post at Search Engine Watch makes three main points: First that some industries may be too difficult to venture into without specialized experience; secondly, you may not want to venture into some industries due to business concerns (he cites Realtors as being especially “difficult” when it comes to payment or buying in to the value); and lastly that your personal belief set may be in conflict with the particular industry, such as Hate or Porn sites, for example.
Ian comment raised some good follow up questions, and he defends his opinion that one should “cut their teeth” by targeting more localized terms. The whole topic leads well to a discussion of the core competency of keyword research. When venturing into a new space, it is likely that many SEO’s are at a slight disadvantage due to being unfamiliar with terms. When deciding on whether to accept a project, it actually takes a fair amount of diligence on the part of any SEO; otherwise they may be simply saying “sure we’ll get you ranked.” This could be an alarm signal.
Using geo-modified keywords as the target can also prove to be difficult if not properly done. In some cases, there may be a majority of searchers using the city or town before the more general term (i.e.: Timbuktu hotel) while in others, people may use it more often after the term. The fact is that without excellent and trusted keyword research, only trial and error will lead to the required log files that report the actual activity. This trial and error period can be greatly eased by having an unlimited paid search budget to run all keywords on broad match across all engines for at least 2 or three months. Unfortunately not everyone has the budget to do that. However, running these types of campaigns on a local basis may be somewhat helpful.
One comment that was very insightful at the SER blog was that “generally the most competitive websites have the highest cost in PPC advertising. Find the biggest spenders and you have the stiffest competition.” Although this is a generalization, it holds fairly true. When making a decision as to whether to venture into an industry for SEO, a quick check of the results pages for Paid Search listings can save a good amount of time for small SEO/SEM shops.
Posted by Chris Boggs at 9:09 AM | Permalink
November 8, 2006
In The Election Results Race, Yahoo's The Winner
Like many Americans, I wanted to know what was up with the vote in the US midterm elections this morning. As a search analyst, I then wanted to know how the search engines performed in helping me find out. The results are in! Yahoo's the winner by far, but I'd still take the New York Times over it. Come along for an illustrated tour.
Google told us last month that Google Earth was all geared up to be an election guide. That's great if you've downloaded Google Earth and wanted to learn more before the election. But how about a quick, fast summary of what happened yesterday? What's Google got for us?
The Google home page is as minimal as always, no help there -- not even a special logo as in the 2004 race.
How about a search for "election results," which I think is a fair query to try. After all, using Google Trends, I can see a huge spike for that term after the last elections in the US:
I also checked the volume for just "results," and that was even higher whereas "elections" was much lower (see them all compared here). So my two queries for this test were "election results" and "results." On Google, both disappointed.
Here are election results on Google:
CNN's top with 2004 results! I know -- web search is always behind the times. That's why Google inserts that big news results OneBox unit above the regular results. Let's click on the main news link there, which takes us to news results:
Pretty bad. News about the dollar, stock prices -- but who won?!!! I've got to really work to figure this out, especially compared to the New York Times, as I'll show at the end of this story.
Maybe I head to the actual Google News home page:
Nope. I get some headlines telling me about the Democratic house victory, but it could be much better.
FYI, checking on a search for just results, I don't even get the news headlines inserted. Overall, I found Google to be a pretty poor resource.
How about Yahoo? The home page there immediately gives you some news:
If I actually gravitate to the picture and resist the pull of the search box, the "Full elections coverage" link takes me to a Full Coverage page with lots of info, including an interactive results page (my link takes that out of the normal pop-up box, but it still works great):
This is very, very nice. I can see at a glance who is ahead in the race for control of the US House Of Representatives, plus with a click I can check out the Senate or governor races. Selecting any state also gives me the information about races with that state.
I love this. It very similar to what impressed me at the New York Times. I hope Yahoo searchers found it. However, I suspect many bypassed it. To understand why, let's do that search for election results:
Similar to Google, Yahoo inserts a big "News Results" shortcut unit above the regular results, to help detour searchers into the freshest results. Of course, searcers might bypass that. If so, unlike Google, Yahoo has managed to get the CNN 2006 results page up rather than the CNN 2004 page. Nice. After that, there's Fox News 2006. But c'mon -- Yahoo's own special election results are third. This is one case where I'd totally applaud a little hand manipulation to get that to the top, especially to highlight that interactive results summary page.
Still, the web search results for this particular day at Yahoo far outshine Google. That's almost certainly due to some human editing, which is fine. Along with the sites I've mentioned, you get the New York Times politics page, USA Today's politics page, C-SPAN's 2006 results page, the ABC News politics page, CBS News's 2006 page, politics from the LA Times, then the Washington Post's 2006 results page. All of these are excellent choices. If Yahoo did human intervention to make this happen, kudos to them. You can check out a snapshot of the entire page here.
Over at Google, nothing is either timely or general enough. The Virginia state election board, California election info, assorted things dating from 2004 -- then oddly Virginia and California get another bump for their 2006 pages. Ugh. See the entire list in the snapshot here.
What happens if we detour into the news area that Yahoo promotes at the top of the page? Disappointment:
Yes, relevant news stories. And the image results to the side are kind of fun. But some hand help could have made a difference. How about a promo for that awesome election map of Yahoo's?
Let's go over to Ask, where I had high hopes. Ask has made a big deal of its special Smart Answers for the election, and they are cool. But will I see them? Yes, if I search for election:
I'd also get to this box if I went to the Ask home page and clicked on the Election Day link there:
But for election results (what I believe to be the more popular query), all I get is a small news unit:
The news unit will take me over to some news results, but like Yahoo's, these aren't thrilling. It's pick and choose through what you want, rather than any type of easy overview. As for a search on just results, that doesn't even bring back the news unit at all.
The overall web search results, similar to Google, are underwhelming. Nothing really helpful for the 2006 results pops up (see the full results in the snapshot here).
Even the special Smart Answers box, had it shown up, isn't that helpful for what I want now -- RESULTS! None of the featured links with it takes me to results.
Microsoft, what have you got for me at Windows Live Search? On the home page, nothing. For search on election results, it's disappointing old or non-targeted results (screenshot here). Unlike the others, there are no news results inserted above these. A search for just results is no better. If I specifically try a news search for election results, as with the others, there's no attempt to get me a comprehensive overview. It's up to me to review each story and hope for a good match.
Ironically, at the largely overshadowed MSN site, similar to Yahoo, I get a big election photo on the home page along with links, including one called "state-by-state results" that leads to MSNBC here. And over there is a pretty neat "Democracy Dashboard" giving me that type of overview I wanted:
It's a pity Windows Live didn't reach out to either MSN or MSNBC and do something special to point to this or somehow integrate it into the results.
What about AOL? From the home page, it's pretty easy to spot a link to a AOL election page with results for the House, Senate and more:
Searching for election results brings back disappointing Google listings in the main results. However, the new FullView column does a good job of dividing news into elections overall, US Senate coverage, US House coverage and more. And clicking on any of the "View all" links brings up the special AOL election page (see the full page here).
Now to the New York Times. I headed over there pretty much by chance. There are any number of newspapers I might of thought of off the top of my head, and usually its my original home town paper of the Los Angeles Times. But I hit the NY Times today, and boy was I glad.
Right on the home page, above the "fold" is an easy-to-spot election map. In seconds, it organized the most important information I was looking for into a way for me to know what was going on:
Drilling into the full map was even better. There, I could click on any state -- in particular the undecided ones -- and see the current situation:
Just when I was thinking "what if," I saw the "Create Outcomes" tab where I could click on a state and flip it to the Democrats or the Republicans to see how it might go with the Senate. Outstanding!
Other newspapers or web sites might have done as well with similar displays. If so, my apologies that this wasn't a review of the best election results sites. Instead, it was really meant to see how well the search engines held up as information resources for this particular news event.
Overall, I've written many times before that there's a role humans can play in search results. Today -- this was a perfect example of that. Yahoo almost certainly put some human effort into crafting results, and it was the clear victor in terms of quality of what was coming up in web search listings. AOL comes in second, again where human effort has helped its FullView listings help make up for the poor crawler-based results from Google.
In third, I put Google and Ask. Google's results were poor, but at least it floated some news results that may have helped. Ask, I was rooting for. But that Smart Answers box simply wasn't showing up for the queries I thought people were doing. Even if people were getting it for "election," it wasn't helpful to get election results. I really appreciate the effort, and if this had been for something other than actual results, Ask would have been great. In last place -- Windows Live.
This campaign of sorts is also one of those classic "what if" races. With just a little more effort, Yahoo would have had a landslide victory by getting people to its great overview page. The same is true for AOL. Ask, with just a bit more thought, could have had that box coming up for "election results" rather than just "elections" and added some links to get people to actual results. Windows Live, if it had remembered its MSN origins, might not be in last. And Google? A company that's all about organizing information might not have put in such a poor performance if it used some human power in the way the New York Times did.
Postscript: See also Case Study: Digg Versus Google News Traffic from me on my personal blog that covers how this article ultimately brought in lots of traffic from Google News from those unable to find election results there, along with lots of other data and a comparison to traffic from a top story at Digg on the same day.
Posted by Danny Sullivan at 4:17 AM | Permalink
October 13, 2006
Consumer Satisfaction Doesn't Equal Market ShareEarlier this week J.D. Power and Associates released the findings of its "2006 Residential Online Service Customer Satisfaction Study." The study, based on a survey of more than 10,000 U.S. residential Internet users, found that Yahoo Messenger ranked the highest among IM clients and that – and this is something of a surprise – InfoSpace's Dogpile had the highest rankings among search engines.
Here are the rankings of IM clients in terms of customer satisfaction:
1. Yahoo! Messenger 2. MSN Messenger 3. Windows Messenger 4. Instant Message Average 5. Google Talk 6. Trillian 7. AIM/AOL Instant Messenger
Compare that to U.S. IM market share (per Nielsen//Netratings):
1. AOL 2. MSN 3. Yahoo 4. Google
The J.D. Power report also said that U.S. residential IM usage was flat vs. 2005, at 36%. Yet the survey found that "among customers who report using IM on a regular or occasional basis, nearly 70 percent report that to some degree, instant messaging has replaced the use of traditional telephones." There are implications here for traditional telephony that are striking and worth further exploration.
Probably more interesting to readers of this blog are the search-engine findings. The survey reported that 75% of residential Internet subscribers used multiple search engines.
Here's the market-share breakdown that J.D. Power found:
1. Google: 51% (up 8 points from 2005) 2. Yahoo: 17% (down 4 points) 3. AOL: 9% (down 1 point)
The release doesn't report on the respective shares of Ask or MSN/Windows Live. Presumably they constitute the remaining 23% of usage or something approaching that.
Compare comScore August search market share data:
1. Google Sites: 44% 2. Yahoo: 28.7% 3. MSN: 12.5% 4. AOL: 5.6% 5. Ask: 5.5%
Here are the J.D. Power survey's customer satisfaction findings. Little used Dogpile was ranked number one:
1. Dogpile 2. Ask.com 3. Google 4. Yahoo! Search 5. AOL Search 6. MSN Search 7. Internet Explorer (treated as a search engine in the survey)
I don't have any insight into the survey methodology so we have to take the results at face value. But 10,000+ respondents is a very large sample. A disconnect is the difference between search engine market share and the satisfaction ratings. Based on these findings one would think that if Ask and (especially) Dogpile could gain broader awareness and visibility they might be able to gain some share.
Posted by Greg Sterling at 11:19 AM | Permalink
June 12, 2006
'Search: Thy Name Is Google'Last week, MediaPost reported on an Outsell report, "How Information Providers Can Keep Pace With User Demands For Time-Saving Solutions," which showed Google increasing its market share dominance but also reportedly increasing dissatisfaction with the quality of search results. To get a better look at these provocative findings, we obtained a copy of the report.
The analysis compared surveys conducted in early 2005 and 2006 among U.S. "knowledge workers" in four sectors, Corporate, Government, Health Care and Academia. It found increasing amounts of time spent on "information tasks" (gathering and analyzing information). The two primary culprits were the increasing proliferation of information and increasing inefficiency in information retrieval in particular of web search.
The report cites two statistics to draw the inference that there's increasing dissatisfaction with web search. First it cites an increase in the use of corporate intranets as a starting point for information (up from 13% to 19%). It also cites "search failure rates" (defined by users as unsuccessful searches for desired information) as a serious problem. Failure rates across the four categories were 31% on average. This is indeed striking.
Unfortunately, the report neglects to further drill down and discuss whether and how these failure rates are specifically affecting knowledge workers' behavior now and attitudes toward usage of search engines in the future. (It does reflect usage of a range of information resources, which are basically flat across the two surveys, except for the rise in intranet usage).
Also striking was the market share data reflected in the report. The sample size was 5,740 respondents in the early 2006 survey. And while these "knowledge workers" are not perfectly representative of the general U.S. population the sample size is large enough to merit some generalizations to that larger population.
Danny last week compared the market share numbers for April from comScore, Nielsen and Hitwise. Of the three, Hitwise had Google with the largest market share at 59%. Outsell's report shows and even larger margin (again, not necessarily representative of the population as a whole). But the Outsell distribution is as follows:
Google: 74% Yahoo: 15% MSN: 7% Other: 4%
During the survey interval Google gained share at the expense of all its competitors.
Another finding of the report is that wireless data usage has now reached "critical mass," with almost 70% of users accessing some sort of content over wireless devices.
In the MediaPost article Outsell analyst Roger Strouse muses on the reasons for failed searches:
"[A] variety of factors are to blame for search failure, including deliberate manipulation of Google's semantic system: 'From early on there's been a problem with what people used to call spamdexing, where people would apply tags to their Web sites that actually had no relation to the content at all. And that problem is still there, and is definitely going to have to be resolved some time soon.'"
Right now there are few effective alternatives to search engines for consumers as a starting point for research when they don't know where to find information. One quick and facile argument is that vertical search threatens Google et al. However I would argue that the relationship between vertical and general search is complex and the emergence of vertical search doesn't negate general search usage in those categories automatically.
The report's findings are interesting and certainly point out that everyone can do better in more efficiently providing relevant results to users. However, the report confirms that until something better and more relevant comes along, for now "Search, They Name Is Google."
Posted by Greg Sterling at 12:42 PM | Permalink
'Search: Thy Name Is Google'Last week, MediaPost reported on an Outsell report, "How Information Providers Can Keep Pace With User Demands For Time-Saving Solutions," which showed Google increasing its market share dominance but also reportedly increasing dissatisfaction with the quality of search results. To get a better look at these provocative findings, we obtained a copy of the report.
The analysis compared surveys conducted in early 2005 and 2006 among U.S. "knowledge workers" in four sectors, Corporate, Government, Health Care and Academia. It found increasing amounts of time spent on "information tasks" (gathering and analyzing information). The two primary culprits were the increasing proliferation of information and increasing inefficiency in information retrieval in particular of web search.
The report cites two statistics to draw the inference that there's increasing dissatisfaction with web search. First it cites an increase in the use of corporate intranets as a starting point for information (up from 13% to 19%). It also cites "search failure rates" (defined by users as unsuccessful searches for desired information) as a serious problem. Failure rates across the four categories were 31% on average. This is indeed striking.
Unfortunately, the report neglects to further drill down and discuss whether and how these failure rates are specifically affecting knowledge workers' behavior now and attitudes toward usage of search engines in the future. (It does reflect usage of a range of information resources, which are basically flat across the two surveys, except for the rise in intranet usage).
Also striking was the market share data reflected in the report. The sample size was 5,740 respondents in the early 2006 survey. And while these "knowledge workers" are not perfectly representative of the general U.S. population the sample size is large enough to merit some generalizations to that larger population.
Danny last week compared the market share numbers for April from comScore, Nielsen and Hitwise. Of the three, Hitwise had Google with the largest market share at 59%. Outsell's report shows and even larger margin (again, not necessarily representative of the population as a whole). But the Outsell distribution is as follows:
Google: 74% Yahoo: 15% MSN: 7% Other: 4%
During the survey interval Google gained share at the expense of all its competitors.
Another finding of the report is that wireless data usage has now reached "critical mass," with almost 70% of users accessing some sort of content over wireless devices.
In the MediaPost article Outsell analyst Roger Strouse muses on the reasons for failed searches:
"[A] variety of factors are to blame for search failure, including deliberate manipulation of Google's semantic system: 'From early on there's been a problem with what people used to call spamdexing, where people would apply tags to their Web sites that actually had no relation to the content at all. And that problem is still there, and is definitely going to have to be resolved some time soon.'"
Right now there are few effective alternatives to search engines for consumers as a starting point for research when they don't know where to find information. One quick and facile argument is that vertical search threatens Google et al. However I would argue that the relationship between vertical and general search is complex and the emergence of vertical search doesn't negate general search usage in those categories automatically.
The report's findings are interesting and certainly point out that everyone can do better in more efficiently providing relevant results to users. However, the report confirms that until something better and more relevant comes along, for now "Search, They Name Is Google."
Posted by Kevin Heisler at 12:42 PM | Permalink
'Search: Thy Name Is Google'Last week, MediaPost reported on an Outsell report, "How Information Providers Can Keep Pace With User Demands For Time-Saving Solutions," which showed Google increasing its market share dominance but also reportedly increasing dissatisfaction with the quality of search results. To get a better look at these provocative findings, we obtained a copy of the report.
The analysis compared surveys conducted in early 2005 and 2006 among U.S. "knowledge workers" in four sectors, Corporate, Government, Health Care and Academia. It found increasing amounts of time spent on "information tasks" (gathering and analyzing information). The two primary culprits were the increasing proliferation of information and increasing inefficiency in information retrieval in particular of web search.
The report cites two statistics to draw the inference that there's increasing dissatisfaction with web search. First it cites an increase in the use of corporate intranets as a starting point for information (up from 13% to 19%). It also cites "search failure rates" (defined by users as unsuccessful searches for desired information) as a serious problem. Failure rates across the four categories were 31% on average. This is indeed striking.
Unfortunately, the report neglects to further drill down and discuss whether and how these failure rates are specifically affecting knowledge workers' behavior now and attitudes toward usage of search engines in the future. (It does reflect usage of a range of information resources, which are basically flat across the two surveys, except for the rise in intranet usage).
Also striking was the market share data reflected in the report. The sample size was 5,740 respondents in the early 2006 survey. And while these "knowledge workers" are not perfectly representative of the general U.S. population the sample size is large enough to merit some generalizations to that larger population.
Danny last week compared the market share numbers for April from comScore, Nielsen and Hitwise. Of the three, Hitwise had Google with the largest market share at 59%. Outsell's report shows and even larger margin (again, not necessarily representative of the population as a whole). But the Outsell distribution is as follows:
Google: 74% Yahoo: 15% MSN: 7% Other: 4%
During the survey interval Google gained share at the expense of all its competitors.
Another finding of the report is that wireless data usage has now reached "critical mass," with almost 70% of users accessing some sort of content over wireless devices.
In the MediaPost article Outsell analyst Roger Strouse muses on the reasons for failed searches:
"[A] variety of factors are to blame for search failure, including deliberate manipulation of Google's semantic system: 'From early on there's been a problem with what people used to call spamdexing, where people would apply tags to their Web sites that actually had no relation to the content at all. And that problem is still there, and is definitely going to have to be resolved some time soon.'"
Right now there are few effective alternatives to search engines for consumers as a starting point for research when they don't know where to find information. One quick and facile argument is that vertical search threatens Google et al. However I would argue that the relationship between vertical and general search is complex and the emergence of vertical search doesn't negate general search usage in those categories automatically.
The report's findings are interesting and certainly point out that everyone can do better in more efficiently providing relevant results to users. However, the report confirms that until something better and more relevant comes along, for now "Search, They Name Is Google."
Posted by Kevin Heisler at 12:42 PM | Permalink
'Search: Thy Name Is Google'Last week, MediaPost reported on an Outsell report, "How Information Providers Can Keep Pace With User Demands For Time-Saving Solutions," which showed Google increasing its market share dominance but also reportedly increasing dissatisfaction with the quality of search results. To get a better look at these provocative findings, we obtained a copy of the report.
The analysis compared surveys conducted in early 2005 and 2006 among U.S. "knowledge workers" in four sectors, Corporate, Government, Health Care and Academia. It found increasing amounts of time spent on "information tasks" (gathering and analyzing information). The two primary culprits were the increasing proliferation of information and increasing inefficiency in information retrieval in particular of web search.
The report cites two statistics to draw the inference that there's increasing dissatisfaction with web search. First it cites an increase in the use of corporate intranets as a starting point for information (up from 13% to 19%). It also cites "search failure rates" (defined by users as unsuccessful searches for desired information) as a serious problem. Failure rates across the four categories were 31% on average. This is indeed striking.
Unfortunately, the report neglects to further drill down and discuss whether and how these failure rates are specifically affecting knowledge workers' behavior now and attitudes toward usage of search engines in the future. (It does reflect usage of a range of information resources, which are basically flat across the two surveys, except for the rise in intranet usage).
Also striking was the market share data reflected in the report. The sample size was 5,740 respondents in the early 2006 survey. And while these "knowledge workers" are not perfectly representative of the general U.S. population the sample size is large enough to merit some generalizations to that larger population.
Danny last week compared the market share numbers for April from comScore, Nielsen and Hitwise. Of the three, Hitwise had Google with the largest market share at 59%. Outsell's report shows and even larger margin (again, not necessarily representative of the population as a whole). But the Outsell distribution is as follows:
Google: 74% Yahoo: 15% MSN: 7% Other: 4%
During the survey interval Google gained share at the expense of all its competitors.
Another finding of the report is that wireless data usage has now reached "critical mass," with almost 70% of users accessing some sort of content over wireless devices.
In the MediaPost article Outsell analyst Roger Strouse muses on the reasons for failed searches:
"[A] variety of factors are to blame for search failure, including deliberate manipulation of Google's semantic system: 'From early on there's been a problem with what people used to call spamdexing, where people would apply tags to their Web sites that actually had no relation to the content at all. And that problem is still there, and is definitely going to have to be resolved some time soon.'"
Right now there are few effective alternatives to search engines for consumers as a starting point for research when they don't know where to find information. One quick and facile argument is that vertical search threatens Google et al. However I would argue that the relationship between vertical and general search is complex and the emergence of vertical search doesn't negate general search usage in those categories automatically.
The report's findings are interesting and certainly point out that everyone can do better in more efficiently providing relevant results to users. However, the report confirms that until something better and more relevant comes along, for now "Search, They Name Is Google."
Posted by Kevin Heisler at 12:42 PM | Permalink
May 2, 2006
MSN Bests Google & Yahoo in Search ShootoutInterlink, a Cincinnati web marketing research, search engine optimization and consulting firm, recently ran some tests to assess performance of major (and minor) search engines. Surprisingly, MSN took top honors in relevancy.
Like the search engine relevance tests we've done from time to time here at SEW, this isn't a controlled experiment, but rather a test of how well search engines return "reasonable" results. As such, this study should be regarded as interesting but not definitive.
Over at the 360, Susan Kuchinskas reports:
MSN bested the top two search providers, along with Ask, AOL, Gigablast and Wisenut, when all five factors were considered: relevancy, freshness of content, failure rate, difficult search results, and non-organic or extra features. Difficult searches were queries such as car dealer fargo north dakota and appliance repair des moines.More information and full results of the study are available here.
Posted by Chris Sherman at 4:43 PM | Permalink
MSN Bests Google & Yahoo in Search ShootoutInterlink, a Cincinnati web marketing research, search engine optimization and consulting firm, recently ran some tests to assess performance of major (and minor) search engines. Surprisingly, MSN took top honors in relevancy.
Like the search engine relevance tests we've done from time to time here at SEW, this isn't a controlled experiment, but rather a test of how well search engines return "reasonable" results. As such, this study should be regarded as interesting but not definitive.
Over at the 360, Susan Kuchinskas reports:
MSN bested the top two search providers, along with Ask, AOL, Gigablast and Wisenut, when all five factors were considered: relevancy, freshness of content, failure rate, difficult search results, and non-organic or extra features. Difficult searches were queries such as car dealer fargo north dakota and appliance repair des moines.More information and full results of the study are available here.
Posted by Kevin Heisler at 4:43 PM | Permalink
MSN Bests Google & Yahoo in Search ShootoutInterlink, a Cincinnati web marketing research, search engine optimization and consulting firm, recently ran some tests to assess performance of major (and minor) search engines. Surprisingly, MSN took top honors in relevancy.
Like the search engine relevance tests we've done from time to time here at SEW, this isn't a controlled experiment, but rather a test of how well search engines return "reasonable" results. As such, this study should be regarded as interesting but not definitive.
Over at the 360, Susan Kuchinskas reports:
MSN bested the top two search providers, along with Ask, AOL, Gigablast and Wisenut, when all five factors were considered: relevancy, freshness of content, failure rate, difficult search results, and non-organic or extra features. Difficult searches were queries such as car dealer fargo north dakota and appliance repair des moines.More information and full results of the study are available here.
Posted by Kevin Heisler at 4:43 PM | Permalink
MSN Bests Google & Yahoo in Search ShootoutInterlink, a Cincinnati web marketing research, search engine optimization and consulting firm, recently ran some tests to assess performance of major (and minor) search engines. Surprisingly, MSN took top honors in relevancy.
Like the search engine relevance tests we've done from time to time here at SEW, this isn't a controlled experiment, but rather a test of how well search engines return "reasonable" results. As such, this study should be regarded as interesting but not definitive.
Over at the 360, Susan Kuchinskas reports:
MSN bested the top two search providers, along with Ask, AOL, Gigablast and Wisenut, when all five factors were considered: relevancy, freshness of content, failure rate, difficult search results, and non-organic or extra features. Difficult searches were queries such as car dealer fargo north dakota and appliance repair des moines.More information and full results of the study are available here.
Posted by Kevin Heisler at 4:43 PM | Permalink
April 26, 2006
Is Google Search More Relevant Or Is There A Brand Factor?The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?
An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.
Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.
Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.
So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.
The bottom-line is that relevancy is an incredibly hard factor to measure.
Posted by Barry Schwartz at 12:26 PM | Permalink
Is Google Search More Relevant Or Is There A Brand Factor?The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?
An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.
Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.
Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.
So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.
The bottom-line is that relevancy is an incredibly hard factor to measure.
Posted by Kevin Heisler at 12:26 PM | Permalink
Is Google Search More Relevant Or Is There A Brand Factor?The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?
An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.
Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.
Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.
So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.
The bottom-line is that relevancy is an incredibly hard factor to measure.
Posted by Kevin Heisler at 12:26 PM | Permalink
Is Google Search More Relevant Or Is There A Brand Factor?The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?
An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.
Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.
Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.
So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.
The bottom-line is that relevancy is an incredibly hard factor to measure.
Posted by Kevin Heisler at 12:26 PM | Permalink
Search Stats Don't Have to be BoringThere's no shortage of data related to search marketing, with everything from market share numbers to frequency of search terms analyzed by market research groups. Some find all this data deathly boring—but it doesn't have to be. In fact, with the appropriate presentation style, search data can be downright entertaining, as it was at a recent Search Engine Strategies panel. Christine Churchill has the rundown on the panel in today's SearchDay article, Searchonomics: Search Statistics Made Fun.
Posted by Chris Sherman at 10:27 AM | Permalink
Search Stats Don't Have to be BoringThere's no shortage of data related to search marketing, with everything from market share numbers to frequency of search terms analyzed by market research groups. Some find all this data deathly boring—but it doesn't have to be. In fact, with the appropriate presentation style, search data can be downright entertaining, as it was at a recent Search Engine Strategies panel. Christine Churchill has the rundown on the panel in today's SearchDay article, Searchonomics: Search Statistics Made Fun.
Posted by Kevin Heisler at 10:27 AM | Permalink
Search Stats Don't Have to be BoringThere's no shortage of data related to search marketing, with everything from market share numbers to frequency of search terms analyzed by market research groups. Some find all this data deathly boring—but it doesn't have to be. In fact, with the appropriate presentation style, search data can be downright entertaining, as it was at a recent Search Engine Strategies panel. Christine Churchill has the rundown on the panel in today's SearchDay article, Searchonomics: Search Statistics Made Fun.
Posted by Kevin Heisler at 10:27 AM | Permalink
Search Stats Don't Have to be BoringThere's no shortage of data related to search marketing, with everything from market share numbers to frequency of search terms analyzed by market research groups. Some find all this data deathly boring—but it doesn't have to be. In fact, with the appropriate presentation style, search data can be downright entertaining, as it was at a recent Search Engine Strategies panel. Christine Churchill has the rundown on the panel in today's SearchDay article, Searchonomics: Search Statistics Made Fun.
Posted by Kevin Heisler at 10:27 AM | Permalink
April 18, 2006
comScore: Google Leads On March 2006 US Search ShareYou may have seen some rumblings a couple of weeks ago about Google being up in search traffic compared to competitors, based on comScore figures. I'm going to do a big drill down on that later this week, bringing in some NetRatings and Hitwise figures as well. But more comScore figures have just arrived for March, so here they are:
Share of Online Searches US Home, Work & University Users
Search Network March 2005 March 2006 Change Google 36.4% 42.7% +6.3% Yahoo 30.6% 28.0% -2.6% MSN/Microsoft 16.5% 13.2% -3.3% AOL/Time Warner 8.9% 7.6% -1.4% Ask 5.5% 5.9% +0.4%Other notes from the comScore release:
- Americans conducted 6.4 billion searches online in March, up 10 percent from last month and 15 percent from last year. The increase in search queries from the previous month marked the largest gain over the past twelve months.
- Google Sites led the pack with 2.7 billion search queries performed, followed by Yahoo Sites (1.8 billion), MSN-Microsoft (849 million), Time-Warner Network (486 million), and Ask Jeeves/Ask Network (376 million).
- The toolbar search market continues to be dominated by Google and Yahoo!, which combined for more than 95 percent of toolbar searches in March. Google led the way with 48.9 percent, while Yahoo! captured 46.5 percent.
Posted by Danny Sullivan at 10:31 AM | Permalink
comScore: Google Leads On March 2006 US Search ShareYou may have seen some rumblings a couple of weeks ago about Google being up in search traffic compared to competitors, based on comScore figures. I'm going to do a big drill down on that later this week, bringing in some NetRatings and Hitwise figures as well. But more comScore figures have just arrived for March, so here they are:
Share of Online Searches US Home, Work & University Users
Search Network March 2005 March 2006 Change Google 36.4% 42.7% +6.3% Yahoo 30.6% 28.0% -2.6% MSN/Microsoft 16.5% 13.2% -3.3% AOL/Time Warner 8.9% 7.6% -1.4% Ask 5.5% 5.9% +0.4%Other notes from the comScore release:
- Americans conducted 6.4 billion searches online in March, up 10 percent from last month and 15 percent from last year. The increase in search queries from the previous month marked the largest gain over the past twelve months.
- Google Sites led the pack with 2.7 billion search queries performed, followed by Yahoo Sites (1.8 billion), MSN-Microsoft (849 million), Time-Warner Network (486 million), and Ask Jeeves/Ask Network (376 million).
- The toolbar search market continues to be dominated by Google and Yahoo!, which combined for more than 95 percent of toolbar searches in March. Google led the way with 48.9 percent, while Yahoo! captured 46.5 percent.
Posted by Kevin Heisler at 10:31 AM | Permalink
comScore: Google Leads On March 2006 US Search ShareYou may have seen some rumblings a couple of weeks ago about Google being up in search traffic compared to competitors, based on comScore figures. I'm going to do a big drill down on that later this week, bringing in some NetRatings and Hitwise figures as well. But more comScore figures have just arrived for March, so here they are:
Share of Online Searches US Home, Work & University Users
Search Network March 2005 March 2006 Change Google 36.4% 42.7% +6.3% Yahoo 30.6% 28.0% -2.6% MSN/Microsoft 16.5% 13.2% -3.3% AOL/Time Warner 8.9% 7.6% -1.4% Ask 5.5% 5.9% +0.4%Other notes from the comScore release:
- Americans conducted 6.4 billion searches online in March, up 10 percent from last month and 15 percent from last year. The increase in search queries from the previous month marked the largest gain over the past twelve months.
- Google Sites led the pack with 2.7 billion search queries performed, followed by Yahoo Sites (1.8 billion), MSN-Microsoft (849 million), Time-Warner Network (486 million), and Ask Jeeves/Ask Network (376 million).
- The toolbar search market continues to be dominated by Google and Yahoo!, which combined for more than 95 percent of toolbar searches in March. Google led the way with 48.9 percent, while Yahoo! captured 46.5 percent.
Posted by Kevin Heisler at 10:31 AM | Permalink
comScore: Google Leads On March 2006 US Search ShareYou may have seen some rumblings a couple of weeks ago about Google being up in search traffic compared to competitors, based on comScore figures. I'm going to do a big drill down on that later this week, bringing in some NetRatings and Hitwise figures as well. But more comScore figures have just arrived for March, so here they are:
Share of Online Searches US Home, Work & University Users
Search Network March 2005 March 2006 Change Google 36.4% 42.7% +6.3% Yahoo 30.6% 28.0% -2.6% MSN/Microsoft 16.5% 13.2% -3.3% AOL/Time Warner 8.9% 7.6% -1.4% Ask 5.5% 5.9% +0.4%Other notes from the comScore release:
- Americans conducted 6.4 billion searches online in March, up 10 percent from last month and 15 percent from last year. The increase in search queries from the previous month marked the largest gain over the past twelve months.
- Google Sites led the pack with 2.7 billion search queries performed, followed by Yahoo Sites (1.8 billion), MSN-Microsoft (849 million), Time-Warner Network (486 million), and Ask Jeeves/Ask Network (376 million).
- The toolbar search market continues to be dominated by Google and Yahoo!, which combined for more than 95 percent of toolbar searches in March. Google led the way with 48.9 percent, while Yahoo! captured 46.5 percent.
Posted by Kevin Heisler at 10:31 AM | Permalink
April 4, 2006
The Thrill & Danger Of Measuring Relevancy Through Ego SearchesLast week, you may recall that Ask got a rave review from the Wall Street Journal. Robert Scoble saw that, then did an ego search for himself and decided Ask doesn't measure up based on that. Fair criticism? Sure, to some degree. But then again, it's easy to take a single search for anything and show that any of the "leaders" in search have problems, as well.
In Search Of The Relevancy Figure from me back in 2002 goes through the various ways that search engines are measured up. Ego Search is a long-standing one, as I described:
"Ego Search" is another style that can be bad, and one that I still see journalists and others often perform. In an ego search, you look for your name. If you fail to come up tops for it, you conclude the search engine's relevancy is poor.
In some cases, perhaps this is true. If I search for "bill gates," it's reasonable to expect to find the official web site for Microsoft Chairman Bill Gates. But what if you aren't as well known as Bill Gates or have a popular name? What if you've built a web site in some free hosting service that is shared by spammers? These might be issues that push you down, and for good reason. Moreover, are you're going to condemn an entire search engine as bad, based on one search? Well, I've seen it happen.
So is Ask bad because it didn't find Robert Scoble's current blog. Yes, for a variety of reasons:
- The new blog's been out there enough that I would expect it to be listed in the top results
- Others can get to it via his old blog, which is listed in the top results, but it would be better not to make that detour
- It matters to Robert
The last point is probably the most important. While ego searches can be bad, they remain one of the main ways anyone will test their favorite search engine. You've got to get them as right as possible, and that's not happening with Robert.
For the record, in my searching today, Google and Yahoo list both the old and new blog (in that order), while MSN gets the new one first, then the old one further down (though it's the opposite if you search for robert scoble rather than scoble)
Also for the record, Robert himself makes the job tough for any search engine. He does no redirection from his old to new site, which pretty much makes it impossible to do the very best thing, list only the new blog.
Think about it. If you search for Robert, why send you to his old blog, just to make you detour? I'm not saying pages from his own blog shouldn't come up for relevant specific queries. But in a search for him by name, you'd really only want to point people to his current blog.
On the flip side, as I wrote on Dave Naylor's blog recently, keeping both blogs going with no redirection is also smart marketing that ensures you dominate the results for your name :)
Back to the search, I also tried using his first and last name. Ask deserves some kudos on that, as robert scoble puts a picture of Robert at the top of the page, based on pulling material from Wikipedia. Nice! Sure, his new blog still isn't listed, but that doesn't mean some other features aren't helpful.
Earlier I said Ask is bad for failing the ego search test for Robert. But bad as in bad on that specific query, rather than bad overall. Condemning an entire search engine based on it going overboard, as Robert does when he says:
But, Walt, this doesn't portend good things for Ask.com. To be included in the top three you've gotta be as good as Google. Ask isn't even in the same neighborhood yet.
Wow. So all I've got to do is find a query where Ask succeeds and the others fail, and then Ask's in the neighborhood. Well, John Battelle helps out here. The Last Name Test post he just put up shows how he can easily find himself on Ask for his own name, just as he's found on Google, Yahoo and MSN.
Then again, that search also points out an Ask Jeeves failure. I remarked about this on our Daily SearchCast show when talking about Ask's rave review by the Wall Street Journal. Ask was having issues with not consolidating web sites correctly.
In particular, the day of our show, I'd done a search at Ask to find one of my articles. I queried invisible tabs, and I got this:
Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor December 2, 2003... searchenginewatch.com/searchday/article.php/3115131 Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor-In-Chief December 2, 2003... www.searchenginewatch.com/searchday/article.php/3115131That's the same page, listed twice. Ask is having a canonical issue, not realizing that the www URL leads to the same page without the www prefix (and this despite us doing redirection).
You can see the same issue with John's search for battelle:
John Battelle's Searchblog John Battelle's Searchblog Thoughts on the intersection of search, media, technology, and more. Search and Newsletter battellemedia.com/John Battelle's Searchblog About John Battelle. Federated Media. UC Berkeley. Columns. That Book I Hear He Wrote. Consulting. Contact... www.battellemedia.com/
All those listings are from the first page of Ask's results. The first and third listing are the same page, just one is shown with the www prefix.
So now we can say that Ask is clearly not in the same neighborhood as Google, Yahoo and MSN! Hold on there.
Let's go to MSN and look for invisible tabs. I'm not saying that my article should be the only thing listed, but look at the result at the bottom from MSN:
invisible girl ABchao Brad Bubs Connie Daynah Dodo Dodo's Tabs Heather Jen Jessie Kristin Leah Leigh Sanne Sarah Shiny Shiny Shopgirls! TechLog Meta </a> .org - asm > Valid XHTML XFN WordPress
In this case, I'm getting a page that has nothing to do with invisible tabs of any type other than they simply having both of those words on the page somewhere. But over at Ask, every listing in the first results is related in some way, either to my article, or to how they apply to a programming issue, or to the idea of guitar tabs you use to play the song Invisible. By this measure, MSN isn't in the neighborhood of Ask.
As for domain issues, MSN's fixed the embarrassing issue it had with every cars.com subdomain dominating (shall I say domainating) a search for cars, but then when I search for movies today, I get all these from the go.com domain:
- movies.go.com
- casanova.movies.go.com
- video.movies.go.com/sincity
None of these are "indented" results, either, so it doesn't feel like the subdomain issue is completely solved. I'm guessing MSN fixed it to check back three levels but not four.
Let's swing back to Yahoo. MSN's domain issues could be excused in part because as a search engine technology, they're still an infant. But here's scoble over at Yahoo showing these domains (numbers show the listing order):
2) scobleizer.wordpress.com
3) www.kunal.org/scoble
5) www.kunal.org/scoble/archives/2004_09.html
6) radio.weblogs.com/0001011/2005/02/19.htm
8) radio.weblogs.com/0001011/2004/01/27.html
9) scobleizer.wordpress.com/2006/02/13/the-brrreeeport-report
OK, it's not unusual for search engines to show up to two pages from the same domain in the first page of results. Google's popularized this concept through its indented results. I'm not even saying that Yahoo should be indenting results. But if you're going to show more than one page per web site, at least get coordinated enough to put then next to each other. If I just saw a scobleizer.wordpress.com page at the top of my result, I don't want to see another page from that domain at the bottom.
How about the widely acknowledged leader in search, Google? Let's do scoble over there. Down at number six is this:
That's a link-only URL, a page Google only knows about through links pointing at it, rather than having visited the page. What happens if I try to visit it? 404 Page Not Found. Heck, I don't even get that. I get a complete non-resolution to the site, a time-out. It doesn't look to exist any more. So why's this making it at number six for that search? Why isn't Google smart enough to drop this entirely useless listing?
To sum up, ego searches will remain a powerful way anyone measures the relevancy of a search engine personally. However, they also remain a dangerous way to assess the overall health and quality of a service. Ultimately, to measure the quality of a service, you remaining wanting an entire battery of different tests run, measuring relevancy and usability.
Posted by Danny Sullivan at 9:57 AM | Permalink
The Thrill & Danger Of Measuring Relevancy Through Ego SearchesLast week, you may recall that Ask got a rave review from the Wall Street Journal. Robert Scoble saw that, then did an ego search for himself and decided Ask doesn't measure up based on that. Fair criticism? Sure, to some degree. But then again, it's easy to take a single search for anything and show that any of the "leaders" in search have problems, as well.
In Search Of The Relevancy Figure from me back in 2002 goes through the various ways that search engines are measured up. Ego Search is a long-standing one, as I described:
"Ego Search" is another style that can be bad, and one that I still see journalists and others often perform. In an ego search, you look for your name. If you fail to come up tops for it, you conclude the search engine's relevancy is poor.
In some cases, perhaps this is true. If I search for "bill gates," it's reasonable to expect to find the official web site for Microsoft Chairman Bill Gates. But what if you aren't as well known as Bill Gates or have a popular name? What if you've built a web site in some free hosting service that is shared by spammers? These might be issues that push you down, and for good reason. Moreover, are you're going to condemn an entire search engine as bad, based on one search? Well, I've seen it happen.
So is Ask bad because it didn't find Robert Scoble's current blog. Yes, for a variety of reasons:
- The new blog's been out there enough that I would expect it to be listed in the top results
- Others can get to it via his old blog, which is listed in the top results, but it would be better not to make that detour
- It matters to Robert
The last point is probably the most important. While ego searches can be bad, they remain one of the main ways anyone will test their favorite search engine. You've got to get them as right as possible, and that's not happening with Robert.
For the record, in my searching today, Google and Yahoo list both the old and new blog (in that order), while MSN gets the new one first, then the old one further down (though it's the opposite if you search for robert scoble rather than scoble)
Also for the record, Robert himself makes the job tough for any search engine. He does no redirection from his old to new site, which pretty much makes it impossible to do the very best thing, list only the new blog.
Think about it. If you search for Robert, why send you to his old blog, just to make you detour? I'm not saying pages from his own blog shouldn't come up for relevant specific queries. But in a search for him by name, you'd really only want to point people to his current blog.
On the flip side, as I wrote on Dave Naylor's blog recently, keeping both blogs going with no redirection is also smart marketing that ensures you dominate the results for your name :)
Back to the search, I also tried using his first and last name. Ask deserves some kudos on that, as robert scoble puts a picture of Robert at the top of the page, based on pulling material from Wikipedia. Nice! Sure, his new blog still isn't listed, but that doesn't mean some other features aren't helpful.
Earlier I said Ask is bad for failing the ego search test for Robert. But bad as in bad on that specific query, rather than bad overall. Condemning an entire search engine based on it going overboard, as Robert does when he says:
But, Walt, this doesn't portend good things for Ask.com. To be included in the top three you've gotta be as good as Google. Ask isn't even in the same neighborhood yet.
Wow. So all I've got to do is find a query where Ask succeeds and the others fail, and then Ask's in the neighborhood. Well, John Battelle helps out here. The Last Name Test post he just put up shows how he can easily find himself on Ask for his own name, just as he's found on Google, Yahoo and MSN.
Then again, that search also points out an Ask Jeeves failure. I remarked about this on our Daily SearchCast show when talking about Ask's rave review by the Wall Street Journal. Ask was having issues with not consolidating web sites correctly.
In particular, the day of our show, I'd done a search at Ask to find one of my articles. I queried invisible tabs, and I got this:
Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor December 2, 2003... searchenginewatch.com/searchday/article.php/3115131 Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor-In-Chief December 2, 2003... www.searchenginewatch.com/searchday/article.php/3115131That's the same page, listed twice. Ask is having a canonical issue, not realizing that the www URL leads to the same page without the www prefix (and this despite us doing redirection).
You can see the same issue with John's search for battelle:
John Battelle's Searchblog John Battelle's Searchblog Thoughts on the intersection of search, media, technology, and more. Search and Newsletter battellemedia.com/John Battelle's Searchblog About John Battelle. Federated Media. UC Berkeley. Columns. That Book I Hear He Wrote. Consulting. Contact... www.battellemedia.com/
All those listings are from the first page of Ask's results. The first and third listing are the same page, just one is shown with the www prefix.
So now we can say that Ask is clearly not in the same neighborhood as Google, Yahoo and MSN! Hold on there.
Let's go to MSN and look for invisible tabs. I'm not saying that my article should be the only thing listed, but look at the result at the bottom from MSN:
invisible girl ABchao Brad Bubs Connie Daynah Dodo Dodo's Tabs Heather Jen Jessie Kristin Leah Leigh Sanne Sarah Shiny Shiny Shopgirls! TechLog Meta </a> .org - asm > Valid XHTML XFN WordPress
In this case, I'm getting a page that has nothing to do with invisible tabs of any type other than they simply having both of those words on the page somewhere. But over at Ask, every listing in the first results is related in some way, either to my article, or to how they apply to a programming issue, or to the idea of guitar tabs you use to play the song Invisible. By this measure, MSN isn't in the neighborhood of Ask.
As for domain issues, MSN's fixed the embarrassing issue it had with every cars.com subdomain dominating (shall I say domainating) a search for cars, but then when I search for movies today, I get all these from the go.com domain:
- movies.go.com
- casanova.movies.go.com
- video.movies.go.com/sincity
None of these are "indented" results, either, so it doesn't feel like the subdomain issue is completely solved. I'm guessing MSN fixed it to check back three levels but not four.
Let's swing back to Yahoo. MSN's domain issues could be excused in part because as a search engine technology, they're still an infant. But here's scoble over at Yahoo showing these domains (numbers show the listing order):
2) scobleizer.wordpress.com
3) www.kunal.org/scoble
5) www.kunal.org/scoble/archives/2004_09.html
6) radio.weblogs.com/0001011/2005/02/19.htm
8) radio.weblogs.com/0001011/2004/01/27.html
9) scobleizer.wordpress.com/2006/02/13/the-brrreeeport-report
OK, it's not unusual for search engines to show up to two pages from the same domain in the first page of results. Google's popularized this concept through its indented results. I'm not even saying that Yahoo should be indenting results. But if you're going to show more than one page per web site, at least get coordinated enough to put then next to each other. If I just saw a scobleizer.wordpress.com page at the top of my result, I don't want to see another page from that domain at the bottom.
How about the widely acknowledged leader in search, Google? Let's do scoble over there. Down at number six is this:
That's a link-only URL, a page Google only knows about through links pointing at it, rather than having visited the page. What happens if I try to visit it? 404 Page Not Found. Heck, I don't even get that. I get a complete non-resolution to the site, a time-out. It doesn't look to exist any more. So why's this making it at number six for that search? Why isn't Google smart enough to drop this entirely useless listing?
To sum up, ego searches will remain a powerful way anyone measures the relevancy of a search engine personally. However, they also remain a dangerous way to assess the overall health and quality of a service. Ultimately, to measure the quality of a service, you remaining wanting an entire battery of different tests run, measuring relevancy and usability.
Posted by Kevin Heisler at 9:57 AM | Permalink
The Thrill & Danger Of Measuring Relevancy Through Ego SearchesLast week, you may recall that Ask got a rave review from the Wall Street Journal. Robert Scoble saw that, then did an ego search for himself and decided Ask doesn't measure up based on that. Fair criticism? Sure, to some degree. But then again, it's easy to take a single search for anything and show that any of the "leaders" in search have problems, as well.
In Search Of The Relevancy Figure from me back in 2002 goes through the various ways that search engines are measured up. Ego Search is a long-standing one, as I described:
"Ego Search" is another style that can be bad, and one that I still see journalists and others often perform. In an ego search, you look for your name. If you fail to come up tops for it, you conclude the search engine's relevancy is poor.
In some cases, perhaps this is true. If I search for "bill gates," it's reasonable to expect to find the official web site for Microsoft Chairman Bill Gates. But what if you aren't as well known as Bill Gates or have a popular name? What if you've built a web site in some free hosting service that is shared by spammers? These might be issues that push you down, and for good reason. Moreover, are you're going to condemn an entire search engine as bad, based on one search? Well, I've seen it happen.
So is Ask bad because it didn't find Robert Scoble's current blog. Yes, for a variety of reasons:
- The new blog's been out there enough that I would expect it to be listed in the top results
- Others can get to it via his old blog, which is listed in the top results, but it would be better not to make that detour
- It matters to Robert
The last point is probably the most important. While ego searches can be bad, they remain one of the main ways anyone will test their favorite search engine. You've got to get them as right as possible, and that's not happening with Robert.
For the record, in my searching today, Google and Yahoo list both the old and new blog (in that order), while MSN gets the new one first, then the old one further down (though it's the opposite if you search for robert scoble rather than scoble)
Also for the record, Robert himself makes the job tough for any search engine. He does no redirection from his old to new site, which pretty much makes it impossible to do the very best thing, list only the new blog.
Think about it. If you search for Robert, why send you to his old blog, just to make you detour? I'm not saying pages from his own blog shouldn't come up for relevant specific queries. But in a search for him by name, you'd really only want to point people to his current blog.
On the flip side, as I wrote on Dave Naylor's blog recently, keeping both blogs going with no redirection is also smart marketing that ensures you dominate the results for your name :)
Back to the search, I also tried using his first and last name. Ask deserves some kudos on that, as robert scoble puts a picture of Robert at the top of the page, based on pulling material from Wikipedia. Nice! Sure, his new blog still isn't listed, but that doesn't mean some other features aren't helpful.
Earlier I said Ask is bad for failing the ego search test for Robert. But bad as in bad on that specific query, rather than bad overall. Condemning an entire search engine based on it going overboard, as Robert does when he says:
But, Walt, this doesn't portend good things for Ask.com. To be included in the top three you've gotta be as good as Google. Ask isn't even in the same neighborhood yet.
Wow. So all I've got to do is find a query where Ask succeeds and the others fail, and then Ask's in the neighborhood. Well, John Battelle helps out here. The Last Name Test post he just put up shows how he can easily find himself on Ask for his own name, just as he's found on Google, Yahoo and MSN.
Then again, that search also points out an Ask Jeeves failure. I remarked about this on our Daily SearchCast show when talking about Ask's rave review by the Wall Street Journal. Ask was having issues with not consolidating web sites correctly.
In particular, the day of our show, I'd done a search at Ask to find one of my articles. I queried invisible tabs, and I got this:
Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor December 2, 2003... searchenginewatch.com/searchday/article.php/3115131 Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor-In-Chief December 2, 2003... www.searchenginewatch.com/searchday/article.php/3115131That's the same page, listed twice. Ask is having a canonical issue, not realizing that the www URL leads to the same page without the www prefix (and this despite us doing redirection).
You can see the same issue with John's search for battelle:
John Battelle's Searchblog John Battelle's Searchblog Thoughts on the intersection of search, media, technology, and more. Search and Newsletter battellemedia.com/John Battelle's Searchblog About John Battelle. Federated Media. UC Berkeley. Columns. That Book I Hear He Wrote. Consulting. Contact... www.battellemedia.com/
All those listings are from the first page of Ask's results. The first and third listing are the same page, just one is shown with the www prefix.
So now we can say that Ask is clearly not in the same neighborhood as Google, Yahoo and MSN! Hold on there.
Let's go to MSN and look for invisible tabs. I'm not saying that my article should be the only thing listed, but look at the result at the bottom from MSN:
invisible girl ABchao Brad Bubs Connie Daynah Dodo Dodo's Tabs Heather Jen Jessie Kristin Leah Leigh Sanne Sarah Shiny Shiny Shopgirls! TechLog Meta </a> .org - asm > Valid XHTML XFN WordPress
In this case, I'm getting a page that has nothing to do with invisible tabs of any type other than they simply having both of those words on the page somewhere. But over at Ask, every listing in the first results is related in some way, either to my article, or to how they apply to a programming issue, or to the idea of guitar tabs you use to play the song Invisible. By this measure, MSN isn't in the neighborhood of Ask.
As for domain issues, MSN's fixed the embarrassing issue it had with every cars.com subdomain dominating (shall I say domainating) a search for cars, but then when I search for movies today, I get all these from the go.com domain:
- movies.go.com
- casanova.movies.go.com
- video.movies.go.com/sincity
None of these are "indented" results, either, so it doesn't feel like the subdomain issue is completely solved. I'm guessing MSN fixed it to check back three levels but not four.
Let's swing back to Yahoo. MSN's domain issues could be excused in part because as a search engine technology, they're still an infant. But here's scoble over at Yahoo showing these domains (numbers show the listing order):
2) scobleizer.wordpress.com
3) www.kunal.org/scoble
5) www.kunal.org/scoble/archives/2004_09.html
6) radio.weblogs.com/0001011/2005/02/19.htm
8) radio.weblogs.com/0001011/2004/01/27.html
9) scobleizer.wordpress.com/2006/02/13/the-brrreeeport-report
OK, it's not unusual for search engines to show up to two pages from the same domain in the first page of results. Google's popularized this concept through its indented results. I'm not even saying that Yahoo should be indenting results. But if you're going to show more than one page per web site, at least get coordinated enough to put then next to each other. If I just saw a scobleizer.wordpress.com page at the top of my result, I don't want to see another page from that domain at the bottom.
How about the widely acknowledged leader in search, Google? Let's do scoble over there. Down at number six is this:
That's a link-only URL, a page Google only knows about through links pointing at it, rather than having visited the page. What happens if I try to visit it? 404 Page Not Found. Heck, I don't even get that. I get a complete non-resolution to the site, a time-out. It doesn't look to exist any more. So why's this making it at number six for that search? Why isn't Google smart enough to drop this entirely useless listing?
To sum up, ego searches will remain a powerful way anyone measures the relevancy of a search engine personally. However, they also remain a dangerous way to assess the overall health and quality of a service. Ultimately, to measure the quality of a service, you remaining wanting an entire battery of different tests run, measuring relevancy and usability.
Posted by Kevin Heisler at 9:57 AM | Permalink
The Thrill & Danger Of Measuring Relevancy Through Ego SearchesLast week, you may recall that Ask got a rave review from the Wall Street Journal. Robert Scoble saw that, then did an ego search for himself and decided Ask doesn't measure up based on that. Fair criticism? Sure, to some degree. But then again, it's easy to take a single search for anything and show that any of the "leaders" in search have problems, as well.
In Search Of The Relevancy Figure from me back in 2002 goes through the various ways that search engines are measured up. Ego Search is a long-standing one, as I described:
"Ego Search" is another style that can be bad, and one that I still see journalists and others often perform. In an ego search, you look for your name. If you fail to come up tops for it, you conclude the search engine's relevancy is poor.
In some cases, perhaps this is true. If I search for "bill gates," it's reasonable to expect to find the official web site for Microsoft Chairman Bill Gates. But what if you aren't as well known as Bill Gates or have a popular name? What if you've built a web site in some free hosting service that is shared by spammers? These might be issues that push you down, and for good reason. Moreover, are you're going to condemn an entire search engine as bad, based on one search? Well, I've seen it happen.
So is Ask bad because it didn't find Robert Scoble's current blog. Yes, for a variety of reasons:
- The new blog's been out there enough that I would expect it to be listed in the top results
- Others can get to it via his old blog, which is listed in the top results, but it would be better not to make that detour
- It matters to Robert
The last point is probably the most important. While ego searches can be bad, they remain one of the main ways anyone will test their favorite search engine. You've got to get them as right as possible, and that's not happening with Robert.
For the record, in my searching today, Google and Yahoo list both the old and new blog (in that order), while MSN gets the new one first, then the old one further down (though it's the opposite if you search for robert scoble rather than scoble)
Also for the record, Robert himself makes the job tough for any search engine. He does no redirection from his old to new site, which pretty much makes it impossible to do the very best thing, list only the new blog.
Think about it. If you search for Robert, why send you to his old blog, just to make you detour? I'm not saying pages from his own blog shouldn't come up for relevant specific queries. But in a search for him by name, you'd really only want to point people to his current blog.
On the flip side, as I wrote on Dave Naylor's blog recently, keeping both blogs going with no redirection is also smart marketing that ensures you dominate the results for your name :)
Back to the search, I also tried using his first and last name. Ask deserves some kudos on that, as robert scoble puts a picture of Robert at the top of the page, based on pulling material from Wikipedia. Nice! Sure, his new blog still isn't listed, but that doesn't mean some other features aren't helpful.
Earlier I said Ask is bad for failing the ego search test for Robert. But bad as in bad on that specific query, rather than bad overall. Condemning an entire search engine based on it going overboard, as Robert does when he says:
But, Walt, this doesn't portend good things for Ask.com. To be included in the top three you've gotta be as good as Google. Ask isn't even in the same neighborhood yet.
Wow. So all I've got to do is find a query where Ask succeeds and the others fail, and then Ask's in the neighborhood. Well, John Battelle helps out here. The Last Name Test post he just put up shows how he can easily find himself on Ask for his own name, just as he's found on Google, Yahoo and MSN.
Then again, that search also points out an Ask Jeeves failure. I remarked about this on our Daily SearchCast show when talking about Ask's rave review by the Wall Street Journal. Ask was having issues with not consolidating web sites correctly.
In particular, the day of our show, I'd done a search at Ask to find one of my articles. I queried invisible tabs, and I got this:
Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor December 2, 2003... searchenginewatch.com/searchday/article.php/3115131 Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor-In-Chief December 2, 2003... www.searchenginewatch.com/searchday/article.php/3115131That's the same page, listed twice. Ask is having a canonical issue, not realizing that the www URL leads to the same page without the www prefix (and this despite us doing redirection).
You can see the same issue with John's search for battelle:
John Battelle's Searchblog John Battelle's Searchblog Thoughts on the intersection of search, media, technology, and more. Search and Newsletter battellemedia.com/John Battelle's Searchblog About John Battelle. Federated Media. UC Berkeley. Columns. That Book I Hear He Wrote. Consulting. Contact... www.battellemedia.com/
All those listings are from the first page of Ask's results. The first and third listing are the same page, just one is shown with the www prefix.
So now we can say that Ask is clearly not in the same neighborhood as Google, Yahoo and MSN! Hold on there.
Let's go to MSN and look for invisible tabs. I'm not saying that my article should be the only thing listed, but look at the result at the bottom from MSN:
invisible girl ABchao Brad Bubs Connie Daynah Dodo Dodo's Tabs Heather Jen Jessie Kristin Leah Leigh Sanne Sarah Shiny Shiny Shopgirls! TechLog Meta </a> .org - asm > Valid XHTML XFN WordPress
In this case, I'm getting a page that has nothing to do with invisible tabs of any type other than they simply having both of those words on the page somewhere. But over at Ask, every listing in the first results is related in some way, either to my article, or to how they apply to a programming issue, or to the idea of guitar tabs you use to play the song Invisible. By this measure, MSN isn't in the neighborhood of Ask.
As for domain issues, MSN's fixed the embarrassing issue it had with every cars.com subdomain dominating (shall I say domainating) a search for cars, but then when I search for movies today, I get all these from the go.com domain:
- movies.go.com
- casanova.movies.go.com
- video.movies.go.com/sincity
None of these are "indented" results, either, so it doesn't feel like the subdomain issue is completely solved. I'm guessing MSN fixed it to check back three levels but not four.
Let's swing back to Yahoo. MSN's domain issues could be excused in part because as a search engine technology, they're still an infant. But here's scoble over at Yahoo showing these domains (numbers show the listing order):
2) scobleizer.wordpress.com
3) www.kunal.org/scoble
5) www.kunal.org/scoble/archives/2004_09.html
6) radio.weblogs.com/0001011/2005/02/19.htm
8) radio.weblogs.com/0001011/2004/01/27.html
9) scobleizer.wordpress.com/2006/02/13/the-brrreeeport-report
OK, it's not unusual for search engines to show up to two pages from the same domain in the first page of results. Google's popularized this concept through its indented results. I'm not even saying that Yahoo should be indenting results. But if you're going to show more than one page per web site, at least get coordinated enough to put then next to each other. If I just saw a scobleizer.wordpress.com page at the top of my result, I don't want to see another page from that domain