Subscribe - Search Engine Watch

June 26, 2008

State of Missouri Releases Collaborative Search Portal

The State of Missouri has launched a collaborative search portal, enabling citizens of the Show-Me state to explore millions of historical documents. The search, dubbed Missouri Digital Heritage, is powered by Deep Web Technologies' Explorit Research Accelerator. Included in the searchable material are nearly 20 "Collections" such as Transportation, Sports, Women, and Agriculture.

"This is a unique project because of the queries this site is designed to handle," said Abe Lederman, CTO of Deep Web Technologies. "A family historian may have only a fragment of a name or a street address that they want to explore. By delivering every hit -- even the most remote -- we significantly increase their opportunity to put another piece of their family tree in place. Missouri Digital Heritage is truly designed to support every local researcher's needs."

Posted by Nathania Johnson at 10:39 AM | Permalink | Comments (0)

December 12, 2006

Searching for Reliable Stats

The web abounds with statistics, but you often have to search around to find a government agency or other reliable source to get the really good stuff. Enter NationMaster.com, a meta-search tool of sorts that focuses exclusively on reputable sources of online statistics, charts, graphs and other goodies to nurture any data-geek's soul. More on this terrific resource in today's article, Geeking Out on Statistics.

Posted by Chris Sherman at 2:23 AM | Permalink

November 8, 2006

In The Election Results Race, Yahoo's The Winner

Like many Americans, I wanted to know what was up with the vote in the US midterm elections this morning. As a search analyst, I then wanted to know how the search engines performed in helping me find out. The results are in! Yahoo's the winner by far, but I'd still take the New York Times over it. Come along for an illustrated tour.

Google told us last month that Google Earth was all geared up to be an election guide. That's great if you've downloaded Google Earth and wanted to learn more before the election. But how about a quick, fast summary of what happened yesterday? What's Google got for us?

The Google home page is as minimal as always, no help there -- not even a special logo as in the 2004 race.

How about a search for "election results," which I think is a fair query to try. After all, using Google Trends, I can see a huge spike for that term after the last elections in the US:

I also checked the volume for just "results," and that was even higher whereas "elections" was much lower (see them all compared here). So my two queries for this test were "election results" and "results." On Google, both disappointed.

Here are election results on Google:

CNN's top with 2004 results! I know -- web search is always behind the times. That's why Google inserts that big news results OneBox unit above the regular results. Let's click on the main news link there, which takes us to news results:

Pretty bad. News about the dollar, stock prices -- but who won?!!! I've got to really work to figure this out, especially compared to the New York Times, as I'll show at the end of this story.

Maybe I head to the actual Google News home page:

Nope. I get some headlines telling me about the Democratic house victory, but it could be much better.

FYI, checking on a search for just results, I don't even get the news headlines inserted. Overall, I found Google to be a pretty poor resource.

How about Yahoo? The home page there immediately gives you some news:

If I actually gravitate to the picture and resist the pull of the search box, the "Full elections coverage" link takes me to a Full Coverage page with lots of info, including an interactive results page (my link takes that out of the normal pop-up box, but it still works great):

This is very, very nice. I can see at a glance who is ahead in the race for control of the US House Of Representatives, plus with a click I can check out the Senate or governor races. Selecting any state also gives me the information about races with that state.

I love this. It very similar to what impressed me at the New York Times. I hope Yahoo searchers found it. However, I suspect many bypassed it. To understand why, let's do that search for election results:

Similar to Google, Yahoo inserts a big "News Results" shortcut unit above the regular results, to help detour searchers into the freshest results. Of course, searcers might bypass that. If so, unlike Google, Yahoo has managed to get the CNN 2006 results page up rather than the CNN 2004 page. Nice. After that, there's Fox News 2006. But c'mon -- Yahoo's own special election results are third. This is one case where I'd totally applaud a little hand manipulation to get that to the top, especially to highlight that interactive results summary page.

Still, the web search results for this particular day at Yahoo far outshine Google. That's almost certainly due to some human editing, which is fine. Along with the sites I've mentioned, you get the New York Times politics page, USA Today's politics page, C-SPAN's 2006 results page, the ABC News politics page, CBS News's 2006 page, politics from the LA Times, then the Washington Post's 2006 results page. All of these are excellent choices. If Yahoo did human intervention to make this happen, kudos to them. You can check out a snapshot of the entire page here.

Over at Google, nothing is either timely or general enough. The Virginia state election board, California election info, assorted things dating from 2004 -- then oddly Virginia and California get another bump for their 2006 pages. Ugh. See the entire list in the snapshot here.

What happens if we detour into the news area that Yahoo promotes at the top of the page? Disappointment:

Yes, relevant news stories. And the image results to the side are kind of fun. But some hand help could have made a difference. How about a promo for that awesome election map of Yahoo's?

Let's go over to Ask, where I had high hopes. Ask has made a big deal of its special Smart Answers for the election, and they are cool. But will I see them? Yes, if I search for election:

I'd also get to this box if I went to the Ask home page and clicked on the Election Day link there:

But for election results (what I believe to be the more popular query), all I get is a small news unit:

The news unit will take me over to some news results, but like Yahoo's, these aren't thrilling. It's pick and choose through what you want, rather than any type of easy overview. As for a search on just results, that doesn't even bring back the news unit at all.

The overall web search results, similar to Google, are underwhelming. Nothing really helpful for the 2006 results pops up (see the full results in the snapshot here).

Even the special Smart Answers box, had it shown up, isn't that helpful for what I want now -- RESULTS! None of the featured links with it takes me to results.

Microsoft, what have you got for me at Windows Live Search? On the home page, nothing. For search on election results, it's disappointing old or non-targeted results (screenshot here). Unlike the others, there are no news results inserted above these. A search for just results is no better. If I specifically try a news search for election results, as with the others, there's no attempt to get me a comprehensive overview. It's up to me to review each story and hope for a good match.

Ironically, at the largely overshadowed MSN site, similar to Yahoo, I get a big election photo on the home page along with links, including one called "state-by-state results" that leads to MSNBC here. And over there is a pretty neat "Democracy Dashboard" giving me that type of overview I wanted:

It's a pity Windows Live didn't reach out to either MSN or MSNBC and do something special to point to this or somehow integrate it into the results.

What about AOL? From the home page, it's pretty easy to spot a link to a AOL election page with results for the House, Senate and more:

Searching for election results brings back disappointing Google listings in the main results. However, the new FullView column does a good job of dividing news into elections overall, US Senate coverage, US House coverage and more. And clicking on any of the "View all" links brings up the special AOL election page (see the full page here).

Now to the New York Times. I headed over there pretty much by chance. There are any number of newspapers I might of thought of off the top of my head, and usually its my original home town paper of the Los Angeles Times. But I hit the NY Times today, and boy was I glad.

Right on the home page, above the "fold" is an easy-to-spot election map. In seconds, it organized the most important information I was looking for into a way for me to know what was going on:

Drilling into the full map was even better. There, I could click on any state -- in particular the undecided ones -- and see the current situation:

Just when I was thinking "what if," I saw the "Create Outcomes" tab where I could click on a state and flip it to the Democrats or the Republicans to see how it might go with the Senate. Outstanding!

Other newspapers or web sites might have done as well with similar displays. If so, my apologies that this wasn't a review of the best election results sites. Instead, it was really meant to see how well the search engines held up as information resources for this particular news event.

Overall, I've written many times before that there's a role humans can play in search results. Today -- this was a perfect example of that. Yahoo almost certainly put some human effort into crafting results, and it was the clear victor in terms of quality of what was coming up in web search listings. AOL comes in second, again where human effort has helped its FullView listings help make up for the poor crawler-based results from Google.

In third, I put Google and Ask. Google's results were poor, but at least it floated some news results that may have helped. Ask, I was rooting for. But that Smart Answers box simply wasn't showing up for the queries I thought people were doing. Even if people were getting it for "election," it wasn't helpful to get election results. I really appreciate the effort, and if this had been for something other than actual results, Ask would have been great. In last place -- Windows Live.

This campaign of sorts is also one of those classic "what if" races. With just a little more effort, Yahoo would have had a landslide victory by getting people to its great overview page. The same is true for AOL. Ask, with just a bit more thought, could have had that box coming up for "election results" rather than just "elections" and added some links to get people to actual results. Windows Live, if it had remembered its MSN origins, might not be in last. And Google? A company that's all about organizing information might not have put in such a poor performance if it used some human power in the way the New York Times did.

Postscript: See also Case Study: Digg Versus Google News Traffic from me on my personal blog that covers how this article ultimately brought in lots of traffic from Google News from those unable to find election results there, along with lots of other data and a comparison to traffic from a top story at Digg on the same day.

Posted by Danny Sullivan at 4:17 AM | Permalink

November 2, 2006

Ask.com Launches Elections Smart Answers

Ask.com launched some new smart answers for the upcoming elections. You can do a search on elections 2006 to see "General Elections are Tuesday, November 7, 2006." The smart answer also gives you links to the Republican Party and the Democrats web sites. Additional quick links they give within the smart answers include; Analysis by Poll Reports, Race Profiles, Key Races, Key Issues, Races by Issue, and Congressional Districts Map. The additional resources include Candidate Reports, Ballot Items, Campaign Fact Checking, Campaign Funding, and Evaluating Initiatives. Finally, you can also break down by state by searching for New York state elections. I have taken some screen captures also.

Posted by Barry Schwartz at 8:37 AM | Permalink

October 24, 2006

November Elections Mashups

Maps have long been important in politics, as anyone who works in politics or watches the TV coverage on election night knows. And then there's the familiar saying, "All politics is local." Now, as reported on the Google Earth Blog, Google Earth has added "U.S. Elections Guide" and "U.S. Congressional Districts" information layers. By selecting these layers in Google Earth (not available on Google Maps) you can see boundaries of U.S. voting districts, links to information and news about the candidates, where to vote, campaign finance data and other election-related information.

Map-based real estate site HotPads has also created an interactive elections map for the coming November election. Click on any state and you can drill down on U.S. Congressional districts and information on representatives from Wikipedia and other sources. Here's an example from Florida.

I'm sure there are more out there. Let us know if you find them.

Posted by Greg Sterling at 10:50 AM | Permalink

September 11, 2006

Track Federal Spending With Government Search Engine

Andy Beal spotted an AP article named Senate backs online search of spending. Yes, the Senate wants to make it easier for normal users to track what the US Government is spending taxpayers money on. The example search given is that, "one could type in "Boeing" to find contracts awarded to the aerospace company or "breast cancer research" to see efforts to battle one of the leading killers of women." Andy asks, why not just have Google do it for the government, since they already know a lot more than the government does anyway? :)

Posted by Barry Schwartz at 8:30 AM | Permalink

June 15, 2006

Google Relaunches US Government Search, Now With Personalized Home Page

Google launched its search engine for US government information, informally known as Google Uncle Sam, many years ago. It's been running since at least 1999. But now the service has received an update giving it a personalized home page and formal branding as Google US Government Search.

By default, the new home page shows the weather in Washington DC (outlook: political infighting, with mudslinging later in the week) and headlines from the US White House, top government stories out of Google News (though a concocted query string), Washington Post headlines, headlines from the American Forces Information Services and headlines from Government Executive magazine.

As before, you can search and have results come back just from US government web sites, along with state and local government sites in the United States. Google's help page explains more. Try net neutrality on Google US government search, and you'll see how compared to a regular Google search, listings disappear from Wikipedia, political groups supporting net neutrality, news organizations reporting on the issue and other sources. Instead, it's just information published on government web servers in the US. And that's a good thing, for those who just want to home in on official government materials.

There's at least one glitch. The search result pages still show the old red, white & blue Google American Flag-styled logo on the government search site, and clicking on the logo takes you back to regular Google rather than the government search home page.

Already have a personalized home page? One nice thing is that you can personalize the US government search page independently of your other page. However, searches on the US government search site do flow into your main search history, if you have the feature enabled.

The relaunch comes on the heels of Google political moves last week. Google tried a last-minute lobbying attempt for net neutrality by cofounder Sergey Brin and an effort to rally Google users to lobby for net neutrality plus harvest their names for future political pushes.

It's hard not to see the updated US government search service as a way to attract government workers and insiders to a place where Google can influence them. Google ultimately controls the personalized home page and can choose to insert material on it any time it wants. That's a powerful tool if many people involved with the government start tuning into the page.

Certainly giving the Washington Post an exclusive on breaking the news helps fuel the idea that Google's doing a push along these lines. The Post is the only media outlet to have been prebriefed on the release, that I can tell. That helps ensure the story gets good play, plus causes competing print media outlets to give the story a second day of coverage doing catch-up stories. Of course, the Post also gets prime space on the new site, as well. That probably won't please some competing political news publications, though anyone can add anything to that page manually.

By the way, to add material, look for the Add Content link in the top left-hand side of the page. Opening that allows you to add the URL of any publication producing an RSS feed with content, or you can also search for matching sites of interest. Google also looks to have added a new Government category of recommended selections, which offer a range of official sources. See also the help page for more on personalization.

Google to Launch Government Search Site is the Washington Post's story on the launch, with Google quotes on seeking to serve government employees and average citizens better with it. The story also list competing government search engines, including the official one at FirstGov.gov. That's powered by Microsoft. We covered this recently here: New Firstgov.gov Search Database Goes Live.

By the way, let's clear up some URL confusion:

http://usgov.google.com is the main URL for the new site.
http://www.google.com/ig/usgov also works. It's where the official URL redirects people to.
http://www.google.com/unclesam is the address of the old service, which is still live and works just like the new service, but without the personalized components. You also find it still listed on Google's special searches page. I expect Google will eventually redirect this URL to the new service when they realize it's still live.

Looking for more about government search engines? See the Search Types: Government category of our Search Topics section, available to Search Engine Watch members.

Postscript: Google now has a press release on the launch up here, though it is fairly sparse. The Google Blog also has a short post here. And Google tells me the Washington Post was the only major daily/wire outlet prebriefed, though some small government trade publications also got info.

Posted by Danny Sullivan at 9:35 AM | Permalink

Google Relaunches US Government Search, Now With Personalized Home Page

By the way, let's clear up some URL confusion:

http://usgov.google.com is the main URL for the new site.
http://www.google.com/ig/usgov also works. It's where the official URL redirects people to.
http://www.google.com/unclesam is the address of the old service, which is still live and works just like the new service, but without the personalized components. You also find it still listed on Google's special searches page. I expect Google will eventually redirect this URL to the new service when they realize it's still live.

Looking for more about government search engines? See the Search Types: Government category of our Search Topics section, available to Search Engine Watch members.

Posted by Kevin Heisler at 9:35 AM | Permalink

Google Relaunches US Government Search, Now With Personalized Home Page

By the way, let's clear up some URL confusion:

http://usgov.google.com is the main URL for the new site.
http://www.google.com/ig/usgov also works. It's where the official URL redirects people to.
http://www.google.com/unclesam is the address of the old service, which is still live and works just like the new service, but without the personalized components. You also find it still listed on Google's special searches page. I expect Google will eventually redirect this URL to the new service when they realize it's still live.

Looking for more about government search engines? See the Search Types: Government category of our Search Topics section, available to Search Engine Watch members.

Posted by Kevin Heisler at 9:35 AM | Permalink

Google Relaunches US Government Search, Now With Personalized Home Page

By the way, let's clear up some URL confusion:

http://usgov.google.com is the main URL for the new site.
http://www.google.com/ig/usgov also works. It's where the official URL redirects people to.
http://www.google.com/unclesam is the address of the old service, which is still live and works just like the new service, but without the personalized components. You also find it still listed on Google's special searches page. I expect Google will eventually redirect this URL to the new service when they realize it's still live.

Looking for more about government search engines? See the Search Types: Government category of our Search Topics section, available to Search Engine Watch members.

Posted by Kevin Heisler at 9:35 AM | Permalink

March 24, 2006

Search National Archives and Records Administration Web Pages With Web Harvest

Gary Price reports on a new search engine named Web Harvest that enables you to search governmental and military Web pages. The FAQ says that the harvest collection contains about "75 million web pages and represents about 50,000 ".gov" and ".mil" unrestricted federal web sites active between 10/14/04 and 11/19/04." The search technology is currently using NUTCH and NUTCHWAX, open source solutions.

Posted by Barry Schwartz at 10:15 AM | Permalink

Search National Archives and Records Administration Web Pages With Web Harvest

Posted by Kevin Heisler at 10:15 AM | Permalink

Search National Archives and Records Administration Web Pages With Web Harvest

Posted by Kevin Heisler at 10:15 AM | Permalink

Search National Archives and Records Administration Web Pages With Web Harvest

Posted by Kevin Heisler at 10:15 AM | Permalink

January 26, 2006

askSam Now Offering Searchable Databases (Free) Containing U.S. Copyright Law, The DMCA, and Every State of the Union Address Back to 1790

On what is a growing number of occasions we've beeb highlighting the good and very useful work that Phil Schnyder and his team at askSam are doing by are providing free searchable and browsable (online or download and use offline) to classic books, government and legal documents, speeches, and more utilizing askSam database software.

Today, we've learned that ask Sam has just released three new databases (what they call eBooks) that might be of interest to some of you, especially those with an interest in copyright issues.

First, U.S. Copyright Law (title 17 of the US Code) "Search and analyze the full text of the Copyright Law of the United States of America & related laws contained in Title 17 of the United States Code. Copyright is a form of protection provided by the laws of the United States (title 17, U.S. Code) to the authors of 'original works of authorship,' including literary, dramatic, musical, artistic, and certain other intellectual works."

Second, The Digital Millennium Copyright Act (DMCA) "Search and analyze the full text of The Digital Millennium Copyright Act (DMCA). Passed in 1998, the DMCA is a bill designed to bring the Copyright Law up to date with digital media."

Third, State of the Union Addresses of the American Presidents "Search and analyze the full-text of all State of the Union Addresses from 1790-2005."

A complete and rapidly expanding list of ask Sam book, all free, can be found here.

Btw, to view offline you'll need a free copy of the askSam reader.

Posted by Gary Price at 12:13 PM | Permalink

askSam Now Offering Searchable Databases (Free) Containing U.S. Copyright Law, The DMCA, and Every State of the Union Address Back to 1790

Today, we've learned that ask Sam has just released three new databases (what they call eBooks) that might be of interest to some of you, especially those with an interest in copyright issues.

Third, State of the Union Addresses of the American Presidents "Search and analyze the full-text of all State of the Union Addresses from 1790-2005."

A complete and rapidly expanding list of ask Sam book, all free, can be found here.

Btw, to view offline you'll need a free copy of the askSam reader.

Posted by Kevin Heisler at 12:13 PM | Permalink

askSam Now Offering Searchable Databases (Free) Containing U.S. Copyright Law, The DMCA, and Every State of the Union Address Back to 1790

Today, we've learned that ask Sam has just released three new databases (what they call eBooks) that might be of interest to some of you, especially those with an interest in copyright issues.

Third, State of the Union Addresses of the American Presidents "Search and analyze the full-text of all State of the Union Addresses from 1790-2005."

A complete and rapidly expanding list of ask Sam book, all free, can be found here.

Btw, to view offline you'll need a free copy of the askSam reader.

Posted by Kevin Heisler at 12:13 PM | Permalink

askSam Now Offering Searchable Databases (Free) Containing U.S. Copyright Law, The DMCA, and Every State of the Union Address Back to 1790

Today, we've learned that ask Sam has just released three new databases (what they call eBooks) that might be of interest to some of you, especially those with an interest in copyright issues.

Third, State of the Union Addresses of the American Presidents "Search and analyze the full-text of all State of the Union Addresses from 1790-2005."

A complete and rapidly expanding list of ask Sam book, all free, can be found here.

Btw, to view offline you'll need a free copy of the askSam reader.

Posted by Kevin Heisler at 12:13 PM | Permalink

January 24, 2006

New Firstgov.gov Search Database Goes Live

The new Firstgov.gov search that offers access to content from U.S. federal, state, local tribal and territorial sources is now live. You can find access to the clean interface (a single search box) either via a link on the FirstGov.gov home page or by going directly to Firstgovsearch.gov.

An advanced search interface with added functionality is also available along with a Spanish-language version of the interface that offers quick access to material in Espanol.

Back in September we posted that Vivisimo (the company and technology that powers Clusty along with MSN Search had won the government contract to power the new FirstGov search engine.

This blog post has more about the contract itself (including financial arrangements).

The feel, look, and organization of the actual FirstGov portal remains the same. Again, what's new today is FirstGov's search functionality.

What's New, Fast Facts + Larger Database Database grows from about 8 million "government related" pages to 40 million.

+ Dynamic Clustering Like what you've come to expect from Clusty.

+ Preview Function Again, a Clusty staple. Click the linked labeled "preview" included in each results "snippet" and view a live version of the result page (not a static image) embedded into the results page.

+ Metasearch from Various U.S. Government Databases Material comes from not only the open web but from specialty government databases like MyMoney.gov, Forms.gov (government forms), and Fueleconomy.gov.

A Few Minutes with the New FirstGov.gov Search Within the first couple of searches I quickly noticed the improvements that the new FirstGov.gov search provides.

Of course, that's just my opinion. If you want to run some comparisons between the new Firstgov search and what was previously available, you can still access to the "old" FirstGov engine for the next week or so here. Simply enter a search in the box at the top of the page and you'll be using the "old database." You can access the "old database" by using the "old" advanced interface here.

The NEW Firstgov.gov search offers a plethora of options on results pages. Here's a simple search for the phrase "student loans".

Worth Noticing on the Results Page + First result is a guide direct from the Dept. of Education. Btw, most web pages offer cached copies.

+ Which databases are you searching? Next to the web search totals you'll see a link labeled "Details," click and find what sources were queried and how many results came from each one.

+ Directly below the search box are tabs. You'll find links to ask and answered questions about student loans (great use of the FirstGov FAQ knowledge base), along with tabs to find results from MyMoney.gov and a few student loan related forms from Forms.gov. Of course, these tabs are based on your query. Another query, produced a tab that lists government podcasts.

+ Note the first result for a search for "navy ships" and you should spot a next to the url in the first result that reads "more from the Navy." Clicking this link runs a site specific search of the US Naval Vessel Register web site. You can see the results here. Impressive and a great way to get the searcher to a particularly useful and authoritative site. Remember, an advanced user interface is also available that offers all sorts of search limits.

+ Here's another search, this time for "trade statistics." In this case now tabs are available but numerous clusters appear in the left column. Click on the link and see the new results and/or click on the plus sign (+) and see "sub clusters."

+ Also, at the top of the cluster section notice the tabs that allow you to cluster results by topic (default), government agency (very useful), and source. It would be great if they would also offer a tab that allowed you to quickly see clusters by U.S. State when running a general search.

Again, don't forget that these clusters, tabs, etc. are all produced dynamically for each search. So, what's available for one query might not be available for another.

Overall A mighty impressive beginning for the new Firstgov.gov search. I'm looking forward to seeing not only what other features they might offer in future releases but also the specialty databases and tools they include in the metasearch portion of the service.

On Dynamic Clustering and Metasearch We've been impressed with both Vivisimo (for many years) and Clusty since it launched in 2004. Here's a the overview tht Chris and I authored for SearchDay on the day Clusty debuted.

Those of you who read the blog on a regular basis also now that I appreciate the potential of metasearch more and more each day. The same is true with one form or another of dynamic clustering and/or query refinement.

As this paper from Vivisimo puts it, dynamic clustering can help provide "selective ignorance" for web searchers. In other words, it can help get the searcher quickly and with little effort to the best results for their query. For many people the "deep web" is anything beyond the sixth or seventh result on a web results page.

Is dynamic clustering perfect? Is it THE solution? No, not at this point but the tech is improving all of the time and with the proliferation of content happening at such a rapid pace, it potential grows more and more useful. Perhaps the biggest challenge that the new FirstGov.gov faces as well as Clusty and other tools that offer clustering and dynamic query modification is one of training. In other words, letting the average searcher know what clustering can offer and how to best take advantage of it.

A Bit of History From 2002 until today launch, FirstGov's search capabilities were powered with a database and search technology provided by Fast Search and Transfer and managed by AT&T. At its inception, FirstGov search was powered with technology from Inktomi.

Posted by Gary Price at 10:23 AM | Permalink