Home > ResourceBlog > Article

« All ResourceBlog Articles


Bookmark and Share   Feed

Monday, 23rd February 2004


Web Search
Source: Searchblog,
A Couple of Comments on Search Variables
A post on J.B.'s site says "bravo" to this post on the Scobelizer blog asking for the big engines to give the public access to the "guts" of their search technologies so end users can better tune their results. Scoble also mentions that small engines (Gigablast, Feedster, Technorati and many others) will be the "real innovators." Tim Bray correctly reminds us that most users don't even take the time to use advanced syntax when developing a search strategy.

A couple of comments
+ I don't see that the "big guys" are going to be sharing the guts of their technology with us very soon. Heck, I can't even get Google to explain how their stemming technology works. Does Coca-Cola give out the exact formulas of how products are made? McDonalds still calls it "secret sauce." (-:
+ Many people believe that a single search tool can be all things to all people? I tend to disagree. Is there just a single reference book on your bookshelf? For the librarians out there, you know that many LexisNexis libraries and Dialog files exist.
+ Scoble mentions that innovations will come from smaller databases. I agree. I would also toss into the mix "focused" and targeted tools like SmealSearch and ResearchIndex. Just last week, NEC was awarded a patent for focused crawler technology.
+ Why shouldn't these -- along with thousands of other smaller, focused, and specialized databases -- be "federated" at search time, which would help create a "better" search tool? Instead of incorporating many search tools, each with a different interface, a common interface could be designed to run "on top" of selected resources. All databases would and should remain completely independent.
For example -- searching for news? Bring together results from Technorati, Feedster, Yahoo News, Rocket News, and perhaps a fee-based service like Factiva (if you have access to it or if the publisher offered a pay-per-view option). At the enterprise level, results from the web, fee-based databases, and local search tools can be combined. Dialog offers something similar to what I'm describing called OneSearch.

+ Technology (similar to what Dialog has offered for years) could, if needed, help the user select the most pertinent databases to incorporate into a search (either based on the specific query or a searcher could simply tell the tool to make the decisions).
+ Large databases like Google and Yahoo could also be exploited (if needed) by using them with advanced search strategies that have been "pre-built" for the user. For example, instead of a searcher having to go to the "advanced" page or know specific syntax to limit the query to government info, the federated tool would automatically append the query with the appropriate syntax, enabling a more precise search.
+ Personalization would also be part of the system. Results could be post-processed with dupes removed, results clustered, and then sorted and re-sorted according to the search need. Initial result sets could be based on a user's profile and/or past usage, but could be easily tweaked at the time of the search. As I mentioned yesterday, I think Yahoo is doing good things with their SmartSort technology. It's easy to understand and use. Other views of the information, including visualization of results, could also be made available if the user found it helpful.
+ A common interface (designed for the needs of a specific user group) will make it easier for people to take full advantage of these disparate tools.
+ Let's not forget that people often don't want links. They want an answer. So, in addition to simultaneously searching disparate sources, technology should also be able to summarize and, if possible (depending on the query), present a possible answer(s) to fact-based queries.
+ We're already starting to see this with FAST's new ESP technology, WebFountain, and other federated search technology* providers.
* Full disclosure: A company offering federated technology is a sponsor of ResourceShelf.
+ Yes, this sounds like meta searching and to a certain degree it is. However, what we've come to think of as meta searching is just the tip of the iceberg. With additional search technologies, access to a wide variety of databases (web, fee-based, local), and post-processing technology, we could create a robust resource that would offer full-power (for those who want it), but also be simple enough for the average user. Btw, NISO is also doing work in the federated search/meta search arena.
+ It's very easy to envision how information professionals could be valuable in the creation and maintenance of tools like these.

Views: 521

« All ResourceBlog Articles



FreePint supports the value of information in the enterprise. Read more »

FeedLatest FreePint Content:

  • Click to view the article Product Review of Reg-Track (Introduction; Contact Details)
    Tuesday, 22nd July 2014

    Reviewer Chris Porter introduces Reg-Track, worldwide regulatory tracking service for compliance professionals, from Reg-Room LLC.


  • Click to view the article It's Okay to Browse the News
    Tuesday, 22nd July 2014

    This article looks at two recent European court decisions relating to online copyright and what it means for everyday internet users and media monitoring services. These cases also highlight some of the difficulties of applying copyright law to the online world.

  • Click to view the article FreePint Launches New Series - What You Need to Know Your Customer (KYC)
    Tuesday, 22nd July 2014

    FreePint is taking a fresh look at the ever-changing world of risk and compliance, with a particular focus on Know Your Customer (KYC) requirements. Chris Porter, co-producer of the series with Andrew Lucas, introduces the series.

  • ... more ...

All FreePint Content »
FreePint Topics »

A FreePint Subscription delivers articles and reports that support your organisation's information practice, content and strategy.

Find out more and order a FreePint Subscription by visiting the
completing our online form: Subscription Order page.

FreePint Testimonials

"It was really useful to get so much input from customers and hear their perspective - I have come into the office this morning full of things ..."

Read more testimonials and supply yours »




Register to receive the free ResourceShelf Newsletter, featuring highlighted posts.

Find out more »

Article Categories

All Article Categories »


All Archives »