Grub, Distributed Crawling Will Be Used to Build Wikia While LookSmart Will Power Wikia Advertising; Other Open Source Crawling Tools
Grub distributed crawling technology is now being tested for use to to build Wikia after being acquired from Looksmart. This news came via comments from Wikia/Wikipedia co-founder, Jimmy Wales on Thursday.
Distributed crawling? Well, the concept has been around for web search for more than seven years. After a client is placed on your computer and during down time, you're computer will be one of many to crawl the web that will then build the overall database. See also the SETI@Home project.
More on that in a moment.
It's important to point out that advertising on the Wikia site will be delivered via a white-label ad serving platform from LookSmart.
It will handle both display and text-based ads and Wikia will be the first organization to use LookSmart's technology for the, "management/serving of display ad units utilizing CMP-based pricing."
"We did a lot of due diligence to find a flexible and intuitive ad serving technology that nets the highest revenue and yield," said Gil Penchina, CEO of Wikia. "We discovered in the process that LookSmart's platform and services not only provide dynamic optimization of both our advertisers and backfill networks, but the white label aspect of it fits perfectly with our brand strategy."
We discontinued the use and support of the Grub distributed crawling technology in 2005 in order to reallocate development and support resources to other revenue-generating initiatives in search technology.
Abstract: This talk will present two research projects under way in the Network and agents Network (NaN), which study ways of leveraging online social behavior for better Web search. GiveALink.org is a social bookmarking site where users donate their personal bookmarks. A search and recommendation engine is built from a similarity network derived from the hierarchical structure of bookmarks, aggregated across users. 6S is a distributed Web search engine based on an ad adaptive peer network. By learning about each other, peers can route queries through the network to efficiently reach knowledgeable nodes. The resulting peer network structures itself as a small world that uncovers semantic communities and outperforms centralized search engines.
A family of resources to help information workers be more effective, raise the value of information in their organisations and contribute to success. Read more »
Recently I have found myself cooing over visualisation maps (and heat maps) of health and well being resources. The content rich data is overlayed with mapping technologies, and some interesting themes and patterns are emerging.
A lot of the talk around social media in the last year has been around information overload. Social media has provided us with new and exciting ways to create content. But it has also meant learning new ways to manage and engage with social media tools. Are we teetering on the edge of an information overload precipice?
Information overload is a figment of your imagination. Or a failure of your filter. Or a symptom of your technological submissiveness. Depends on who you ask.
What if you had to sort through 3.5 million articles and social media posts a day and try to pull out the most relevant items for your organisation? What if you then had to cobble it all together into something readable for your top groups and executives in your organisation?
Alacra Compliance saves time by aggregating information from both free and fee-based sources and enabling users to conduct an accurate federated search across these sources (coined “simultaneous search” by Alacra).
FreePint Family
Quilting big data threads
The fallacy of information overload
Information overload: fact, fantasy or filter failure? 
