Based On An Exclusive Interview With Ask's Jim Lanzone. It appears that Yahoo!'s bold and less than brilliant foray into " Looksmart-like " paid inclusion may have been the final nudge that AskJeeves needed to shut down their paid inclusion program, Index Express (not Index Connect which is Inktomi). This significant shift of AskJeeves away from their 18 month-old paid inclusion program appears to be a timely distancing from the pending storm coming to Yahoo! after it announced its new Site Match system.

Why did AskJeeves shut down their Index Express service? To get to the bottom of that I spoke today with Jim Lanzone, VP of product management at AskJeeves. First I should mention that he very carefully noted he does not believe there is a 'dark underbelly' to monetary search engine inclusion models. He noted Yahoo!, Looksmart, and many others when he emphasized that. When we concentrated on the topic of the cancelled Index Express service he explained that AskJeeves came to this decision based on two elements; the first was technical and attributed to significant testing of their paid inclusion model, the second was entirely monetary. The testing revealed that the differences between a page submitted via a trusted feed (xml feeds via Index Express customers) and a page indexed by the Ask spider were so significant that attributing proper relevance was very difficult. As a result, users, advertisers and Ask technicians alike were finding Index Express submitted pages ranking in odd places; sometimes ranking inordinately high or low. The second reason focuses on what is likely the shareholder's bottom line; the model was “not a very good monetization vehicle."

Will the AskJeeves database take a big hit with this change? This is difficult to say but considering that Jim Lanzone said 30,000 of the 2 Billion pages indexed in Ask were Index Express pages there could be a miniscule drop in Ask's database size. Other than that I cannot foresee any significant negative impact. In fact, I only see a brilliant move here since paid inclusion models will undeniably be under the FTC and SEO microscope for the next few months, what with Yahoo!'s 6 web properties adapting to it with gusto.

Note: It is important that our readers understand that the paid submission process at Ask Jeeves is still active and recommended by the staff at StepForth. According to Jim Lanzone, the sites that are submitted via Site Submit will be indexed within one week and then repeatedly 2 times per week. Considering that sites which do not pay submit may not be found or may only be indexed sporadically, this appears to be a very worthwhile service.

An Inside Glimpse of AskJeeves

Right now AskJeeves has a search engine that, in my opinion, is truly impressive. The natural language processing and wealth of quality information in their database has become so good that searching by query actually provides relevant results 90% of the time! This is a vast improvement over the original natural language system that Ask had in place just a year ago. When asked what made AskJeeves so different from its competitors, Jim answered decisively that it was Ask's search technology that put it in a category all of its own. Why the technology? Well Jim argued that the intuitive query performance of the search and the system's ability to reliably show only the experts in every field was Ask's ‘secret sauce'. When I personally put this to the test I had to agree that at the very least the top results I found were relevant and spam-free… an impressive characteristic. What I must enter into consideration, however, is the considerable difference in database size in comparison to Ask's competitors; Ask has only 2 billion pages, whereas Google claims a 6 billion count and Yahoo! over 4.5 billion. In this case size does matter… especially when you consider trying to filter twice to three times more content.

During my interview with Jim Lanzone, we discussed Ask's current standing and where he expects the prominent search engine to appear within the next few years. Obviously Jim could not provide specifics on the technology they plan on including; however, I was able to garner some idea of the company's vision:

Jim gave serious kudos to Google and Yahoo! for creating relationships with the ‘hidden web'; vast information resources once missed by the average search engine such as the Library of Congress, US Supreme Court Audio, the NPR, etc.. According to Jim these types of relationships are definitely going to play a role in future development at Ask. The problem is time, they plan on making some inroads this year but it will take a while before Ask can match the kind of advances that Yahoo or Google have made. This is especially true since Index Express was phased out; initially this was to be the model for uncovering the hidden web.

AskJeeves is very focused on providing a quality user experience. This is evidenced strongly by Ask's current clean interface and Smart Search ideology that “search experience is as important as results themselves”. From what I could gather, Ask's goal is to minimize the successful search experience to one click.

A fresher index was noted which indicates a strong desire to begin spidering web sites more frequently in the near future. Jim did not elaborate on this, however, I speculate that this means isolating web sites that are updated regularly and spidering them more often.

Currently the News section of AskJeeves is populated using Moreover; a popular and reliable news syndication resource. At the moment, Ask only minimally controls the results of its Moreover results with a basic algorithm. This is a major difference between AskJeeves and its search competitors Google and Yahoo!; Ask is the only engine without its own news spider! When asked, Jim noted that advances in Ask's news asset will begin to take place in the second quarter of this year.

What else? At this point in the interview I encountered the familiar and completely understandable ‘wall of vague'; to quote Jim Lanzone, AskJeeves plans to “move into the different areas of search and apply our search engines to new areas of the web and make improvements to the methodologies that determine the relevance of the web.” Well said!