RSS feeds for search results

I did a web search for ‘RSS feeds for search results’ and found a Live Search blog entry dating from 1995 that talks about the alpha feature of providing RSS feeds from search results: just add ‘&format=rss’ to the URL of the results and copy the full URL into your RSS Reader.

This still works on the main Live search site and is made easier by just clicking on the orange RSS icon in the URL. Here’s the ‘RSSed’ URL for Flooding information from the UK government: http://search.msn.com/results.aspx?q=flooding+site%3agov.uk&format=rss

Very nifty, but interestingly, Live don’t seem to draw users’ attention to it or even provide information in Help.

Yahoo! offers the similar features to Live, with different syntax, but the RSS icon.

In contrast, Google only offers RSS feeds (RSS and Atom) for its news results but does publicise it clearly. Similarly, the BBC offers RSS of its news and sports results.

Other web search engines such as Ask, Exalead and Clusty do not appear to offer RSS at all.

Search behaviour patterns

John Ferrara writes an excellent piece at Boxes and Arrows on designing for different Search behaviour patterns. He argues that the simple model of search; enter search term/get matching results/read and select ‘best’ result’ overlooks complex human behaviours and differing information needs according to many factors.

He lists six factors, among others, that influence search behavior:

1. Domain expertise
2. Search experience
3. Cognitive style (global thinkers <—-> analytical thinkers)
4. Goal type: navigational searches (to reach a particular location), informational searches (to seek out any documents relating to a topic), Transactional searches (to achieve something online)
5. Mode of searching - browsing and ‘berrypicking’
6. Situational idiosyncracies - search behavior may vary due to mood, workin context, time pressures etc.

Despite the wide variety of variables, Ferrara has observed six behaviour patterns that should be accommodated in design:

1. Alternating between search and browse - have robust cross-linking and hierarchical clues such as crumb trails
2. Minimizing the results set - let users filter search results by categories, include a numeric count of the total number of results and the total number for each category , use “and” as the default operator rather than “or,” so the number of results narrows instead of growing as the user adds more terms
3. Surveying quickly - ensure that result titles are comprehensible at a glance, including application files like PDFs, highlight the terms that match the words originally submitted, let users change the number of results shown per page
4. Making immediate judgments - optimise results for common queries and use best bets
5. Agonizing over the query - provide suggestions as user types, show ‘top’ searches, store user’s previous searches
6. Pogosticking - where user’s bounce around - clear titles and descriptions, show visited links on the results page and consider preview windows of the landing page

This is a really useful reference piece and I’d agree with nearly all of it, except “Highlighting the terms that match the words originally submitted to help people scan the titles and descriptions more easily.”

Progressive disclosure of search options

Writing recently at Boxes and Arrows, Stephen Turbek advocates Advancing Advanced Search. I agree with his articulation that “The essential problem of search — too many irrelevant results — has not gone away.” and that most implementations of Advanced Search make little contribution to fixing the problem.

Little contribution because:

• The link to advanced search is usually small and hard to find
• Advanced search pages are usually confusing
• Functionality is often little more than with simple search and it’s hard to revise your query
• You have to make a choice between doing a search and clicking a link to do a search. “In other words, do you want it now, or want to go somewhere else to look?”

After a brief review of alternative approaches - defined parameters, drilling down on tags, and facets, Turbek suggests an alternative approach of progressively disclosing further options.

So after the initial search, a user is presented with a very visible options box above the results. Once the user has invoked this, further options could be built up.

Of course, this is best suited to an enterprise or e-commerce context, where there is likely to be more metadata and users may be more accustomed to filtering their results.

Nevertheless it’s an interesting way of making more advanced search tools available to basic searchers and helps expose relationships the user may not have thought of.

Strategies for improving enterprise search

John Ferrara argues the importance of going beyond the out-of-box experience at Boxes and Arrows Sep 07.

I’d agree with his argument that quality results only come about through applied effort.

The conceptual task

1. Search engine. The algorithmic gears that parse the query and assign pages relevance.
2. Content. The documents searched.
3. Index. A catalogue of the locations of every word in every document.
4. User input. The keywords and other parameters the user submits.
5. Results display. The way the data returned by the search engine is presented.

Strategies

1. Make content machine readable.
   • Structural markup - using semantic elements and class attributes.
   • Use the keywords and descriptions tags and build a controlled vocabulary to populate. Well, I think there’s rooms for a and the tactical application of wild keywords.
   • More metadata - audience, sector etc.
   • Ontology - Get the search engine to exploit the relationships between concepts.
2. Index All of the Right Data
   • Ignore unnecessary content - navigation, footers, adverts etc.
   • Get all resources - what about content in PDFs, Word documents etc?. I’d agree that search needs to be aware of them, but disagree about indexing them all. I often find a more usable experience is to index a web page that provides links to and a summary of such documents. The ranking of digital assets can be unpredictable and getting it right is, as Ferrara says, a big job
3. Make the most of user input - it is important to make the best of the user’s intent on their first search attempt- after all the user has probably a complex intent behind those few keywords and users are much less likely to make a second. (People Search Once, Maybe Twice - Jared Spool 2001).
   • Query expansion - stemming, thesaurus.
   • Syntax conventions - get the parser to understand common human syntax - and use AND as the default operator - not sure about that; it can be too restrictive.
   • Assisting query formulation - Did you mean?, others searched for…
4. Build results around the user’s needs
   • Demonstrate relevance
     • Generate a snippet which contains the user’s terms. I disagree - in an enterprise solution, they often not very good and you have the opportunity to teach editors to write good, relevant descriptions which include the most important keywords.
     • Bold the terms that match the user’s query terms.
   • Best bets/ recommended links. I’m a huge fan of these as they can really help orientate users and push important content for the business.
   • Conditional content. Take best bets further and display contextually appropriate content when a query indicates a user has a particular interest.

Information behaviours

“People don’t understand their own information behaviors, and they don’t really understand much about search or the web, so they will have to learn. It could take generations.”

Christina Wodtke from Boxes and Arrows interviews Amanda Spink in Oct 2006.

Most web queries short:
2- 3 terms

Sessions:
2-3 queries in length with little query modification

Long tail:
The long tail makes ‘relevant’ retrieval very hard - especially if users use only 2 or 3 words - search is not very interactive. Personalisation is an attempt to get more info from user to help determine relevance.

Complex search behaviours:
Often long and complex - beyond one topic/one search paradigm most search engine assume

• Successive searching (e.g. looking for information on cars)
• Multitasking search (batch searches - time constraint or new topics emerge)

Search is challenging and interactive:

• no silver bullet or ’single’ feature that’s effective
• clustering + relevance feedback +??
• lack of people trained in info and web retrieval, web design and usability
• how to elucidate real intent from a small number of keywords?

GoPubMed

An interesting use of facets based on an ontology. Sorting abstracts according to the concept hierarchies of GO and MeSH enable a combined search in molecular biology and medicine. The “What” categories help to systematically explore the results. The tree displayed to the left of results serves as table of contents in order to structure the millions of articles of the PubMed/MEDLINE data base. By navigating the tree users can narrow down from thousands of search results to a few in seconds.

GoPubMed retrieves PubMed abstracts for a search query and sorts relevant information to the 4 top level categories:

• What
• Who
• Where
• When

The seven deadly sins of site search according to Vivisimo

A Vivisimo white paper, available at searchdoneright.com lists the seven deadly sins of site search. It’s a basic, but useful introduction:

1. Omission - Make sure you offer search - some users prefer it and it’s an escape route when navigation fails.
2. Apathy - Make sure you check how search is functioning - relevancy, scent of information from titles an descriptions.
3. Complexity - Keep search interface simple - don’t offer >1 search box on a page.
4. Omnipotence - Beware the conceit that your engine will understand the user’s intent and respond with an ‘answer’ to a ‘question’.
5. Egoism - Don’t ask too much of a visitor - make it user-friendly.
6. Brand confusion - Ensure the search results has your brand and your URL.
7. Multiple personality - Ensure search pages have a consistent branding and look and feel to the rest of the site.

Eye tracking in MSN Search: Investigating snippet length, target position and task types

A Microsoft Research report using eye tracking techniques to investigate user strategies for web search.

Trust
How people respond to search results when the target is systematically manipulated to be displayed at different positions on two kinds of search tasks and found that users seem to exhibit an implicit trust for the rank generated by the search engine, particularly for informational tasks.

How varying the amount of information in Web search results affected user performance on the same tasks.

As the length of the query-dependent contextual snippet in search results was increased, performance improved for informational queries, while it degraded for navigational queries.

Eye tracking results suggest this difference in performance was due to the fact that as the snippet length increased, users paid more attention to the snippet and less attention to the URL located at the bottom of the search result.

Web search is a very attractive domain for the use of eye tracking techniques.

There are many kinds of metadata that are potentially useful for Web search. How can this information be presented to users in such a way that is complementary to existing information in search results?