Wednesday, March 26, 2008

How good is PubMed?

Around this time last year I wrote that while no search engine had a singular advantage in all situations PubMed was the most reliable. I still believe that’s true, at least for clinical medicine.

Last Saturday virologist PhD student and blogger Anna Kushnir expressed a different view:

For a site that is as vital to scientific progress as PubMed is, their search engine is shamefully bad. It’s embarrassingly, frustratingly, painfully bad.

She gave these examples:

I can hold a paper in my hands, search for two authors’ last names and have PubMed come up with nothing. My friend searched for microRNAs and her virus of interest. The search engine (can I even call it an engine? It’s more like a tricycle) came up with papers dating back to 1997. I am pretty sure no one knew about microRNAs in 1997. Yet another friend was only able to find publications about his compound of choice after empirically defining one of its functions in the cell… which is when he found out this information had been available all along. He couldn’t pull up the relevant papers without searching specifically for the compound and that one effect on the cell.

While the issues here may stem from limitations of PubMed they may also reflect a misunderstanding of search strategies.

She asks, for example:

How does it even work? Does it search only the abstract? Does it also search the body of the papers that are available online?

Those answers are readily available and there are two ways to know. You can tell PubMed exactly how to do the search through the use of Boolean operators and PubMed field tags or you can type in a simple free text search word or phrase and let PubMed decide the strategy. In the former case you know what PubMed did because you told it what to do. In the latter case you can find out by clicking the details tab on the results page to see the strategy. The information displayed there is extremely useful because it suggests ways to refine your search for better results.

The details display also reveals some of the hazards of simple free text searching. Here’s what PubMed did, for example, when I simply typed “diabetes” (click image to zoom):


It retrieved results for both diabetes mellitus and diabetes insipidus. Chances are I’d be looking for one or the other, not both. It also tells me it searched text words through the title and abstract in addition to applicable subject headings (MESH terms).

Sound complicated? You can always get help from your medical librarian but you don’t have to. You can learn to do it yourself. After some initial effort (take the tutorial) and a little practice it’s not that hard. If you can blog you can do PubMed. If you can navigate an electronic medical record you can do PubMed.

That said, I have little experience using PubMed for basic topics in molecular biology and the like, where its performance may not be as suitable as in clinical medicine.

Via White Coat Notes and Kevin M.D.

4 comments:

Martin said...

This girl is completely ignorant. There's a lot of training material available to master PubMed - and there are helpful tools like "clinical queries" and "citation matcher" which she obviously did not knew or used (though they're one click away). Pubmed is not Google and it does not make sense to use it like Google (though it does quite a lot of "search magic" with it's query translation).
There's a shortcoming of PubMed that the girl does not realize: PubMed contains only a subset of all available journals.

Anonymous said...

Couple of things:
Dr, RW, you're right - there are other NCBI database that may be more suitable for molecular biology questions, depending on what you're looking for.

"Pubmed is not Google" - Indeed. With a little bit of training, PubMed can be a very powerful tool. It is not the right/only tool for every single question you could possibly have, nor is Google.

As a medical librarian, I would not call Anna ignorant. She is frustrated because it didn't work how she expected and could use a bit of training on PubMed, as Dr RW mentioned. I personally offered to be available via email to give advice on search strategies in that thread, and repeat the same offer here - it's not my job to judge someone's approach, just to help them make it better.

Anonymous said...

I found some interesting information about Diabetes Insipidus here. Check it out!

Clinton said...

Perhaps the point is that a casual user ought to be able to master something like "appropriate" search terms within the Search Box of PubMed in a matter of minutes rather than poring through a long text on Boolean search strings and MESH terms.

After all, accessibility and speed are what matter most to people on the internet these days and if they can't find their answer within five minutes, an answer doesn't exist (to most.)

Surely it wouldn't be so difficult for PubMed to make their search box into something more friendly. Do we really need to click on Single Citation Matcher (SCM) in order to find that one article we're looking for because typing in the name of two authors, the journal and part of the title don't bring up the desired results?

Frankly, I've wasted my time doing exactly that. A lot of my classmates (MS-II, btw) don't even KNOW how to search for articles, much less know the little "tricks" like SCM.

I think that PubMed is archaic and could use an overhaul to make it more Web 2.0. (Of course, the old version could stay in place so people aren't totally disoriented with a major transition.) The equivalent of Diggs, tag clouds or shared bookmarklets could all benefit the scientific community!

-not my second opinion