Google Flu Trends is not as accurate at estimating rates of laboratory-confirmed influenza as CDC national surveillance programs, according to a new study from the University of Washington.
“We knew from the Google Flu Trends validation study that it is highly correlated with surveillance for the non-specific syndrome of influenza-like illness,” said Justin Ortiz, M.D., clinical fellow at the University of Washington who led the study. “However, it has never been evaluated against a gold standard of actual laboratory tests positive for influenza virus infection. When we compared Google Flu Trends data to CDC’s national surveillance for influenza laboratory tests positive, we found that Google Flu Trends was 25 percent less accurate at estimating rates of laboratory confirmed influenza virus infection.”
Google Flu Trends uses the popularity of certain Google search queries in real time to estimate nationwide rates of influenza-like illness activity, a non-specific combination of symptoms including a fever with either a cough or a sore throat without any confirmatory laboratory testing. While some traditional flu surveillance systems may take days or weeks to collect and release data, Google search queries can be counted almost instantaneously.
The problem is that studies have shown that influenza-like illnesses are actually caused by the influenza virus in only 20 percent to 70 percent of cases during the influenza season.
[Snip]
“Internet search behavior is likely different during anomalous seasons such as during 2003-4,” explained Dr. Ortiz. “We hypothesize that during periods of intense media interest or unexpected influenza activity such as the 2009 H1N1 influenza pandemic, Google Flu Trends may be least accurate at estimating influenza activity.”
“Still, Google Flu Trends influenza surveillance provides an excellent public health service, because it provides nationwide influenza activity data in a cheap and timely manner,” said Dr. Ortiz. “Nevertheless, our study demonstrates that its data should be interpreted with caution and that other surveillance systems more accurately reflect influenza activity in the United States.”
So, we have the doctor who lead of the U. of Washington study saying that while the estimates are off Google Flu Trends is still an "excellent public health service" because of it costs very little and the data is very timely. OK, if that's what someone who has spent hours pouring over the data says, let's take him at his word.
However, it once again illustrates the need for end users to have top-notch information literacy skills. The announcement itself includes Dr. Ortiz saying, "... [the] data should be interpreted with caution and that other surveillance systems more accurately reflect influenza activity in the United States.”
So are people, of all ages (particularly younger people) being taught how to proceed with caution with information they find on the web, in a magazine, see on tv, etc.? Do they know how to identify and access other sources (let's say a reference book or a remotely accessible database from their school or public library) and do they know about their existence in the first place? Can they quickly take a step back from the information and ask themselves where is it coming from, its currency, what's the reputation of the publisher and/or researcher, etc.? Knowledge of all of this is also essential for those who are assigning and grading the work.
Can it be demonstrated that while much of what you find is current and accurate much of what you can find and access on the open web is usable and accurate info BUT at other times and for a variety of reasons it's out of date, factually incorrect or even purposely posted to mislead? Perhaps the most important point with this type of research is that those doing it use several sources from a variety of organizations and publishers. Never rely on only one and to have an idea about key resources before the research process begins.
Finally, for many people in the general public (and the media) the company that provides access to a lot of web info, Google, has such a four-star stellar reputation and brand. However, users don't realize how Google works and that Google is most often the messenger and not the message. Just "Googling it" and getting results doesn't say all that needs to be said and understood about the quality, accuracy, reputation, etc. of the data being reviewed.
Note: Google Flu Trends offers info for 121 countries. We would love to see how the accuracy of these reports vs. what has been found for the U.S.
The FreePint Family is a family of resources to help information workers be more effective, raise the value of information in their organisations and contribute to success.
'FreePint... provides most of my professional development because it won't come through work and [other resources] just don't cut it.'
FUMSI Forum: Do you have a research question? Post it to the FUMSI Forum, where professionals share Q&A and useful tips on how to Find, Use, Manage and Share Information. It's free.
FreePint Shop:
VIP LiveWire:
