Using Twitter as a Trendspotting Tool
Mark Dredze and Michael J. Paul fed 2 billion public tweets posted between May 2009 and October 2010 into computers. Then, they used software to filter out the 1.5 million messages that referred to health matters.
By organizing health-related tweets into categories, Dredze and Paul were able to uncover interesting patterns about allergies, depression, cancer, obesity and other health problems.
In addition, they were able to observe which medications ill tweeters used and identify patterns of misuse (taking antibiotics for the flu, e.g.). They were even able to track some trends by time and place –such as when the allergy and flu seasons peaked in various parts of the country.
“When we started, I didn’t even know if people talked about allergies on Twitter,” said Paul. “But we found out that they do. And there was one thing I didn’t expect: The system found two different types of allergies: the type that causes sniffling and sneezing and the kind that causes skin rashes and hives.”
Not surprisingly, Dredze and Paul also discovered that using Twitter as a public health research tool has its limitations. For example,Twitter users did not comment more than once on their particular condition, making it difficult to track how long the illness lasted and whether it recurred. Twitter users also are primarily young and are tweeting from the US, making Twitter less useful for observing trends in other countries. Further, Twitter research may only reach a certain depth, because of the superficial nature of personal health tweets.
Still, I can’t help but get excited about Twitter’s potential as a market research tool. With Twitter adoption still on the rise and users now sending some 200 million tweets per day, it’s clear we have only begun to scratch the surface of what all this data can reveal.