Quantifying Mental Health Signals in Twitter

Glen Coppersmith, Mark Dredze, and Craig Harman. 2014. Quantifying Mental Health Signals in Twitter. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 51–60, Baltimore, Maryland, USA. Association for Computational Linguistics.

---

Summary

• Introduction
  The study demonstrates quantifiable signals in Twitter data relevant to four mental illnesses: post-traumatic stress disorder (PTSD), depression, bipolar disorder, and seasonal affective disorder (SAD). It introduces a novel method for automatic data collection, rather than relying on surveys.

• Methodology
  The authors sought users who publicly stated that they have been diagnosed with mental illnesses by using regular expressions, and retrieved the recent tweets to build the "diagnosed group" data. The "control group" data was constituted with tweets of randomly selected users.

  The research validates its method by 1) replicating previous findings using LIWC and 2) constructing classifiers that can separate the diagnosed from the control users. Also, it takes an introspect on those classifiers to gain intuition.

• Result

  • 1) Validation
    It is demonstrated that language use, as measured by LIWC, is statistically significantly different between control and diagnosed users.

  • 2) Classification
    1-gram LM and character 5-gram LM were used. In addition, a non-verbal pattern of life analytics was also used including social engagement, insomnia, exercise, and sentiment analysis.
    -> LMs showed the ability to separate the class and also provided superior performance to the other analytics.

  • 3) Analytic Introspection
    The authors conducted correlation studies between the various analytics investigated to provide some insight.

• Conclusion
  The authors expect that these novel data collection methods can provide complementary information to existing survey-based methods.

---

Comments

Things to study more

• What is the Linguistic Inquiry Word Count (LIWC)?
  It is a well-known validated tool for psychometric analysis of language data and has been used to study language associated with all types of disorders.

Possible Future Research Ideas

• Better methods for automatic sample derivation
  To build the 'diagnosed group' data, the authors collected tweets using regular expressions, e.g. "I was diagnosed with X." The matched diagnosis tweets were manually labeled as to whether the tweet contained a genuine statement.
  I believe there could be some missing tweets left uncaptured, mainly because some tweets are written in incomplete sentences with shortened internet words.

• Investigate the relation between diseases in terms of linguistic content of the diagnosed users
  Correlation analysis of the linguistic content suggests that similar language is employed by users diagnosed with occasionally comorbid disorders. The relation between diseases in terms of the linguistic content of the diagnosed users could be worth investigating.

Questions

• What kind of language model do the recent studies use?
  The study used an n-gram model to classify the diagnosed group and the control group. It is said that the n-gram language models are less than ideal for applications in social media partly due to spelling errors, shortenings, space removal, and other aspects of social media data.
  Then, what kind of language model do recent studies leverage?

---

Things to note

• How to classify the diagnosed group and the control group
  The study used two n-gram LMs (ULM, CLM) for classification. The authors built one of each model from the positive class, yielding ULM+ and CLM+. They also built one of each model from the negative class (control users), yielding ULM- and CLM-. Then, they scored each tweet by computing probabilities and classifying it according to which model has a higher probability (e.g., is ULM+ > ULM-?).