Methods in predictive techniques for mental health status on social media: a critical review

Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: A critical review. Npj Digital Medicine, 3(1), 1-11. https://doi.org/10.1038/s41746-020-0233-7

---

Summary

• Introduction
  Computer scientists are now using quantitative techniques to predict the presence of specific mental disorders and symptomatology, as this research promises great benefits to monitoring effects, diagnostics, and intervention design.
  Yet, there is no standardized process for evaluating the validity of this research and the methods. Thus, the authors conducted a systematic literature review of the state-of-the-art in predicting mental health status using social media data.

• Methodology
  The research found 75 studies and outlined the methods of data annotation for mental health status, data collection and quality management, pre-processing and feature selection, and model selection and verification.

• Result
  The researchers identified concerning trends around construct validity, and a lack of reflection in the methods used to operationalize and identify mental health status.

• Conclusion
  The study suggests some recommendations to address these challenges, including a list of proposed reporting standards for publications and collaboration opportunities in this interdisciplinary space.

---

Comments

Things to study more

• Previous study about suicidality
  One previous study studied suicidality by distinguishing between suicidal ideation and other discussions of suicide. I wonder what logic or models the research used. -> Could also be leveraged as a future research idea

  Burnap, P., Colombo, W. & Scourfield, J. Machine Classification and analysis of suicide-related communication on Twitter. In Proc. ACM Conf. of HyperText (HT). 75–84. (ACM, 2015).

• Previous study of cognitive distortions
  Of studies that analyzed symptomatology related to mental disorders, one research focused on cognitive distortions. I wonder based on what logic they identify cognitive distortions.

  Simms, T. et al. Detecting cognitive distortions through machine learning text analytics. In Proc. 2017 IEEE International Conference on Healthcare Informatics (ICHI). http://ieeexplore.ieee.org/abstract/document/8031202/. (IEEE, 2017).

• Previous study of applying DL methods

  • One research used deep learning approaches to model language through convolutional neural networks.

    Yates, A., Cohan, A. & Goharian, N. Depression and self-harm risk assessment in online forums. In Proc. 2017 Conference on Empirical Methods in Natural Language Processing 2968–2978 (ACL, 2017).

  • This research adopted recurrent neural networks as their algorithms.

    Ive, J., Gkotsis, G., Dutta, R., Stewart, R. & Velupillai, S. Hierarchical neural model with attention mechanisms for the classification of social media text related to mental health. In Proc. Fifth Workshop on Computational Linguistics and Clinical Psychology 69–77 (ACL, 2018).

  • This research chose a multitask neural network to share information between prediction tasks.

    • Benton, A., Mitchell, M. & Hovy, D. Multitask learning for mental health conditions with limited social media data. In Proc. 15th Conference of the European
      Chapter of the Association for Computational Linguistics: Volume 1. http://www.
      aclweb.org/anthology/E17-1015. (ACL, 2017).
    • Lin, H., Jia, J., Nie, L., Shen, G. & Chua, T.-S. What Does Social Media Say about Your Stress?. In Proc.Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) http://www.ijcai.org/Proceedings/16/Papers/531.pdf. (IJCAI, 2016).

• Incorporating image features
  Some researchers extracted visual information from the images of posts for prediction tasks. This information includes color themes/Hue-Saturation-Value values, if the images include a face, brightness and saturation values, and the types of colors used.

  • Reece, A. G. & Danforth, C. M. Instagram photos reveal predictive markers of depression. EPJ Data Science 6, 1–34 (2017).
  • Shen, G. et al. Depression detection via harvesting social media: A multimodal dictionary learning solution. In Proc. Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI) (IJCAI, 2017).

Possible Future Research Ideas

• How can we figure out which data annotation method has the strongest validity?
  There exist various data annotation methods. (Human assessments, community or network affiliations, self-disclosure, administering screening questionnaires, keyword use, news reports, and medical diagnostic codes.) Which strategy holds the strongest validity? What could be a way to calculate the validity of data annotation methods?
  Also, the study comments that there was no reflection across the documents on what ground truth approach was appropriate for establishing construct validity. There was also no validation of applying constructs to social media data, for instance, how strongly clinically valid screening questionnaires evaluate social media data.

• How can we figure out what kind of control data/negative examples sourcing method has the strongest validity?
  Various methods exist of how to source control data or negative examples. (validated to no MHS, random selection of control users, lack of mental health disclosure, and matching strategies) Which strategy holds the strongest validity? What could be a way to calculate the validity of each method?

• Idea of using DSM-5 for assessment
  The research emphasizes that this field of research should be established with stronger connections to the traditions of clinical psychiatry. Once, I used DSM-5 to operationalize symptoms and automatic assessments. How can this be improved?

Questions

• What is the ICD-10?
  One study used ICD-10 codes from patient health records to detect depression during the data annotation process.
  ICD-10 is the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD), a medical classification list by the World Health Organization (WHO). It contains codes for diseases, signs and symptoms, abnormal findings, complaints, social circumstances, and external causes of injury or diseases.

---

Things to note

• How previous studies conceptualize their research questions
  It is notable that almost all papers conceptualized their research questions as a classification problem, such as the categorical distinction between high and low stress. Only six papers used a model that predicts continuous or discrete values.

• Features for predictions
  The study summarizes variables used for prediction. Followings are features used.

1. Language Features
   a. structural/syntactic
   b. character and word models
   c. topical
   d. linguistic style
   e. domain-specific
   f. general language measures
2. Behavior
   a. activity
   b. interaction
   c. network
   d. domain-specific
3. Emotion and Cognition
   a. sentiment, affect and valence
   (* Valence describes the extent to which an emotion is positive or negative, whereas arousal refers to its intensity.)
   b. psycholinguistic
   c. domain-specific
4. Demographic Features
5. Image Features

• How to operationalize MHS better
  • Draw on relevant literature from clinical psychiatry and psychology. (e.g. assessing MHS using administering screeners like CES-D, and ICD-10)
  • Collaborate with domain experts to guide the operationalization process for MHS
    (e.g. One study partnered with an expert on suicidality to build a classifier that distinguishes tweets about suicide.)

    Burnap, P., Colombo, W. & Scourfield, J. Machine Classification and analysis of suicide-related communication on Twitter. In Proc. ACM Conf. of HyperText (HT). 75–84. (ACM, 2015)