Introduction
- 최근 딥러닝의 발전과 EHR와 같은 Biomedical data의 증폭은 다양한 의료분야에서 personalised predictions쪽으로 엄청난 발전을 가져옴.
- 이러한 발전은 events간의 Long-term dependency를 포착할 수 있게 함.
- 여러 EHR-specific challenges들과 질병 예측 정확도의 개선을 고려할 수 있는 Transformer architecture : 의료Domain BERT (for EHR)
Data
- CPRD(Clinical Practice Research Datalink)
- only for data linked to EHS records that can be mapped from ICD and Read Code to Caliber code
- more than 5 visits
Method
- 질병은 words로 each visit을 sentence로 전체 의료기록을 document로 묘사하여 기존 BERT의 multihead self-attention, positional encoding, MLM을 활용함.
- 임베딩 layer in BEHRT는 위 그림에서 보듯, 네 가지 임베딩의 combination으로 구함. (disease, position, age, visit segment)
- MLM : Pre-training
a. EHR 데이터에 대해서도, 직관적으로 deep bidirectional model이 한 방향 모델보다 우수한 성능을 낼 것이라는 것을 알 수 있기에, BEHRT를 BERT paper에서 학습 시켰던 MLM을 이용해 학습을 시킴.
b. 2의 4가지 임베딩을 랜덤으로 초기화하고 disease words를 86.5%(unchanged), 12%(replaced with mask token), 1.5% (randomly chosen disease words) 로 나눔.
c. 3의 세팅에서 BEHRT는 주변의 맥락적 표현을 저장하면서 학습을 시킴. - DownStream Task로 Disease Prediction 수행.
Result
2개의 댓글
A transformer for electronic health records revolutionizes healthcare management, particularly in contexts like private adult adhd assessment london. This advanced system employs AI to streamline record-keeping, ensuring accuracy and accessibility. By leveraging natural language processing, it interprets complex medical data, facilitating efficient diagnosis and treatment planning. Specifically, in London's private healthcare sector, where demand for ADHD assessments is significant, such technology enhances patient care by expediting processes and reducing errors. Moreover, it facilitates seamless communication among healthcare professionals, fostering collaborative care models. Ultimately, the transformer optimizes healthcare delivery, promoting better outcomes for individuals seeking private adult ADHD assessments in London.
Thank you, edenkim00, for the insightful review of BEHRT and its application as a Transformer for Electronic Health Records (EHRs). Your exploration of the advancements in deep learning and the amplification of biomedical data, particularly in personalized predictions within various medical fields, is commendable.
In the context of healthcare technology development, I'd like to recommend an article that discusses the integration of medical devices with Electronic Health Records (EHR): Medical Device Integration with EHR. This piece provides additional perspectives on how seamless integration contributes to the efficiency and accuracy of medical records.
Your review sheds light on the significance of addressing challenges specific to EHR and improving disease prediction accuracy through the Transformer architecture, such as the medical Domain BERT. I appreciate your contribution to the ongoing discussions about the intersection of deep learning and healthcare.