Session 5: Assistive & Gaze

Visual Hearing Aids: Artificial Visual Speech Stimuli for Audiovisual Speech Perception in Noise

• poor audio or noisy environment - but speaker의 입모양으로 판단: Unity에서 Lip Sync softwawre with virtual human 가능

conditions

• 3 visualization : virtual face, virtual lips, oval shape
• 2 baseline : audio only, real face

Modulation techniques

• amplitude based (virtual human - speech waveform - primitive shape)
• viseme-based: SALSA LipSunc Suite (real-time lip-sync for virutal human)

design

: 3가지 숫자를 virtual human이 말한 다음 참가자는 이에 맞는 숫자를 keypad로 입력 이때, 맞으면 2dB 어렵게, 틀리면 2dB 쉽게 설정함

Results

SRT (lower is better) = speech dB - noise dB : audio only에서 가장 높음

• oval, virtuallips, aplitude modulation이 audio-only보다 imporve됨
• Lip sync does not provide helpful moth cue(viseme-based는 performed poorer)
  

---

Music Therapy in Virtual Reality for Autistic Children with Severe Learning Disabilities

• Autistic learning disability: low functioning verbal communication
• Music Therapy: 음악으로 social interaction(communication) - mirroring (하나, 둘 셋,..), joint attention, expectation building 가능
• 악기 연주자, music therapist모두 HMD를 착용 - social communication

---

Gaze Assistance for Older Adults during Throwing Training in Virtual Reality and its Effects on Performance and Motivation

Pre-study

• target을 향해 눈뭉치를 던지는 상황에서 record eye movement - Gaze assistane influence

---

GazeRayCursor: Facilitating Virtual Reality Target Selection by Blending Gaze and Controller Raycasting

• disambiguation in 3D Environments : manual depth adjustment, declustering area
  

• 기존의 gaze-assisted selection : gaze를 preselection방법으로 사용
  

Design

• convergence-based raycasting
• gaze raycursor - gaze ray, controlller ray

Study1

exploring combinaiton of gaze and controller ray

Results

CGC가 best performance

Study2

Design and evaluation of GazeRayCursor
cf. baseline: raycursor : controller touchpad로 depth 설정

Results

conclusion