local에서는 integration test가 성공하지만, 같은 환경의 CI에서는 일부 테스트가 실패하는 것에 대한 짧은 분석

문제 상황

local에서 실행했을 때는 잘 동작하던 integration test가 CI 에서는 동작하지 않는 오류가 발생했다. 조금 이상한 점은, CI 서버를 내 노트북으로 설정하고 실행했기 때문에 실행환경도 크게 다르지 않다는 점이었다.

테스트에서 발생한 오류의 스택 트레이스는 아래와 같다.

Warning: A call to tap() with finder "Found 1 widget with key [<'carrierDropdown'>]: [

[38](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:39) DropdownButton2<String>-[<'carrierDropdown'>](dependencies: [Directionality, DropdownButtonHideUnderline, InheritedCupertinoTheme, MediaQuery, _InheritedTheme, _LocalizationsScope-[GlobalKey#da70a]], state: DropdownButton2State<String>#1173b),

[39](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:40) ]" derived an Offset (Offset(215.0, 627.0)) that would not hit test on the specified widget.

[40](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:41) Maybe the widget is actually off-screen, or another widget is obscuring it, or the widget cannot receive pointer events.

[41](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:42) The finder corresponds to this RenderBox: RenderSemanticsAnnotations#53779 relayoutBoundary=up13

[42](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:43) The hit test result at that offset is: HitTestResult(_RenderInkFeatures#132d9@Offset(215.0, 627.0), RenderPhysicalModel#c8c43@Offset(215.0, 627.0), RenderSemanticsAnnotations#04e5f@Offset(215.0, 627.0), RenderRepaintBoundary#812db@Offset(215.0, 627.0), RenderIgnorePointer#2e28e@Offset(215.0, 627.0), RenderStack#ec5f7@Offset(215.0, 627.0), RenderDecoratedBox#91abd@Offset(215.0, 627.0), RenderRepaintBoundary#9c430@Offset(215.0, 627.0), RenderSemanticsAnnotations#d3208@Offset(215.0, 627.0), RenderOffstage#c6d1e@Offset(215.0, 627.0), RenderSemanticsAnnotations#ffbf3@Offset(215.0, 627.0), _RenderTheater#4c490@Offset(215.0, 627.0), RenderAbsorbPointer#0f5b6@Offset(215.0, 627.0), RenderPointerListener#8bc2d@Offset(215.0, 627.0), RenderSemanticsAnnotations#18524@Offset(215.0, 627.0), RenderCustomPaint#fcb1c@Offset(215.0, 627.0), RenderSemanticsAnnotations#8d411@Offset(215.0, 627.0), RenderSemanticsAnnotations#f564b@Offset(215.0, 627.0), RenderTapRegionSurface#9618f@Offset(215.0, 627.0), RenderSemanticsAnnotations#210d0@Offset(215.0, 627.0), RenderSemanticsAnnotations#2d55a@Offset(215.0, 627.0), RenderSemanticsAnnotations#97a8d@Offset(215.0, 627.0), RenderSemanticsAnnotations#9313f@Offset(215.0, 627.0), HitTestEntry<HitTestTarget>#a8cf7(_ReusableRenderView#a02cd NEEDS-LAYOUT), HitTestEntry<HitTestTarget>#3c1b2(<IntegrationTestWidgetsFlutterBinding>))

[43](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:44) #0 WidgetController._getElementPoint (package:flutter_test/src/controller.dart:2077:25)

[44](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:45) #1 WidgetController.getCenter (package:flutter_test/src/controller.dart:1861:12)

[45](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:46) #2 WidgetController.tap (package:flutter_test/src/controller.dart:1041:7)

[46](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:47) #3 main.<anonymous closure> (file:///Users/jeonghyohyeog/workspace/actions-runner/_work/disease_network_healthcare/disease_network_healthcare/integration_test/screens/sign_up/ui/sign_up_screen_test.dart:45:18)

[47](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:48) <asynchronous suspension>

[48](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:49) #4 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:193:15)

[49](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:50) <asynchronous suspension>

[50](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:51) #5 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1064:5)

[51](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:52) <asynchronous suspension>

[52](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:53) #6 StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:114:42)

[53](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:54) <asynchronous suspension>

[54](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:55) To silence this warning, pass "warnIfMissed: false" to "tap()".

[55](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:56) To make this warning fatal, set WidgetController.hitTestWarningShouldBeFatal to true.

[56](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:57)

[57](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:58) ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════

[58](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:59) The following StateError was thrown running a test:

[59](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:60) Bad state: No element

[60](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:61)

[61](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:62) When the exception was thrown, this was the stack:

[62](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:63) #0 Iterable.last (dart:core/iterable.dart:661:7)

[63](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:64) #1 _LastFinderMixin.filter (package:flutter_test/src/finders.dart:1367:28)

[64](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:65) #3 Iterable.isEmpty (dart:core/iterable.dart:542:33)

[65](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:66) #4 WidgetController._getElementPoint (package:flutter_test/src/controller.dart:2008:18)

[66](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:67) #5 WidgetController.getCenter (package:flutter_test/src/controller.dart:1861:12)

[67](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:68) #6 WidgetController.tap (package:flutter_test/src/controller.dart:1041:7)

[68](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:69) #7 main.<anonymous closure> (file:///Users/jeonghyohyeog/workspace/actions-runner/_work/disease_network_healthcare/disease_network_healthcare/integration_test/screens/sign_up/ui/sign_up_screen_test.dart:47:18)

[69](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:70) <asynchronous suspension>

[70](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:71) #8 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:193:15)

[71](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:72) <asynchronous suspension>

[72](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:73) #9 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1064:5)

[73](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:74) <asynchronous suspension>

[74](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:75) <asynchronous suspension>

[75](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:76) (elided 2 frames from dart:async-patch and package:stack_trace)

[76](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:77)

[77](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:78) The test description was:

[78](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:79) 올바른 입력 시 다음 버튼 활성화

[79](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:80) ════════════════════════════════════════════════════════════════════════════════════════════════════

[80](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:81) Test failed. See exception logs above.

[81](https://github.com/CryptoLabInc/disease_network_healthcare/actions/runs/13722494349/job/38381014067#step:11:82) The test description was: 올바른 입력 시 다음 버튼 활성화

Widget은 찾았지만 Widget의 위치(Offset(215.0, 627.0))때문에 tap을 진행할 수는 없는 상황이라고 한다.

문제 해결 방법

GPT에게 물어보니,

await tester.ensureVisible(find.byKey(const Key('carrierDropdown')));

ensureVisible 함수를 넣는 것을 추천했다. 해당 함수는 스크롤 가능하다면 스크롤해서 해당 키를 가진 위젯을 화면에 보이게 만들어주는 함수이다. 해당 함수를 추가하니 테스트가 문제 없이 실행되어 CI가 성공했다.

근본적인 문제 발생의 이유

그렇다면, 왜 이런 문제가 발생했을까? iphone 15 pro max의 viewport는 430px / 932px 이라서 해당 warning이 나올 수 없을 것 같은데 말이다. 다행히, CI 서버는 내 노트북이기 때문에 실제로 어떻게 Integration test가 돌고 있는지 확인해보았다.

      - name: Launch Simulator
        uses: futureware-tech/simulator-action@v3
        with:
          model: "iPhone 15 Pro Max"

해당 action을 이용해 시뮬레이터를 실행시키고 있었는데, 실제 GUI 없이 headless 모드로 실행되고 있었다. headless 모드로 실행된다면, TestWidgetsFlutterBinding의 설명에 따라 800px / 600px viewport 크기로 실행된다.

IntegrationTestWidgetsFlutterBinding도 TestWidgetsFlutterBinding을 상속받았으므로, 실행되는 뷰포트의 크기는 동일하다.

800px / 600px의 viewport 크기로 실행되었으므로 Offset(215.0, 627.0)의 위치에 있는 위젯에 탭이 불가능했고, tester.ensureVisible 메서드를 사용해 스크롤을 시켜 탭이 가능하게 만들어 테스트가 성공했던 것이다.

알게된 점

flutter test integration_test를 실행할 때는 반드시 실제 기기나 시뮬레이터가 필요했고, 실제로 테스트가 실행되는 모습이 보였기에 해당 문제의 발생이 의아했었다.

CI 환경에서는 시뮬레이터도 headless 모드로 실행되어 viewport의 크기가 자동으로 변경된다는 점(물론 수정할 수 있지만)을 알게 되어 좋았고 또 테스트할 때 유용한 메서드를 알게 되어서 좋았다. 어떤 문제를 겪고 어떻게 해결하게 될지, 역시 어렵지만 재미있다.