Flutter driver 初探
原帖:http://andward.coding.me/flutter/test/2019/01/22/dig-with-flutter-driver.html
Flutter 1.0問世後關注度越來越高,在github上已經5萬顆星星了。前些日子看了下它的test tutorial已然支援了integration test -> flutter driver。起初看名字以為是flutter版的wedriver(再包裝),研究了才發現另有一番天地。
Flutter的test分三個級別:
- Unit test。單元測試,函式級別的邏輯驗證(no-UI)。
- Widget test。元件測試,元件的互動測試。
- Integration test。整合測試,app內widgets的整合測試
說重點:
三種測試基礎環境是一樣的
Unit test呼叫的是'package:test'。原文:The package lets you run your tests in a local Dart VM with a headless version of the Flutter Engine, which supplies these libraries. Using this command you can run any test, whether it depends on Flutter libraries or not. Unit跑測試時default option是Dart VM, 也可以選擇跑在某一個browser或者platform。具體介紹可以看測試文件 。
Widget test是UT的升級版,呼叫的'package:flutter_test'。Widget元件是flutter的佈局核心(萬物皆是widget),因此Widget test在flutter test中佔有重要的地位。Widget test在VM(沒錯,widget test也跑在Dart VM裡面)通過flutter engine render一個互動的視覺化元件(UI)直接測試渲染和互動效果,準確性和效率都能得到保證。
Integration test是widget test的拓展版(後文分析),呼叫的是'package:flutter_driver'。其實整個flutter app就是一個大的widget,所以integration test主要聚焦widget聚合後的互動表現。flutter driver test build-in在專案裡,執行時test app也會在Dart VM裡面render,通過test script與其互動。
三種test物件都(可以)在VM裡面執行,那麼怎麼跟VM互動呢?答案是Dart VM Service Protocol 。它封裝了JSON-RPC 2.0,來處理WebSocket request/response。
Service extension模式
我們來看例子。Flutter test的目錄結構大概長這樣:
-
lib
- main.dart
-
test_driver
- tap.dart
- tap_test.dart
'main.dart'是app的入口。Folder 'test_driver'用來專門用來裝integration test。
'tap.dart':
import 'package:flutter_driver/driver_extension.dart'; import 'package:flutter_todo/main.dart' as app; void main() { enableFlutterDriverExtension(); app.main(); }
'tap.dart'是flutter driver的target file,它先enable了FlutterDriverExtension,再啟動了app。enableFlutterDriverExtension()做了些什麼呢:
註冊了名為'ext.flutter.driver'的service extension。它呼叫了'flutter/foundation/binding.dart'中的registerServiceExtension(最終是呼叫dart/developer/extension中的service protocol extension handler),是_DriverBinding的一個function。
... void initServiceExtensions() { super.initServiceExtensions(); final FlutterDriverExtension extension = FlutterDriverExtension(_handler, _silenceErrors); registerServiceExtension( name: _extensionMethodName, callback: extension.call, ); } ...
啟用Flutter Driver VM service extension。
... void enableFlutterDriverExtension({ DataHandler handler, bool silenceErrors = false }) { assert(WidgetsBinding.instance == null); _DriverBinding(handler, silenceErrors); assert(WidgetsBinding.instance is _DriverBinding); } ...
接收來自FlutterDriver的command去呼叫相應的處理。程式碼太長不貼了,簡單的說就是定義了_commandHandlers, _commandDeserializers和_finders三個map對映來接收command,去呼叫它們的對映函式。handle這個過程的就是extension.call。所有在FlutterDriver中的方法最終都通過command對映到extension中對應的函式。
'tap_test.dart'是test script,下面是一個簡單的flow。
'tap_test.dart':
import 'dart:async'; // Imports the Flutter Driver API import 'package:flutter_driver/flutter_driver.dart'; import 'package:test/test.dart'; void main() { group('Tap test', () { FlutterDriver driver; setUpAll(() async { // Connects to the app driver = await FlutterDriver.connect(); }); tearDownAll(() async { if (driver != null) { // Closes the connection driver.close(); } }); test('measure', () async { // Record the performance timeline of things that happen inside the closure Timeline timeline = await driver.traceAction(() async { // Find the press button SerializableFinder pressButton = find.byTooltip('Increment'); // Scroll down 5 times for (int i = 0; i < 5; i++) { // Tap for 300 millis await driver.tap( pressButton, timeout: Duration(milliseconds: 300)); } }); // The `timeline` object contains all the performance data recorded during // the session. It can be digested into a handful of useful // aggregate numbers, such as "average frame build time". TimelineSummary summary = TimelineSummary.summarize(timeline); // The following line saves the timeline summary to a JSON file. summary.writeSummaryToFile('tap_performance', pretty: true); // The following line saves the raw timeline data as JSON. summary.writeTimelineToFile('tap_performance', pretty: true); }); }); }
初始化FlutterDriver後,driver做了幾件事:
連線VM裡面執行的test app。'connect()'中的關鍵部分如下,它去嘗試連線VMServiceClient,拿到app的isolate(isolate可以認為是dart裡面的多執行緒):
... final VMServiceClientConnection connection = await vmServiceConnectFunction(dartVmServiceUrl); final VMServiceClient client = connection.client; final VM vm = await client.getVM(); final VMIsolateRef isolateRef = isolateNumber == null ? vm.isolates.first : vm.isolates.firstWhere( (VMIsolateRef isolate) => isolate.number == isolateNumber); _log.trace('Isolate found with number: ${isolateRef.number}'); VMIsolate isolate = await isolateRef .loadRunnable() .timeout(isolateReadyTimeout, onTimeout: () { throw TimeoutException( 'Timeout while waiting for the isolate to become runnable'); }); ...
在isolate中呼叫invokeExtension,將script裡面的finder/interaction通過extension轉化成command,來呼叫對映函式完成互動:
... Future<Map<String, dynamic>> _sendCommand(Command command) async { Map<String, dynamic> response; try { final Map<String, String> serialized = command.serialize(); _logCommunication('>>> $serialized'); response = await _appIsolate .invokeExtension(_flutterExtensionMethodName, serialized) .timeout(command.timeout + _rpcGraceTime(timeoutMultiplier)); _logCommunication('<<< $response'); } on TimeoutException catch (error, stackTrace) { ... } catch (error, stackTrace) { ... } ... return response['response']; } ...
一些細節
extension中查詢元素都是通過flutter test的find實現的。它可以在render tree中搜索想要的widget。具體細節可以參考'flutter_test/lib/finder.dart'。
extension中的互動都是通過LiveWidgetController中的方式實現的。具體細節可以參考'flutter_test/lib/controller.dart'
方法forceGC()往VM發一條‘_collectAllGarbage’的命令強行GC。是不是remote interaction容易memory leak?
方法traceAction()收集performance timeline,提供了一個容易的方法track performance。結果長如下的樣子:
{ "average_frame_build_time_millis": 12.4331875, "90th_percentile_frame_build_time_millis": 18.181, "99th_percentile_frame_build_time_millis": 160.043, "worst_frame_build_time_millis": 160.043, "missed_frame_build_budget_count": 8, "average_frame_rasterizer_time_millis": 9.003393939393936, "90th_percentile_frame_rasterizer_time_millis": 9.284, "99th_percentile_frame_rasterizer_time_millis": 129.851, "worst_frame_rasterizer_time_millis": 129.851, "missed_frame_rasterizer_budget_count": 4, "frame_count": 32, "frame_build_times": [ 160043, 6363, 20967, ... ], "frame_rasterizer_times": [ 129851, 15393, ... ] }
有待更深入發掘...