CTS 测试之Media相关测试failed 小结

CTS

CTS 即兼容性测试套件，CTS 在桌面设备上运行，并直接在连接的设备或模拟器上执行测试用例。CTS 是一套单元测试，旨在集成到工程师构建设备的日常工作流程（例如通过连续构建系统）中。其目的是尽早发现不兼容性，并确保软件在整个开发过程中保持兼容性。

CTS 是一个自动化测试工具，其中包括两个主要软件组件：

1. CTS tradefed 自动化测试框架会在桌面设备上运行，并管理测试执行情况。
2. 单独的测试用例会在被测设备 (DUT) 上执行。测试用例采用 Java 语言编写为 JUnit 测试，并打包为 Android .apk 文件，以在实际目标设备上运行。

最近遇到的Cts fails

android.media.cts.ExifInterfaceTest

android.media.cts.ExifInterfaceTest#testReadExifDataFromCanonG7XCr2 failjava.io.FileNotFoundException: /storage/emulated/0/test/images/canon_g7x.cr2 (No such file or directory)android.media.cts.ExifInterfaceTest#testReadExifDataFromExifByteOrderIIJpeg failjava.io.FileNotFoundException: /storage/emulated/0/test/images/image_exif_byte_order_ii.jpg (No such file or directory)android.media.cts.ExifInterfaceTest#testReadExifDataFromExifByteOrderMMJpeg failjava.io.FileNotFoundException: /storage/emulated/0/test/images/image_exif_byte_order_mm.jpg (No such file or directory)android.media.cts.ExifInterfaceTest#testReadExifDataFromFujiX20Raf fail

新的CTS Media Files，CTS Media 1.4： android-cts-media-1.4 中还有images资源，运行相关的测试需要先push 这些资源：执行 copy_images.sh

android.media.cts.MediaCodecCapabilitiesTest和android.media.cts.StreamingMediaPlayerTest

以上两个测试类共9条fail，failed原因基本都是“ Stream did not play successfully after all attempts”
可能的原因是测试网络的问题，这几个测试从测试类中可以看到测试访问的url来自：
android/cts/common/util/src/com/android/compatibility/common/util/DynamicConfig.java

CONFIG_FOLDER_ON_DEVICE = "/sdcard/dynamic-config-files/

从该目录导出CtsMediaTestCases.dynamic查看url，访问其中一条如下：
http://redirector.gvt1.com/videoplayback?id=271de9756065677e&amp，本地网络无法访问，可能需要翻墙测试，其他一样。

连接可以翻墙访问http://redirector.gvt1.com/videoplayback?id=271de9756065677e&amp的网络，测试pass

android.media.cts.VideoEncoderTest#testGoogVP9FlexNearMaxMax

12-19 14:33:22.378 10117 8402 8423 I TestRunner: failed: testGoogVP9FlexNearMaxMax(android.media.cts.VideoEncoderTest) 12-19 14:33:22.379 10117 8402 8423 I TestRunner: ----- begin exception ----- 12-19 14:33:22.380 10117 8402 8423 I TestRunner: junit.framework.AssertionFailedError: timed out after 5 decoder output and 0 encoder input buffers 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at junit.framework.Assert.fail(Assert.java:50) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest$VideoProcessor.processLoop(VideoEncoderTest.java:496) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest$Encoder.test(VideoEncoderTest.java:1142) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest$Encoder.test(VideoEncoderTest.java:1125) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest$Encoder.testExtreme(VideoEncoderTest.java:1018) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest.extreme(VideoEncoderTest.java:1738) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest.nearmaxmax(VideoEncoderTest.java:1728) 12-19 14:33:22.380 10117 8402 8423 I TestRunner: at android.media.cts.VideoEncoderTest.testGoogVP9FlexNearMaxMax(VideoEncoderTest.java:1401) 

测试平台8937/17/20/40 没有 VP9 encoder. 因此使用的是 google encoder .
Google encoder 不能满足desired frame rate.

解决办法： Google codec capabilities need to be restricted, please merge google’s patch as below to test again.

commit a221cb4ab070a2315cbeea965d0f74abf309c278Author: Lajos Molnar <[e-mail]>Date:   Fri Aug 11 16:29:28 2017 -0700    stagefright: update google video codec capabilities    VP9 encoder can only encode at 3-5pfs at 1080p on 2016 devices,    which is not really useful. Advertise only 720p support.    Increase advertised bitstream support to match level claimed    for HEVC decoder and AVC codecs that are used for thumbnail    extraction.    Bug: 64568747    Bug: 64425166    Bug: 31387661    Change-Id: I6e85613826b81fd92aa28c197ae9911be52476b5diff --git a/media/libstagefright/data/media_codecs_google_video.xml b/media/libstagefright/data/media_codecs_google_video.xmlindex ce164a2..a127843 100644--- a/media/libstagefright/data/media_codecs_google_video.xml+++ b/media/libstagefright/data/media_codecs_google_video.xml@@ -34,20 +34,21 @@             <Feature name="adaptive-playback" />         </MediaCodec>         <MediaCodec name="OMX.google.h264.decoder" type="video/avc">-            <!-- profiles and levels:  ProfileHigh : Level41 -->-            <Limit name="size" min="16x16" max="1920x1088" />+            <!-- profiles and levels:  ProfileHigh : Level52 -->+            <Limit name="size" min="2x2" max="4096x4096" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="blocks-per-second" range="1-244800" />-            <Limit name="bitrate" range="1-12000000" />+            <Limit name="block-count" range="1-32768" /> <!-- max 4096x2048 -->+            <Limit name="blocks-per-second" range="1-1966080" />+            <Limit name="bitrate" range="1-48000000" />             <Feature name="adaptive-playback" />         </MediaCodec>         <MediaCodec name="OMX.google.hevc.decoder" type="video/hevc">             <!-- profiles and levels:  ProfileMain : MainTierLevel51 -->-            <Limit name="size" min="2x2" max="2048x2048" />+            <Limit name="size" min="2x2" max="4096x4096" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="8x8" />-            <Limit name="block-count" range="1-139264" />+            <Limit name="block-count" range="1-196608" /> <!-- max 4096x3072 -->             <Limit name="blocks-per-second" range="1-2000000" />             <Limit name="bitrate" range="1-10000000" />             <Feature name="adaptive-playback" />@@ -56,6 +57,7 @@             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />+            <Limit name="block-count" range="1-16384" />             <Limit name="blocks-per-second" range="1-1000000" />             <Limit name="bitrate" range="1-40000000" />             <Feature name="adaptive-playback" />@@ -64,6 +66,7 @@             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />+            <Limit name="block-count" range="1-16384" />             <Limit name="blocks-per-second" range="1-500000" />             <Limit name="bitrate" range="1-40000000" />             <Feature name="adaptive-playback" />@@ -79,10 +82,11 @@         </MediaCodec>         <MediaCodec name="OMX.google.h264.encoder" type="video/avc">             <!-- profiles and levels:  ProfileBaseline : Level41 -->-            <Limit name="size" min="16x16" max="1920x1088" />+            <Limit name="size" min="16x16" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="blocks-per-second" range="1-244800" />+            <Limit name="block-count" range="1-8192" /> <!-- max 2048x1024 -->+            <Limit name="blocks-per-second" range="1-245760" />             <Limit name="bitrate" range="1-12000000" />             <Feature name="intra-refresh" />         </MediaCodec>@@ -98,6 +102,9 @@             <!-- profiles and levels:  ProfileMain : Level_Version0-3 -->             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />+            <Limit name="block-size" value="16x16" />+            <!-- 2016 devices can encode at about 10fps at this block count -->+            <Limit name="block-count" range="1-16384" />             <Limit name="bitrate" range="1-40000000" />             <Feature name="bitrate-modes" value="VBR,CBR" />         </MediaCodec>@@ -105,6 +112,9 @@             <!-- profiles and levels:  ProfileMain : Level_Version0-3 -->             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />+            <Limit name="block-size" value="16x16" />+            <!-- 2016 devices can encode at about 8fps at this block count -->+            <Limit name="block-count" range="1-3600" /> <!-- max 1280x720 -->             <Limit name="bitrate" range="1-40000000" />             <Feature name="bitrate-modes" value="VBR,CBR" />         </MediaCodec>diff --git a/media/libstagefright/data/media_codecs_google_video_le.xml b/media/libstagefright/data/media_codecs_google_video_le.xmlindex 034a038..d7c6570 100644--- a/media/libstagefright/data/media_codecs_google_video_le.xml+++ b/media/libstagefright/data/media_codecs_google_video_le.xml@@ -34,22 +34,22 @@             <Feature name="adaptive-playback" />         </MediaCodec>         <MediaCodec name="OMX.google.h264.decoder" type="video/avc">-            <!-- profiles and levels:  ProfileBaseline : Level51 -->+            <!-- profiles and levels:  ProfileHigh : Level51 -->             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="block-count" range="1-8160" />-            <Limit name="blocks-per-second" range="1-489600" />+            <Limit name="block-count" range="1-16384" />+            <Limit name="blocks-per-second" range="1-491520" />             <Limit name="bitrate" range="1-40000000" />             <Feature name="adaptive-playback" />         </MediaCodec>         <MediaCodec name="OMX.google.hevc.decoder" type="video/hevc">             <!-- profiles and levels:  ProfileMain : MainTierLevel51 -->-            <Limit name="size" min="2x2" max="1280x1280" />+            <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="8x8" />-            <Limit name="block-count" range="1-139264" />-            <Limit name="blocks-per-second" range="1-432000" />+            <Limit name="block-count" range="1-65536" />+            <Limit name="blocks-per-second" range="1-491520" />             <Limit name="bitrate" range="1-5000000" />             <Feature name="adaptive-playback" />         </MediaCodec>@@ -57,7 +57,7 @@             <Limit name="size" min="2x2" max="2048x2048" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="block-count" range="1-8160" />+            <Limit name="block-count" range="1-8192" /> <!-- max 2048x1024 -->             <Limit name="blocks-per-second" range="1-500000" />             <Limit name="bitrate" range="1-40000000" />             <Feature name="adaptive-playback" />@@ -66,7 +66,7 @@             <Limit name="size" min="2x2" max="1280x1280" />             <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="block-count" range="1-3600" />+            <Limit name="block-count" range="1-3600" /> <!-- max 1280x720 -->             <Limit name="blocks-per-second" range="1-108000" />             <Limit name="bitrate" range="1-5000000" />             <Feature name="adaptive-playback" />@@ -81,12 +81,14 @@             <Limit name="bitrate" range="1-128000" />         </MediaCodec>         <MediaCodec name="OMX.google.h264.encoder" type="video/avc">-            <!-- profiles and levels:  ProfileBaseline : Level2 -->-            <Limit name="size" min="16x16" max="896x896" />-            <Limit name="alignment" value="16x16" />+            <!-- profiles and levels:  ProfileBaseline : Level3 -->+            <Limit name="size" min="16x16" max="1808x1808" />+            <Limit name="alignment" value="2x2" />             <Limit name="block-size" value="16x16" />-            <Limit name="blocks-per-second" range="1-11880" />+            <Limit name="block-count" range="1-1620" />+            <Limit name="blocks-per-second" range="1-40500" />             <Limit name="bitrate" range="1-2000000" />+            <Feature name="intra-refresh" />         </MediaCodec>         <MediaCodec name="OMX.google.mpeg4.encoder" type="video/mp4v-es">             <!-- profiles and levels:  ProfileCore : Level2 -->@@ -100,7 +102,8 @@             <!-- profiles and levels:  ProfileMain : Level_Version0-3 -->             <Limit name="size" min="2x2" max="1280x1280" />             <Limit name="alignment" value="2x2" />-            <Limit name="block-count" range="1-3600" />+            <Limit name="block-size" value="16x16" />+            <Limit name="block-count" range="1-3600" /> <!-- max 1280x720 -->             <Limit name="bitrate" range="1-20000000" />             <Feature name="bitrate-modes" value="VBR,CBR" />         </MediaCodec>

android.provider.cts.MediaStoreUiTest#testGetDocumentUri

android.provider.cts.MediaStoreUiTest#testGetDocumentUri failjava.lang.NullPointerException: Attempt to invoke virtual method 'android.support.test.uiautomator.UiObject2 android.support.test.uiautomator.UiObject2.findObject(android.support.test.uiautomator.BySelector)' on a null object reference

很明显的空指针异常
从android/cts/tests/tests/provider/src/android/provider/cts/MediaStoreUiTest.java如下测试代码可以发现此testGetDocumentUri测试出现NullPointerException是因为在com.android.documentsui中找不到id为buttonPanel的控件返回的buttonPanel是null；

// Granting the accessBySelector buttonPanelSelector = By.pkg("com.android.documentsui").res("android:id/buttonPanel");mDevice.wait(Until.hasObject(buttonPanelSelector), 30 * DateUtils.SECOND_IN_MILLIS);final UiObject2 buttonPanel = mDevice.findObject(buttonPanelSelector);final UiObject2 allowButton = buttonPanel.findObject(By.res("android:id/button1"));

测试Nexus android O发现确实testGetDocumentUri是pass的，那么获取原生最新的DocumentsUI，编译push到手机system/priv-app/DocumentsUI/目录下测试发现failed。

从Nexus中通过adb pull system/priv-app/DocumentsUI/DocumentsUI.apk 获取DocumentsUI.apk然后反编译；在反编译的结果中通过查找可以找到buttonPanel如下：

./res/layout/abc_alert_dialog_button_bar_material.xml:<ScrollView android:id="@id/buttonPanel" android:layout_width="fill_parent" android:layout_height="wrap_content" ......

进入项目的DocumentsUI的layout目录是找不到abc_alert_dialog_button_bar_material.xml的
在反编译的DocumentsUI中继续查找abc_alert_dialog_button_bar_material得到：

./res/layout/abc_alert_dialog_material.xml: <include android:layout_width="fill_parent" android:layout_height="wrap_content" layout="@layout/abc_alert_dialog_button_bar_material" />

接着查abc_alert_dialog_material得到结果：

./res/values/styles.xml: <style name="Base.AlertDialog.AppCompat" parent="@android:style/Widget"><item name="android:layout">@layout/abc_alert_dialog_material</item><item name="listItemLayout">@layout/select_dialog_item_material</item><item name="listLayout">@layout/abc_select_dialog_material</item><item name="multiChoiceItemLayout">@layout/select_dialog_multichoice_material</item><item name="singleChoiceItemLayout">@layout/select_dialog_singlechoice_material</item></style>

继续查找：

<style name="AlertDialog.AppCompat" parent="@style/Base.AlertDialog.AppCompat" />

我们查看源码发现AlertDialog.AppCompat是android.support.v7.app.AlertDialog引入的,我们在项目源码中搜索abc_alert_dialog_button_bar_material.xml，发现也是在android/frameworks/support/v7/appcompat/res/layout/abc_alert_dialog_button_bar_material.xml，进一步印证AlertDialog.AppCompat是android.support.v7.app.AlertDialog引入的.

奇怪的是DocumentUri中的AlertDialog有一处使用android.support.v7.app.AlertDialog，其他使用的是android.app.AlertDialog,将其他改为android.support.v7.app.AlertDialog，经过测试发现仍然没有pass。

那么从手机中获取项目的DocumentUri.apk，发现system/priv-app/下没有DocumentsUI，进一步查找发现项目使用的是自己在DocumentsUI基础上开发的新DocumentsUI，在另外一个目录，这个新DocumentsUI删除，替换为原生system/priv-app/DocumentsUI/DocumentsUI.apk，经过测试pass