Quality Summary ------------------------------------------- QA complete Crosswalk App Tools Fullpass Testing based on Commit ID f356ec0538ca2452bfb7207907604c662e0cc261. Glad to see that multiple backends is implemented successfully (Fixed XWALK-4499<https://crosswalk-project.org/jira/browse/XWALK-4499>). However, although multiple backends works fine on Windows and Linux, it still fail on OSX (XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>). Besides, six new issues include five regression issues found:[REG] platform cannot work for windows platform(XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>);[REG]Build both x86 and arm apk using crosswalk-pkg with "--targets=x86"(XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>);[REG] Fail to build 64bit apk when use crosswalk-pkg with --targets=x86_64(XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> );[REG] The ios project name lack of its version after building on ios platform(XWALK-6059<https://crosswalk-project.org/jira/browse/XWALK-6059>);[REG] Fail to build .deb project on Deepin 64bit OS(XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064>); Crosswalk-pkg is not working on IOS platform(XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>).
The overall test results: Test Scope Windows 8.1 Windows 10 OS X Deepin Comment Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) App Tools Android 32-bit Tests(Embedded) N/A N/A N/A 276 93% -5% 276 93% -5% N/A N/A N/A Bugs on Windows/IOS:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254>,XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-5264<https://crosswalk-project.org/jira/browse/XWALK-5264> Passrate downgrade due to XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012> App Tools Android 32-bit Tests(Shared) N/A N/A N/A 276 93% -5% 276 93% -5% N/A N/A N/A Bugs on Windows/IOS:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254>,XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-5264<https://crosswalk-project.org/jira/browse/XWALK-5264> Passrate downgrade due to XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012> App Tools Android 64-bit Tests(Embedded) N/A N/A N/A 276 86% -11% 276 86% -11% N/A N/A N/A Bugs on Windows/IOS:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254>,XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-5264<https://crosswalk-project.org/jira/browse/XWALK-5264>, XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> Passrate downgrade due to XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> App Tools Android 64-bit Tests (Shared) N/A N/A N/A 276 86% -11% 276 86% -11% N/A N/A N/A Bugs on Windows/IOS:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254>,XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-5264<https://crosswalk-project.org/jira/browse/XWALK-5264>, XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> Passrate downgrade due to XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> App Tools SampleApp Tests N/A N/A N/A 10 100% -- 10 100% -- 2 0% -100% Bugs on Deepin: XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> Passrate downgrade due to XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> App Tools UseCase Tests N/A N/A N/A 10 90% -10% 10 100% -- 1 0% -100% Bugs on Windows/Deepin: XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> Passrate downgrade due to XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> App Tools Tests 141 81% -14% 141 81% -14% 117 87% -13% 113 75% -25% Bugs on Windows/IOS/Deepin: XWALK-4916<https://crosswalk-project.org/jira/browse/XWALK-4916>, XWALK-4835<https://crosswalk-project.org/jira/browse/XWALK-4835>,XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6059<https://crosswalk-project.org/jira/browse/XWALK-6059>, XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> Passrate downgrade due to XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6059<https://crosswalk-project.org/jira/browse/XWALK-6059>,XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> Total 141 81% -14% 1265 90% -7% 1241 91% -6% 116 73% -27% Passrate downgrade due to XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016>, XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> ,XWALK-6059<https://crosswalk-project.org/jira/browse/XWALK-6059>,XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> (*) instead of the previous test commit id: 4d9d28afa5e34e6bc11c5e9da4dffbbfac5def1e New Issues: ----------------------- XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012> [App-tools][REG] --platform cannot work for windows platform XWALK-6016<https://crosswalk-project.org/jira/browse/XWALK-6016> [App-tools][REG] Build both x86 and arm apk using crosswalk-pkg with "--targets=x86" XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018> [Apptools] Multiple backends cannot work on IOS platform XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020> [App-tools] Crosswalk-pkg is not working on IOS platform XWALK-6049 <https://crosswalk-project.org/jira/browse/XWALK-6049> [App-tools][REG] Fail to build 64bit apk when use crosswalk-pkg with --targets=x86_64 XWALK-6059<https://crosswalk-project.org/jira/browse/XWALK-6059> [Apptools][REG] The ios project name lack of its version after building on ios platform XWALK-6064<https://crosswalk-project.org/jira/browse/XWALK-6064> [Apptools][REG] Fail to build .deb project on Deepin 64bit OS. Verified Issues: ----------------------- XWALK-4499<https://crosswalk-project.org/jira/browse/XWALK-4499> Support for multiple backends Crosswalk-app-tools Repositories: ----------------------- https://github.com/crosswalk-project/crosswalk-app-tools Release Tag or Commit ID: f356ec0538ca2452bfb7207907604c662e0cc261 Debian Backend: https://github.com/crosswalk-project/crosswalk-app-tools-deb Release Tag or Commit ID: c5212e631dc37807be394dd2c86fa0dad9c42767 iOS Backend: https://github.com/crosswalk-project/crosswalk-app-tools-ios Release Tag or Commit ID: abd046da6a127142b01a87e2343f52f0c79bd215 Test Suite Repositories: ----------------------- Command-line Interface: https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-android-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-ios-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-linux-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-sampleapp-tests https://github.com/crosswalk-project/crosswalk-test-suite/blob/master/usecase/usecase-apptools-tests/ Common module: https://github.com/crosswalk-project/crosswalk-app-tools/tree/master/test Crosswalk Binary on Android: https://download.01.org/crosswalk/releases/crosswalk/android/beta/17.46.448.3/ Crosswalk Binary on Windows: https://download.01.org/crosswalk/releases/crosswalk/windows/canary/17.46.443.0/ [no beta builds for Windows] Test Environment: ----------------------- Android host: Ivy Bridge Desktop, Ubuntu 14, Windows 8.1,Windows10 Android device: ASUS Memo Pad 8, ECS2-8A ,Nexus 5, Nexus 9 iOS host: MacBook Pro 10.10 with XCode 7.1 iOS device: iPad mini 8.2 iOS backend: https://github.com/crosswalk-project/crosswalk-app-tools-ios Linux/Deepin desktop: Deepin 2014.2 (http://www.linuxdeepin.com/download.cn.html) x86_64 with Chinese Linux/Deepin backend: https://github.com/crosswalk-project/crosswalk-app-tools-deb Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
