New Benchmark Shows ITeM-HM Triples Success Rate for Migrating Android Tests to OpenHarmony

A research team has published results showing that automated UI test migration between mobile operating systems remains a largely unsolved problem, but one particular technique is pulling far ahead of its rivals. In a paper submitted to arXiv on July 13, 2026, researchers introduced ITeM-HM, a method for converting Android application test cases into equivalent tests for OpenHarmony, Huawei's alternative mobile operating system. The approach reached an 81% success rate on the task, a 214% relative improvement over the original ITeM method it builds on, which managed only 26%.

Why the Android-to-OpenHarmony Gap Matters

OpenHarmony has been positioned as a genuine alternative to Android in markets where Huawei's ecosystem carries weight, but the practical barrier for developers has always been the cost of porting existing software, including the test suites that keep apps stable across releases. Manually rewriting UI tests for a new operating system is slow and error-prone, and prior automated tools have struggled to bridge the structural and semantic differences between the two platforms' testing frameworks. The paper's benchmark results make that struggle concrete: a competing method called ReSPlay, evaluated under the same conditions, achieved just a 15% success rate, far below both ITeM and ITeM-HM.

Inside the ATH Benchmark

To make these comparisons possible, the researchers built what they call the ATH Benchmark, a dataset drawn from 36 commercial applications that collectively account for more than 9 billion downloads. That scale matters because it grounds the evaluation in software people actually use, rather than synthetic or academic test apps that might not reflect real-world UI complexity. Alongside the app corpus, the team manually authored 108 test cases to serve as ground truth for measuring whether a migrated test actually functions correctly on the target platform.

The near-tripling of success rate between ITeM and ITeM-HM suggests the enhancements introduced in the newer method address specific failure modes that the original approach could not handle, though the paper does not claim to have solved cross-platform test migration outright. An 81% success rate still leaves roughly one in five test cases failing to migrate cleanly, which for large commercial applications with extensive test suites could still translate into meaningful manual cleanup work.

What the results do establish clearly is a wide performance gap among current approaches. With ReSPlay trailing at 15% and the baseline ITeM method reaching only 26%, ITeM-HM's 81% marks a substantial jump rather than an incremental gain, at least on the benchmark as constructed. For teams weighing whether to bring Android applications to OpenHarmony, that gap could shape which migration tooling gets adopted first, and it gives the broader research community a concrete target to try to beat in future work on cross-platform UI test transfer.