September 7, 2024
Recently, the paper titledConstructing a Supplementary Benchmark Suite to Represent Android Applications with User Interactions by Using Performance Counters, authored by PhD student Ouyang Chenghao under the guidance of Professor Li Guohui, has been accepted by ACM Transactions on Architecture and Code Optimization (TACO).
Existing benchmark suites for smartphone CPU micro-architecture design such as Geekbench 5.0 fail to authentically represent the micro-architecture level performance behavior of widely used real Android applications with interactive operations such as screen sliding. It is therefore crucial to systematically construct a benchmark suite as a supplementary to Geekbench to represent the user interaction behavior of Android applications for CPU micro-architecture design. The key is to identify a small number of representative programs from a large number of real applications. To this end, a set of features used to represent a program need to be constructed, and these features should be fair for different micro-architectures and can be collected efficiently. However, this is extremely difficult for Android applications. For example, the feature collection tools for Android applications are unavailable for benchmark selection.
Hence, the teamproposesa novel benchmark suite construction approach dubbed BEMAP to efficiently build a supplementary benchmark suite from real-world Android applications to represent their user interaction behavior. BEMAP innovates four techniques. The first technique, called two-stage RFC (representative feature construction), constructs program features from performance counters (events) to represent a program for selecting benchmarks from a large number of real Android applications in two stages. The first stage identifies a set of important performance events in terms of IPC (instructions per cycle) by employing a machine learning algorithm named SGBRT (Stochastic Gradient Boosted Regression Tree). The second stage constructs representative features based on the important performance events by using ICA (independent component analysis). The second technique, named SPC-MMA (source performance counters from multiple micro-architectures), collects the performance events from multiple mobile CPUs with different micro-architectures and mixes them as the source of RFC. The goal of these two innovations is to make the program features fair to different mobile CPU micro-architectures. The third technique is that we design a new tool named AutoProfiler to automatically profile the micro-architecture events of Android applications with interactive operations.Using the proposed BEMAP methodology, we constructed SPBench, a novel benchmark suite supplementary to traditional mobile benchmark suites like Geekbench, for mobile CPU micro-architecture design. It consists of fifteen benchmarks selected from one hundred real Android applications with three common user interaction operations. The experimental results on four significantly different micro-architectures show that SPBench can represent the micro-architecture performance behaviors of the one hundred real-world applications with three common user interactive operations on each micro-architecture with significantly higher accuracy than benchmark suites produced by the state-of-the-art approaches.
The ACM Transactions on Architecture and Code Optimization (TACO) is a flagship academic journal in computer system architecture and compilation. It is classified as a CCF A-level journal by the China Computer Federation (CCF). The journal publishes quarterly issues,witharound 20 papersaccepted in each issue, and primarily focuses on research in hardware, software, and system studies related to computer system architecture and code optimization.
