Blueprint changed by Antonio Rosales: Whiteboard changed: - I'm deferring this due to no workitems defined. -robbiew - From Etherpad: Workload tests: - bzr branch lp:server-workload-testing - Currently done and maintained by Canonical - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - Linaro is looking at extracting the power consumed when stressing the board support - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - Not that clear at the moment - Generally looking at the usual test cases/benchmarks for a start - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - KVM - LXC - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with "MONITOR=all" environmental variable to see all sensors... or see included HTML/PDF documentation... or run "phoronix-test-suite system-sensors" to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] ------ =User Stories= - - George wants to setup an openmpi cluster on his new ARM server nodes. + - George wants to setup an openmpi cluster on his new ARM server nodes and would like a common set of tools to measure performance of openMPI on his ARM cluster. + - Rebekah is setting up her new ARM cluster and would like a set of common tools to measure performance of virtualization technologies such as KVM and LXC. + - Jacob has his new ARM server nodes up and running with Ubuntu and would like use a set of common tools to test performance of workloads such as Hadoop in his environment. =Assumptions= - -All of the mentioned pieces of software already work on ARM platforms. + -All of the mentioned pieces of software already work on ARM platforms. + -The work here is to create a common set of scripts for folks to use to tune and analyze performance on their ARM servers. This goal is not to produce performance results. =Test Plans= - - Run each script on all available ARM server hardware and some x86 hardware. + - Identify appropriate benchmark for each workload + - Run each script on all available ARM server hardware and some x86 hardware by first set + - Automate running of scripts for each workload. + * Script will be complete for this blueprint when it accurately exercises the workloads, provides benchmarking data for performance evaluation, and is able to executed in automated process. =Release Note= - - Automated testing of key clustered workloads and core Linux stacks has been performed to ensure a more reliable experience on ARM and x86 hardware. + - Common set of performance set of testing scripts available for ARM servers + *Once work is complete need to details what the scripts are and where they can be obtained.
-- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
