[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Work items changed: Work items: - [server-workload-testing-team] Add a test for KVM: TODO + [server-workload-testing-team] Add a test for KVM: POSTPONED [server-workload-testing-team] Add a test for LXC: TODO [craig.magina] Add a test for OpenMPI: INPROGRESS [server-workload-testing-team] Add a test for Membase: POSTPONED [craig.magina] Add a test for Hadoop and other workloads: DONE [server-workload-testing-team] Add a test for AHCI: POSTPONED [server-workload-testing-team] Ensure that multi-system testing setups are standardized: POSTPONED [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): DONE [server-workload-testing-team] Find an IPV6 testing tool: POSTPONED [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): POSTPONED [server-workload-testing-team] Test on real hardware when available: DONE [server-workload-testing-team] Compare P test results against O: POSTPONED [server-workload-testing-team] Investigate PTS logging sensor data: POSTPONED [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Work items changed: Work items: [server-workload-testing-team] Add a test for KVM: POSTPONED - [server-workload-testing-team] Add a test for LXC: TODO + [craig.magina] Add a test for LXC: TODO [craig.magina] Add a test for OpenMPI: INPROGRESS [server-workload-testing-team] Add a test for Membase: POSTPONED [craig.magina] Add a test for Hadoop and other workloads: DONE [server-workload-testing-team] Add a test for AHCI: POSTPONED [server-workload-testing-team] Ensure that multi-system testing setups are standardized: POSTPONED [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): DONE [server-workload-testing-team] Find an IPV6 testing tool: POSTPONED [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): POSTPONED [server-workload-testing-team] Test on real hardware when available: DONE [server-workload-testing-team] Compare P test results against O: POSTPONED [server-workload-testing-team] Investigate PTS logging sensor data: POSTPONED [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Work items changed: Work items: [server-workload-testing-team] Add a test for KVM: TODO [server-workload-testing-team] Add a test for LXC: TODO [craig.magina] Add a test for OpenMPI: INPROGRESS - [server-workload-testing-team] Add a test for Membase: TODO + [server-workload-testing-team] Add a test for Membase: POSTPONED [craig.magina] Add a test for Hadoop and other workloads: DONE - [server-workload-testing-team] Add a test for AHCI: TODO - [server-workload-testing-team] Ensure that multi-system testing setups are standardized: TODO + [server-workload-testing-team] Add a test for AHCI: POSTPONED + [server-workload-testing-team] Ensure that multi-system testing setups are standardized: POSTPONED [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO - [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): TODO - [server-workload-testing-team] Find an IPV6 testing tool: TODO - [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): TODO - [server-workload-testing-team] Test on real hardware when available: TODO - [server-workload-testing-team] Compare P test results against O: TODO - [server-workload-testing-team] Investigate PTS logging sensor data: TODO + [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): INPROGRESS + [server-workload-testing-team] Find an IPV6 testing tool: POSTPONED + [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): POSTPONED + [server-workload-testing-team] Test on real hardware when available: DONE + [server-workload-testing-team] Compare P test results against O: POSTPONED + [server-workload-testing-team] Investigate PTS logging sensor data: POSTPONED [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by MMorana: Work items changed: Work items: [server-workload-testing-team] Add a test for KVM: TODO [server-workload-testing-team] Add a test for LXC: TODO [craig.magina] Add a test for OpenMPI: INPROGRESS [server-workload-testing-team] Add a test for Membase: POSTPONED [craig.magina] Add a test for Hadoop and other workloads: DONE [server-workload-testing-team] Add a test for AHCI: POSTPONED [server-workload-testing-team] Ensure that multi-system testing setups are standardized: POSTPONED [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO - [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): INPROGRESS + [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): DONE [server-workload-testing-team] Find an IPV6 testing tool: POSTPONED [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): POSTPONED [server-workload-testing-team] Test on real hardware when available: DONE [server-workload-testing-team] Compare P test results against O: POSTPONED [server-workload-testing-team] Investigate PTS logging sensor data: POSTPONED [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Work items changed: Work items: [server-workload-testing-team] Add a test for KVM: TODO [server-workload-testing-team] Add a test for LXC: TODO - [server-workload-testing-team] Add a test for OpenMPI: TODO + [craig.magina] Add a test for OpenMPI: INPROGRESS [server-workload-testing-team] Add a test for Membase: TODO - [server-workload-testing-team] Add a test for Hadoop and other workloads: TODO + [craig.magina] Add a test for Hadoop and other workloads: DONE [server-workload-testing-team] Add a test for AHCI: TODO [server-workload-testing-team] Ensure that multi-system testing setups are standardized: TODO [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): TODO [server-workload-testing-team] Find an IPV6 testing tool: TODO [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): TODO [server-workload-testing-team] Test on real hardware when available: TODO [server-workload-testing-team] Compare P test results against O: TODO [server-workload-testing-team] Investigate PTS logging sensor data: TODO [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Work items set to: Work items: [server-workload-testing-team] Add a test for KVM: TODO [server-workload-testing-team] Add a test for LXC: TODO [server-workload-testing-team] Add a test for OpenMPI: TODO [server-workload-testing-team] Add a test for Membase: TODO [server-workload-testing-team] Add a test for Hadoop and other workloads: TODO [server-workload-testing-team] Add a test for AHCI: TODO [server-workload-testing-team] Ensure that multi-system testing setups are standardized: TODO [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): TODO [server-workload-testing-team] Find an IPV6 testing tool: TODO [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): TODO [server-workload-testing-team] Test on real hardware when available: TODO [server-workload-testing-team] Compare P test results against O: TODO [server-workload-testing-team] Investigate PTS logging sensor data: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Christopher Chaltain: Assignee: Michael Casadevall = Craig Magina -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Craig Magina: Whiteboard changed: I'm deferring this due to no workitems defined. -robbiew From Etherpad: Workload tests: - bzr branch lp:server-workload-testing - Currently done and maintained by Canonical - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - Linaro is looking at extracting the power consumed when stressing the board support - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - Not that clear at the moment - Generally looking at the usual test cases/benchmarks for a start - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - KVM - LXC - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with MONITOR=all environmental variable to see all sensors... or see included HTML/PDF documentation... or run phoronix-test-suite system-sensors to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] -- - Information on integrating Blueprint spec integration into the whiteboard is at https://wiki.ubuntu.com/BlueprintSpec + =User Stories= + - George wants to setup an openmpi cluster on his new ARM server nodes. + + =Assumptions= + -All of the mentioned pieces of software already work on ARM platforms. + + =Test Plans= + - Run each script on all available ARM server hardware and some x86 hardware. + + =Release Note= + - Automated testing of key clustered workloads and core Linux stacks has been performed to ensure a more reliable experience on ARM and x86 hardware. -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Work items changed: Work items: [server-workload-testing-team] Add a test for KVM: TODO [server-workload-testing-team] Add a test for LXC: TODO [server-workload-testing-team] Add a test for OpenMPI: TODO [server-workload-testing-team] Add a test for Membase: TODO [server-workload-testing-team] Add a test for Hadoop and other workloads: TODO [server-workload-testing-team] Add a test for AHCI: TODO [server-workload-testing-team] Ensure that multi-system testing setups are standardized: TODO [server-workload-testing-team] Setup weekly meetings to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC): TODO [server-workload-testing-team] Automate testing (explore PTS documentation about batch-run options): TODO [server-workload-testing-team] Find an IPV6 testing tool: TODO [server-workload-testing-team] Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench): TODO [server-workload-testing-team] Test on real hardware when available: TODO [server-workload-testing-team] Compare P test results against O: TODO [server-workload-testing-team] Investigate PTS logging sensor data: TODO + [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions: TODO -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Whiteboard changed: - I'm deferring this due to no workitems defined. -robbiew - From Etherpad: Workload tests: - bzr branch lp:server-workload-testing - Currently done and maintained by Canonical - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - Linaro is looking at extracting the power consumed when stressing the board support - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - Not that clear at the moment - Generally looking at the usual test cases/benchmarks for a start - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - KVM - LXC - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with MONITOR=all environmental variable to see all sensors... or see included HTML/PDF documentation... or run phoronix-test-suite system-sensors to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] -- =User Stories= - - George wants to setup an openmpi cluster on his new ARM server nodes. + - George wants to setup an openmpi cluster on his new ARM server nodes and would like a common set of tools to measure performance of openMPI on his ARM cluster. + - Rebekah is setting up her new ARM cluster and would like a set of common tools to measure performance of virtualization technologies such as KVM and LXC. + - Jacob has his new ARM server nodes up and running with Ubuntu and would like use a set of common tools to test performance of workloads such as Hadoop in his environment. =Assumptions= - -All of the mentioned pieces of software already work on ARM platforms. + -All of the mentioned pieces of software already work on ARM platforms. + -The work here is to create a common set of scripts for folks to use to tune and analyze performance on their ARM servers. This goal is not to produce performance results. =Test Plans= - - Run each script on all available ARM server hardware and some x86 hardware. + - Identify appropriate benchmark for each workload + - Run each script on all available ARM server hardware and some x86 hardware by first set + - Automate running of scripts for each workload. + * Script will be complete for this blueprint when it accurately exercises the workloads, provides benchmarking data for performance evaluation, and is able to executed in automated process. =Release Note= - - Automated testing of key clustered workloads and core Linux stacks has been performed to ensure a more reliable experience on ARM and x86 hardware. + - Common set of performance set of testing scripts available for ARM servers + *Once work is complete need to details what the scripts are and where they can be obtained. -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Priority: Undefined = Medium -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Definition Status: Discussion = Approved -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Robbie Williamson: Whiteboard changed: + I'm deferring this due to no workitems defined. -robbiew + From Etherpad: Workload tests: - bzr branch lp:server-workload-testing - Currently done and maintained by Canonical - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - Linaro is looking at extracting the power consumed when stressing the board support - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - Not that clear at the moment - Generally looking at the usual test cases/benchmarks for a start - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - KVM - LXC - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with MONITOR=all environmental variable to see all sensors... or see included HTML/PDF documentation... or run phoronix-test-suite system-sensors to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] -- Information on integrating Blueprint spec integration into the whiteboard is at https://wiki.ubuntu.com/BlueprintSpec -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Whiteboard changed: From Etherpad: Workload tests: - - bzr branch lp:server-workload-testing - - Currently done and maintained by Canonical - - Linaro could help getting it to work across the ARM boards available there as well as with LAVA + - bzr branch lp:server-workload-testing + - Currently done and maintained by Canonical + - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - - Linaro is looking at extracting the power consumed when stressing the board support - - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run + - Linaro is looking at extracting the power consumed when stressing the board support + - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - - Not that clear at the moment - - Generally looking at the usual test cases/benchmarks for a start - - One way would be to get feedback from the hw vendors to understand what would be the priority + - Not that clear at the moment + - Generally looking at the usual test cases/benchmarks for a start + - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - - KVM - - LXC - - OpenMPI + - KVM + - LXC + - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with MONITOR=all environmental variable to see all sensors... or see included HTML/PDF documentation... or run phoronix-test-suite system-sensors to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] + + -- + Information on integrating Blueprint spec integration into the whiteboard is at https://wiki.ubuntu.com/BlueprintSpec -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Antonio Rosales: Approver: Antonio Rosales = Christopher Chaltain -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by MMorana: Whiteboard set to: From Etherpad: Workload tests: - bzr branch lp:server-workload-testing - Currently done and maintained by Canonical - Linaro could help getting it to work across the ARM boards available there as well as with LAVA openssl: openssl-test? gnu-tls? ARM Workload: - Linaro is looking at extracting the power consumed when stressing the board support - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run Areas to investigate while checking for performance: - Not that clear at the moment - Generally looking at the usual test cases/benchmarks for a start - One way would be to get feedback from the hw vendors to understand what would be the priority ACTION: [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions IO/Storage: Ceph, Swift, Gluster? Areas of focus on feature/functional side: - KVM - LXC - OpenMPI Calxeda is interested in OpenMPI performance/testing. From P: As actual ARM server hardware begins to enter the marketplace, a standardized way to test and benchmark performance must be developed and run on all active platforms we support for ARM server. http://www.phoronix-test-suite.com/ http://openbenchmarking.org/ Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not be public), Unixbench, plus others (can also easily add in any others, just ask) https://wiki.ubuntu.com/ARM/QA/Server https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation - membase? - Ensure that multi-system testing setups are standardized - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC) ACTIONS: - Automate testing [explore PTS documentation about batch-run options] - Find an IPV6 testing tool - Add a test for Membase - Add a test for Hadoop and other workloads - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench) - Test on real hardware when available - Compare P test results against O. - AHCI testing - Investigate PTS logging sensor data [Run any test(s) with MONITOR=all environmental variable to see all sensors... or see included HTML/PDF documentation... or run phoronix-test-suite system-sensors to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.] -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Robbie Williamson: Assignee: (none) = Michael Casadevall -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance
Blueprint changed by Robbie Williamson: Drafter: (none) = Ubuntu Server Team -- ARM Server Benchmarking and Performance https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
