[ https://issues.apache.org/jira/browse/SYSTEMML-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985030#comment-15985030 ]
Krishna Kalyan commented on SYSTEMML-1451: ------------------------------------------ Hello All, I tried to run benchmark test on my local VM. (Current Configuration) 2 Cores 8 GB RAM However my script failed with `There is insufficient memory for the Java Runtime Environment to continue`. I would like to know hardware recommendations to run these scripts. (Based on which I will request for EC2 spot instance). (Error Log) https://gist.github.com/krishnakalyan3/ea370347df756b34b32a7200b641d72d > Automate performance testing and reporting > ------------------------------------------ > > Key: SYSTEMML-1451 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1451 > Project: SystemML > Issue Type: Improvement > Components: Infrastructure, Test > Reporter: Nakul Jindal > Labels: gsoc2017, mentor, performance, reporting, testing > > As part of a release (and in general), performance tests are run for SystemML. > Currently, running and reporting on these performance tests are a manual > process. There are helper scripts, but largely the process is manual. > The aim of this GSoC 2017 project is to automate performance testing and its > reporting. > These are the tasks that this entails > 1. Automate running of the performance tests, including generation of test > data > 2. Detect errors and report if any > 3. Record performance benchmarking information > 4. Automatically compare this performance to previous version to check for > performance regressions > 5. Automatically compare to Spark MLLib, R?, Julia? > 6. Prepare report with all the information about failed jobs, performance > information, perf info against other comparable projects/algorithms > (plotted/in plain text in CSV, PDF or other common format) > 7. Create scripts to automatically run this process on a cloud provider that > spins up machines, runs the test, saves the reports and spins down the > machines. > 8. Create a web application to do this interactively without dropping down > into a shell. > As part of this project, the student will need to know scripting (in Bash, > Python, etc). It may also involve changing error reporting and performance > reporting code in SystemML. > Rating - Medium (for the amount of work) > Mentor - [~nakul02] (Other co-mentors will join in) -- This message was sent by Atlassian JIRA (v6.3.15#6346)