[jira] Commented: (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902958#action_12902958 ] Joydeep Sen Sarma commented on HIVE-1487: - can people with experience running java tests in parallel comment on this? So far these seem the choices: * upgrade to junit4 and use custom runner that runs in parallel. the downside here is that junit does not seem to come with this parallel runner (but there's additional code on the web from the junit authors that does the same) * use parallel-junit. this seems the least disruptive - but this seems like an old/dead project * use TestNG - this is a replacement for junit that has inbuilt parallel execution support. but we would not be using junit anymore at all. any other thoughts on better test setup welcome as well. parallelize test query runs --- Key: HIVE-1487 URL: https://issues.apache.org/jira/browse/HIVE-1487 Project: Hadoop Hive Issue Type: Improvement Components: Testing Infrastructure Reporter: Joydeep Sen Sarma HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902967#action_12902967 ] John Sichi commented on HIVE-1487: -- Zheng has a private ptest shell script which takes a different approach: it enumerates all of the .q files, partitions them modulo the degree of parallelism, and then executes them in separate environments as separate processes. One advantage is that there's no cross-talk from threading (although in the long run testing with multithreading for independent tests would be good for coverage). I'll send you a pointer. parallelize test query runs --- Key: HIVE-1487 URL: https://issues.apache.org/jira/browse/HIVE-1487 Project: Hadoop Hive Issue Type: Improvement Components: Testing Infrastructure Reporter: Joydeep Sen Sarma HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
