[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170497#comment-13170497 ] Ashutosh Chauhan commented on HIVE-1487: Awesome work! Thanks, Marek and everyone else involved for making life better for us : ) > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170491#comment-13170491 ] John Sichi commented on HIVE-1487: -- https://cwiki.apache.org/confluence/display/Hive/Unit+Test+Parallel+Execution > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168059#comment-13168059 ] Hudson commented on HIVE-1487: -- Integrated in Hive-trunk-h0.21 #1141 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1141/]) Fix executable permissions left out of commit for HIVE-1487. jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213433 Files : * /hive/trunk/testutils/ptest/Report.py * /hive/trunk/testutils/ptest/hivetest.py > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167864#comment-13167864 ] Hudson commented on HIVE-1487: -- Integrated in Hive-trunk-h0.23.0 #17 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/17/]) Fix executable permissions left out of commit for HIVE-1487. jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213433 Files : * /hive/trunk/testutils/ptest/Report.py * /hive/trunk/testutils/ptest/hivetest.py > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166878#comment-13166878 ] Hudson commented on HIVE-1487: -- Integrated in Hive-trunk-h0.21 #1137 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1137/]) HIVE-1487 [jira] Add ptest script (Marek Sapota via John Sichi) Summary: HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 603 jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1212697 Files : * /hive/trunk/testutils/ptest * /hive/trunk/testutils/ptest/Buffer.py * /hive/trunk/testutils/ptest/Process.py * /hive/trunk/testutils/ptest/README * /hive/trunk/testutils/ptest/Report.py * /hive/trunk/testutils/ptest/Ssh.py * /hive/trunk/testutils/ptest/config.py * /hive/trunk/testutils/ptest/hivetest.py * /hive/trunk/testutils/ptest/templates * /hive/trunk/testutils/ptest/templates/Properties.html * /hive/trunk/testutils/ptest/templates/TestCase.html * /hive/trunk/testutils/ptest/templates/TestRun.html * /hive/trunk/testutils/ptest/templates/TestSuite.html * /hive/trunk/testutils/ptest/templates/common.css > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166836#comment-13166836 ] Hudson commented on HIVE-1487: -- Integrated in Hive-trunk-h0.23.0 #13 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/13/]) HIVE-1487 [jira] Add ptest script (Marek Sapota via John Sichi) Summary: HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 603 jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1212697 Files : * /hive/trunk/testutils/ptest * /hive/trunk/testutils/ptest/Buffer.py * /hive/trunk/testutils/ptest/Process.py * /hive/trunk/testutils/ptest/README * /hive/trunk/testutils/ptest/Report.py * /hive/trunk/testutils/ptest/Ssh.py * /hive/trunk/testutils/ptest/config.py * /hive/trunk/testutils/ptest/hivetest.py * /hive/trunk/testutils/ptest/templates * /hive/trunk/testutils/ptest/templates/Properties.html * /hive/trunk/testutils/ptest/templates/TestCase.html * /hive/trunk/testutils/ptest/templates/TestRun.html * /hive/trunk/testutils/ptest/templates/TestSuite.html * /hive/trunk/testutils/ptest/templates/common.css > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166706#comment-13166706 ] Phabricator commented on HIVE-1487: --- mareksapotafb has committed the revision "HIVE-1487 [jira] Add ptest script". REVISION DETAIL https://reviews.facebook.net/D603 COMMIT https://reviews.facebook.net/rHIVE1212697 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166697#comment-13166697 ] John Sichi commented on HIVE-1487: -- Also thanks Zheng for the original ptest script, and Charles Chen for hacking up an intermediate version :) > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Fix For: 0.9.0 > > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166628#comment-13166628 ] Phabricator commented on HIVE-1487: --- jsichi has accepted the revision "HIVE-1487 [jira] Add ptest script". +1, committing now. Can you grant rights to ASF? REVISION DETAIL https://reviews.facebook.net/D603 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166523#comment-13166523 ] Phabricator commented on HIVE-1487: --- mareksapotafb has commented on the revision "HIVE-1487 [jira] Add ptest script". Just use instructions from the wiki, the instance set up in my home directory uses this patch. REVISION DETAIL https://reviews.facebook.net/D603 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166434#comment-13166434 ] Phabricator commented on HIVE-1487: --- jsichi has commented on the revision "HIVE-1487 [jira] Add ptest script". Do you have this deployed somewhere I can test it? REVISION DETAIL https://reviews.facebook.net/D603 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, > HIVE-1487.D603.4.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163972#comment-13163972 ] Phabricator commented on HIVE-1487: --- jsichi has requested changes to the revision "HIVE-1487 [jira] parallelize test query runs". We can commit this once we get to the bottom of minimr failures. INLINE COMMENTS testutils/ptest/README:48 Typo: loose -> lose testutils/ptest/Ssh.py:21 Let's expunge this comment. testutils/ptest/config.py:83 Typo: faled -> failed REVISION DETAIL https://reviews.facebook.net/D603 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma >Assignee: Marek Sapota > Attachments: HIVE-1487.D603.1.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163096#comment-13163096 ] Phabricator commented on HIVE-1487: --- mareksapotafb has added reviewers to the revision "HIVE-1487 [jira] parallelize test query runs". Added Reviewers: jsichi REVISION DETAIL https://reviews.facebook.net/D603 > parallelize test query runs > --- > > Key: HIVE-1487 > URL: https://issues.apache.org/jira/browse/HIVE-1487 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Joydeep Sen Sarma > Attachments: HIVE-1487.D603.1.patch > > > HIVE-1464 speeded up serial runs somewhat - but looks like it's still too > slow. we should use parallel junit or some similar setup to run test queries > in parallel. this should be really easy as well need to just use a separate > warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
