[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940253#comment-13940253 ] Jason Dere commented on HIVE-6658: -- +1 > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.2.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938916#comment-13938916 ] Szehon Ho commented on HIVE-6658: - Yea, looks good, thanks > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.2.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938587#comment-13938587 ] Laljo John Pullokkaran commented on HIVE-6658: -- I assumed there were older versions of hadoop that Hive supported (< 0.20). I have reworked the patch. > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.2.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[
https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13936072#comment-13936072
]
Hive QA commented on HIVE-6658:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12634652/HIVE-6658.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1802/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1802/console
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/hwi/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-hwi ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/tmp/conf
[copy] Copying 5 files to
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-hwi ---
[INFO] Compiling 2 source files to
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-hwi ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hwi ---
[INFO] Building jar:
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/hive-hwi-0.14.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
hive-hwi ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hwi ---
[INFO] Installing
/data/hive-ptest/working/apache-svn-trunk-source/hwi/target/hive-hwi-0.14.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hwi/0.14.0-SNAPSHOT/hive-hwi-0.14.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/hwi/pom.xml
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hwi/0.14.0-SNAPSHOT/hive-hwi-0.14.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive ODBC 0.14.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-odbc ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/odbc (includes
= [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-odbc ---
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-odbc ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-odbc ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/odbc/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/odbc/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/odbc/target/tmp/conf
[copy] Copying 5 files to
/data/hive-ptest/working/apache-svn-trunk-source/odbc/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
hive-odbc ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-odbc ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/odbc/pom.xml
to
/data/hive-ptest/working/maven/org/apache/hive/hive-odbc/0.14.0-SNAPSHOT/hive-odbc-0.14.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive Shims Aggregator 0.14.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-shims-aggregator
---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/shims
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @
hive-shims-aggregator ---
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @
hive-shims-aggregator ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @
hive-shims-aggregator ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptes
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935853#comment-13935853 ] Szehon Ho commented on HIVE-6658: - Yea we should have two tests and keep testing 0.20. Sorry for not being clear, my suggestion was : the hadoop1 version can have --include 0.20,0.20S and the hadoop2 version can have --exclude 0.20,0.20S. This is how most tests are split right now. In this patch, hadoop1 version says --exclude hadoop0.23 and hadoop2 version says --include 0.23. But then, any later version of hadoop beyond 0.23 will test using hadoop1 version, which would be the 'wrong' result in this case. Just a thought. > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935843#comment-13935843 ] Laljo John Pullokkaran commented on HIVE-6658: -- Szehon, Shouldn't it depend on de-supporting time line for HADOOP 0.20 ? In my opinion till HADOOP 0.20 is de-supported we would want to test the behavior (alter bucket) for HADOOP 0.20 and hence require two different set of tests. > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6658) Modify Alter_numbuckets* test to reflect hadoop2 changes
[ https://issues.apache.org/jira/browse/HIVE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935362#comment-13935362 ] Szehon Ho commented on HIVE-6658: - Yea I saw this too. Thanks for fixing this, one comment for consideration- Looking at other q-tests with two versions, original includes 0.20,0.20S, and new version exclude 0.20,20S. Do we want to do that instead for consistency, as future versions of hadoop should probably adhere to 'new' behavior (which is proper #buckets in this case). > Modify Alter_numbuckets* test to reflect hadoop2 changes > > > Key: HIVE-6658 > URL: https://issues.apache.org/jira/browse/HIVE-6658 > Project: Hive > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-6658.patch > > > Hadoop2 now honors number of reducers config while running in local mode. > This affects bucketing tests as the data gets properly bucketed in Hadoop2 > (In hadoop1 all data ended up in same bucket while in local mode). -- This message was sent by Atlassian JIRA (v6.2#6252)
