[jira] [Commented] (HIVE-4993) Add HCatalog documentation to Hive wiki
[ https://issues.apache.org/jira/browse/HIVE-4993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743165#comment-13743165 ] Lefty Leverenz commented on HIVE-4993: -- The HCat docs are all converted to wikidocs. You can find them here: * HCatalog home page: [https://cwiki.apache.org/confluence/display/Hive/HCatalog] ** Overview: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+Overview] ** Installation from Tarball: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+Install] ** Load and Store Interfaces: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore] ** Input and Output Interfaces: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+InputOutput] ** Reader and Writer Interfaces: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+ReaderWriter] ** Command Line Interface: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+CLI] ** Storage Formats: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+StorageFormats] ** Dynamic Partitioning: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+DynamicPartitions] ** Notification: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+Notification] ** Storage Based Authorization: [https://cwiki.apache.org/confluence/display/Hive/HCatalog+Authorization] Some of them need review & revision. In particular: # Installation ([https://cwiki.apache.org/confluence/display/Hive/HCatalog+Install]) starts with this disclaimer: #* "This information is adapted from the HCatalog 0.5.0 installation instructions. Now that HCatalog is part of the Hive project, it is installed with Hive and therefore much of the information below is obsolete. Furthermore, there is no such thing as 'HCatalog 0.11.0' although that fiction has been substituted below for 'HCatalog 0.5.0' — until this wikidoc is revised to reflect current realities, you should be skeptical of what you read here." # Load and Store has obsolete version numbers for jars ([https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore#HCatalogLoadStore-JarsandConfigurationFiles]) so I just called them 0.5.0 examples but they should be updated: #* The version number found in each filepath will be substituted for *. For example, HCatalog release 0.5.0 uses these jars and conf files: #** $HCAT_HOME/share/hcatalog/hcatalog-core-0.5.0.jar #** $HCAT_HOME/share/hcatalog/hcatalog-pig-adapter-0.5.0.jar #** $HIVE_HOME/lib/hive-metastore-0.10.0.jar #** $HIVE_HOME/lib/libthrift-0.7.0.jar #** $HIVE_HOME/lib/hive-exec-0.10.0.jar #** $HIVE_HOME/lib/libfb303-0.7.0.jar #** $HIVE_HOME/lib/jdo2-api-2.3-ec.jar #** $HIVE_HOME/conf #** $HADOOP_HOME/conf #** $HIVE_HOME/lib/slf4j-api-1.6.1.jar # Storage Formats ([https://cwiki.apache.org/confluence/display/Hive/HCatalog+StorageFormats]) might need updates for two sentences: #* "SerDes are provided for RCFile, CSV text, JSON text, and SequenceFile formats." --- What about ORC? #* "However, Hive does not know where to find the HCatalog jar by default, ..." --- Is that still true? # Authorization's section on Minimum Permissions ([https://cwiki.apache.org/confluence/display/Hive/HCatalog+Authorization#HCatalogAuthorization-MinimumPermissions]) has needed more information since it first appeared, but nobody had time for it. > Add HCatalog documentation to Hive wiki > --- > > Key: HIVE-4993 > URL: https://issues.apache.org/jira/browse/HIVE-4993 > Project: Hive > Issue Type: Bug > Components: Documentation >Affects Versions: 0.11.0 >Reporter: Lefty Leverenz >Assignee: Lefty Leverenz > > The documentation for HCatalog versions 0.5.0 and earlier (Apache incubator) > had xml source files which generated html & pdf output files. Now that > HCatalog is part of the Hive project, the HCatalog documents need to be added > to the Hive wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[
https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743152#comment-13743152
]
Hive QA commented on HIVE-5115:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598646/HIVE-5115.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2886 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udtf_not_supported2
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/468/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/468/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
> Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for
> Hadoop 2
> --
>
> Key: HIVE-5115
> URL: https://issues.apache.org/jira/browse/HIVE-5115
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 0.12.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-5115.patch
>
>
> In the webhcat e2e testsuite we have two MR job submission tests
> TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example.
> Intention of this one is to test MR job submission using WebHCat.
> TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example.
> Intention of this one is to test a long running (>10min) WebHCat MR job, see
> HIVE-4808.
> In Hadoop 1, both of these example MR applications are packaged in
> hadoop-examples.jar
> In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar
> and "wordcount" is bundled in hadoop-mapreduce-examples.jar
> Currently the webhcat tests assume that both these MR applications are in one
> jar that we copy as hexamples.jar.
> To run these against Hadoop 2 I can think of three simple solutions:
> (1) Stick with one jar and run "sleep" application in the TestMapReduce_1
> test as well.
> (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as
> well.
> (3) Require two different jars for Hadoop 2 and call them hclient.jar
> (containing "sleep" application) and hexamples.jar (containing "wordcount"
> application). For Hadoop 1, we would make two copies of the same
> hadoop-examples.jar application and call them hsleep.jar and examples.jar.
> The three approaches mentioned here would require least of changes. My
> inclination is towards (2).
> Let me know what you think and I can provide the patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal updated HIVE-5115: - Status: Patch Available (was: Open) > Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for > Hadoop 2 > -- > > Key: HIVE-5115 > URL: https://issues.apache.org/jira/browse/HIVE-5115 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.12.0 >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > Attachments: HIVE-5115.patch > > > In the webhcat e2e testsuite we have two MR job submission tests > TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. > Intention of this one is to test MR job submission using WebHCat. > TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. > Intention of this one is to test a long running (>10min) WebHCat MR job, see > HIVE-4808. > In Hadoop 1, both of these example MR applications are packaged in > hadoop-examples.jar > In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar > and "wordcount" is bundled in hadoop-mapreduce-examples.jar > Currently the webhcat tests assume that both these MR applications are in one > jar that we copy as hexamples.jar. > To run these against Hadoop 2 I can think of three simple solutions: > (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 > test as well. > (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as > well. > (3) Require two different jars for Hadoop 2 and call them hclient.jar > (containing "sleep" application) and hexamples.jar (containing "wordcount" > application). For Hadoop 1, we would make two copies of the same > hadoop-examples.jar application and call them hsleep.jar and examples.jar. > The three approaches mentioned here would require least of changes. My > inclination is towards (2). > Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal updated HIVE-5115: - Attachment: HIVE-5115.patch Thanks [~thejas]. Attached the patch with the suggested change. > Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for > Hadoop 2 > -- > > Key: HIVE-5115 > URL: https://issues.apache.org/jira/browse/HIVE-5115 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.12.0 >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > Attachments: HIVE-5115.patch > > > In the webhcat e2e testsuite we have two MR job submission tests > TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. > Intention of this one is to test MR job submission using WebHCat. > TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. > Intention of this one is to test a long running (>10min) WebHCat MR job, see > HIVE-4808. > In Hadoop 1, both of these example MR applications are packaged in > hadoop-examples.jar > In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar > and "wordcount" is bundled in hadoop-mapreduce-examples.jar > Currently the webhcat tests assume that both these MR applications are in one > jar that we copy as hexamples.jar. > To run these against Hadoop 2 I can think of three simple solutions: > (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 > test as well. > (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as > well. > (3) Require two different jars for Hadoop 2 and call them hclient.jar > (containing "sleep" application) and hexamples.jar (containing "wordcount" > application). For Hadoop 1, we would make two copies of the same > hadoop-examples.jar application and call them hsleep.jar and examples.jar. > The three approaches mentioned here would require least of changes. My > inclination is towards (2). > Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743124#comment-13743124 ] Eugene Koifman commented on HIVE-5115: -- I also think 3 is better long term as we add more/different tests in the future. > Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for > Hadoop 2 > -- > > Key: HIVE-5115 > URL: https://issues.apache.org/jira/browse/HIVE-5115 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.12.0 >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > > In the webhcat e2e testsuite we have two MR job submission tests > TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. > Intention of this one is to test MR job submission using WebHCat. > TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. > Intention of this one is to test a long running (>10min) WebHCat MR job, see > HIVE-4808. > In Hadoop 1, both of these example MR applications are packaged in > hadoop-examples.jar > In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar > and "wordcount" is bundled in hadoop-mapreduce-examples.jar > Currently the webhcat tests assume that both these MR applications are in one > jar that we copy as hexamples.jar. > To run these against Hadoop 2 I can think of three simple solutions: > (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 > test as well. > (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as > well. > (3) Require two different jars for Hadoop 2 and call them hclient.jar > (containing "sleep" application) and hexamples.jar (containing "wordcount" > application). For Hadoop 1, we would make two copies of the same > hadoop-examples.jar application and call them hsleep.jar and examples.jar. > The three approaches mentioned here would require least of changes. My > inclination is towards (2). > Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743103#comment-13743103 ] Brock Noland commented on HIVE-4388: That'd be great as out test fail with: hbase-default.xml file seems to be for and old version of HBase (0.95.2-hadoop2), this version is 0.95.2-hadoop1 > HBase tests fail against Hadoop 2 > - > > Key: HIVE-4388 > URL: https://issues.apache.org/jira/browse/HIVE-4388 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Reporter: Gunther Hagleitner >Assignee: Brock Noland > Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388-wip.txt > > > Currently we're building by default against 0.92. When you run against hadoop > 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. > HIVE-3861 upgrades the version of hbase used. This will get you past the > problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743083#comment-13743083 ] Thejas M Nair commented on HIVE-5115: - I think it is useful to have a MR test that reads an input file and writes an ouput. It can help when we are validating things like the proxy-user functionality. So I would go for option 3. > Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for > Hadoop 2 > -- > > Key: HIVE-5115 > URL: https://issues.apache.org/jira/browse/HIVE-5115 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.12.0 >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > > In the webhcat e2e testsuite we have two MR job submission tests > TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. > Intention of this one is to test MR job submission using WebHCat. > TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. > Intention of this one is to test a long running (>10min) WebHCat MR job, see > HIVE-4808. > In Hadoop 1, both of these example MR applications are packaged in > hadoop-examples.jar > In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar > and "wordcount" is bundled in hadoop-mapreduce-examples.jar > Currently the webhcat tests assume that both these MR applications are in one > jar that we copy as hexamples.jar. > To run these against Hadoop 2 I can think of three simple solutions: > (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 > test as well. > (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as > well. > (3) Require two different jars for Hadoop 2 and call them hclient.jar > (containing "sleep" application) and hexamples.jar (containing "wordcount" > application). For Hadoop 1, we would make two copies of the same > hadoop-examples.jar application and call them hsleep.jar and examples.jar. > The three approaches mentioned here would require least of changes. My > inclination is towards (2). > Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743075#comment-13743075 ] stack commented on HIVE-4388: - [~brocknoland] Hmm Just ran through the build and seems to put the right value in there. I must have mangled the build somehow. Want me to regen? > HBase tests fail against Hadoop 2 > - > > Key: HIVE-4388 > URL: https://issues.apache.org/jira/browse/HIVE-4388 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Reporter: Gunther Hagleitner >Assignee: Brock Noland > Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388-wip.txt > > > Currently we're building by default against 0.92. When you run against hadoop > 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. > HIVE-3861 upgrades the version of hbase used. This will get you past the > problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal updated HIVE-5115: - Component/s: (was: WebHCat) Tests > Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for > Hadoop 2 > -- > > Key: HIVE-5115 > URL: https://issues.apache.org/jira/browse/HIVE-5115 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.12.0 >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > > In the webhcat e2e testsuite we have two MR job submission tests > TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. > Intention of this one is to test MR job submission using WebHCat. > TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. > Intention of this one is to test a long running (>10min) WebHCat MR job, see > HIVE-4808. > In Hadoop 1, both of these example MR applications are packaged in > hadoop-examples.jar > In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar > and "wordcount" is bundled in hadoop-mapreduce-examples.jar > Currently the webhcat tests assume that both these MR applications are in one > jar that we copy as hexamples.jar. > To run these against Hadoop 2 I can think of three simple solutions: > (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 > test as well. > (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as > well. > (3) Require two different jars for Hadoop 2 and call them hclient.jar > (containing "sleep" application) and hexamples.jar (containing "wordcount" > application). For Hadoop 1, we would make two copies of the same > hadoop-examples.jar application and call them hsleep.jar and examples.jar. > The three approaches mentioned here would require least of changes. My > inclination is towards (2). > Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5115) Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2
Deepesh Khandelwal created HIVE-5115: Summary: Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2 Key: HIVE-5115 URL: https://issues.apache.org/jira/browse/HIVE-5115 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Deepesh Khandelwal Assignee: Deepesh Khandelwal In the webhcat e2e testsuite we have two MR job submission tests TestMapReduce_1 (in jobsubmission.conf) runs the hadoop "wordcount" example. Intention of this one is to test MR job submission using WebHCat. TestHeartbeat_2 (in jobsubmission2.conf) runs the hadoop "sleep" example. Intention of this one is to test a long running (>10min) WebHCat MR job, see HIVE-4808. In Hadoop 1, both of these example MR applications are packaged in hadoop-examples.jar In Hadoop 2, "sleep" job is bundled in hadoop-mapreduce-client-jobclient.jar and "wordcount" is bundled in hadoop-mapreduce-examples.jar Currently the webhcat tests assume that both these MR applications are in one jar that we copy as hexamples.jar. To run these against Hadoop 2 I can think of three simple solutions: (1) Stick with one jar and run "sleep" application in the TestMapReduce_1 test as well. (2) Eliminate the test TestMapReduce_1 as TestHeartbeat_2 runs a MR job as well. (3) Require two different jars for Hadoop 2 and call them hclient.jar (containing "sleep" application) and hexamples.jar (containing "wordcount" application). For Hadoop 1, we would make two copies of the same hadoop-examples.jar application and call them hsleep.jar and examples.jar. The three approaches mentioned here would require least of changes. My inclination is towards (2). Let me know what you think and I can provide the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4940) udaf_percentile_approx.q is not deterministic
[ https://issues.apache.org/jira/browse/HIVE-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743032#comment-13743032 ] Hudson commented on HIVE-4940: -- ABORTED: Integrated in Hive-trunk-hadoop2 #366 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/366/]) HIVE-4940 udaf_percentile_approx.q is not deterministic (Navis via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514771) * /hive/trunk/ql/src/test/queries/clientpositive/udaf_percentile_approx.q * /hive/trunk/ql/src/test/queries/clientpositive/udaf_percentile_approx_20.q * /hive/trunk/ql/src/test/queries/clientpositive/udaf_percentile_approx_23.q * /hive/trunk/ql/src/test/results/clientpositive/udaf_percentile_approx.q.out * /hive/trunk/ql/src/test/results/clientpositive/udaf_percentile_approx_20.q.out * /hive/trunk/ql/src/test/results/clientpositive/udaf_percentile_approx_23.q.out > udaf_percentile_approx.q is not deterministic > - > > Key: HIVE-4940 > URL: https://issues.apache.org/jira/browse/HIVE-4940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Navis >Assignee: Navis >Priority: Minor > Fix For: 0.12.0 > > Attachments: HIVE-4940.D12189.1.patch > > > Makes different result for 20(S) and 23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5105) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap
[ https://issues.apache.org/jira/browse/HIVE-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743031#comment-13743031 ] Hudson commented on HIVE-5105: -- ABORTED: Integrated in Hive-trunk-hadoop2 #366 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/366/]) HIVE-5105 HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap (Eugene Koifman via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514929) * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/data/schema/HCatSchema.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hcatalog/data/schema/TestHCatSchema.java > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up > fieldPositionMap > - > > Key: HIVE-5105 > URL: https://issues.apache.org/jira/browse/HIVE-5105 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 0.12.0 > > Attachments: HIVE-5105.patch > > > org.apache.hcatalog.data.schema.HCatSchema.remove(HCatFieldSchema > hcatFieldSchema) makes the following call: > fieldPositionMap.remove(hcatFieldSchema); > but fieldPositionMap is of type Map so the element is not > getting removed > Here's a detailed comment from [~sushanth] > The result is that that the name will not be removed from fieldPositionMap. > This results in 2 things: > a) If anyone tries to append a field to a hcatschema after having removed > that field, it shouldn't fail, but it will. > b) If anyone asks for the position of the removed field by name, it will > still give the position. > Now, there is only one place in hcat code where we remove a field, and that > is called from HCatOutputFormat.setSchema, where we try to detect if the user > specified partition column names in the schema when they shouldn't have, and > if they did, we remove it. Normally, people do not specify this, and this > check tends to be superfluous. > Once we do this, we wind up serializing that new object (after performing > some validations), and this does appear to stay through the serialization > (and eventual deserialization) which is very worrying. > However, we are luckily saved by the fact that we do not append that field to > it at any time(all appends in hcat code are done on newly initialized > HCatSchema objects which have had no removes done on them), and we don't ask > for the position of something we do not expect to be there(harder to verify > for certain, but seems to be the case on inspection). > The main part that gives me worry is that HCatSchema is part of our public > interface for HCat, in that M/R programs that use HCat can use it, and thus, > they might have more interesting usage patterns that are hitting this bug. > I can't think of any currently open bugs that is caused by this because of > the rarity of the situation, but nevertheless, something we should fix > immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-5100) RCFile::sync(long) missing 1 byte in System.arraycopy()
[ https://issues.apache.org/jira/browse/HIVE-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-5100: - Assignee: Gopal V > RCFile::sync(long) missing 1 byte in System.arraycopy() > - > > Key: HIVE-5100 > URL: https://issues.apache.org/jira/browse/HIVE-5100 > Project: Hive > Issue Type: Bug >Reporter: tagus wang >Assignee: Gopal V > > this has a bug in this: > System.arraycopy(buffer, buffer.length - prefix - 1, buffer, 0, prefix); > it should be > System.arraycopy(buffer, buffer.length - prefix, buffer, 0, prefix); > it is missing 1 byte at the end. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742925#comment-13742925 ] Brock Noland commented on HIVE-4388: [~stack] looks like the hbase.defaults.for.version in hbase-common-0.95.2-hadoop1.jar is 0.95.2-hadoop2. > HBase tests fail against Hadoop 2 > - > > Key: HIVE-4388 > URL: https://issues.apache.org/jira/browse/HIVE-4388 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Reporter: Gunther Hagleitner >Assignee: Brock Noland > Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, > HIVE-4388.patch, HIVE-4388-wip.txt > > > Currently we're building by default against 0.92. When you run against hadoop > 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. > HIVE-3861 upgrades the version of hbase used. This will get you past the > problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5105) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap
[ https://issues.apache.org/jira/browse/HIVE-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742894#comment-13742894 ] Hudson commented on HIVE-5105: -- FAILURE: Integrated in Hive-trunk-h0.21 #2274 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2274/]) HIVE-5105 HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap (Eugene Koifman via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514929) * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/data/schema/HCatSchema.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hcatalog/data/schema/TestHCatSchema.java > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up > fieldPositionMap > - > > Key: HIVE-5105 > URL: https://issues.apache.org/jira/browse/HIVE-5105 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 0.12.0 > > Attachments: HIVE-5105.patch > > > org.apache.hcatalog.data.schema.HCatSchema.remove(HCatFieldSchema > hcatFieldSchema) makes the following call: > fieldPositionMap.remove(hcatFieldSchema); > but fieldPositionMap is of type Map so the element is not > getting removed > Here's a detailed comment from [~sushanth] > The result is that that the name will not be removed from fieldPositionMap. > This results in 2 things: > a) If anyone tries to append a field to a hcatschema after having removed > that field, it shouldn't fail, but it will. > b) If anyone asks for the position of the removed field by name, it will > still give the position. > Now, there is only one place in hcat code where we remove a field, and that > is called from HCatOutputFormat.setSchema, where we try to detect if the user > specified partition column names in the schema when they shouldn't have, and > if they did, we remove it. Normally, people do not specify this, and this > check tends to be superfluous. > Once we do this, we wind up serializing that new object (after performing > some validations), and this does appear to stay through the serialization > (and eventual deserialization) which is very worrying. > However, we are luckily saved by the fact that we do not append that field to > it at any time(all appends in hcat code are done on newly initialized > HCatSchema objects which have had no removes done on them), and we don't ask > for the position of something we do not expect to be there(harder to verify > for certain, but seems to be the case on inspection). > The main part that gives me worry is that HCatSchema is part of our public > interface for HCat, in that M/R programs that use HCat can use it, and thus, > they might have more interesting usage patterns that are hitting this bug. > I can't think of any currently open bugs that is caused by this because of > the rarity of the situation, but nevertheless, something we should fix > immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5105) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap
[ https://issues.apache.org/jira/browse/HIVE-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742877#comment-13742877 ] Hudson commented on HIVE-5105: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #131 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/131/]) HIVE-5105 HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap (Eugene Koifman via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514929) * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/data/schema/HCatSchema.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hcatalog/data/schema/TestHCatSchema.java > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up > fieldPositionMap > - > > Key: HIVE-5105 > URL: https://issues.apache.org/jira/browse/HIVE-5105 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 0.12.0 > > Attachments: HIVE-5105.patch > > > org.apache.hcatalog.data.schema.HCatSchema.remove(HCatFieldSchema > hcatFieldSchema) makes the following call: > fieldPositionMap.remove(hcatFieldSchema); > but fieldPositionMap is of type Map so the element is not > getting removed > Here's a detailed comment from [~sushanth] > The result is that that the name will not be removed from fieldPositionMap. > This results in 2 things: > a) If anyone tries to append a field to a hcatschema after having removed > that field, it shouldn't fail, but it will. > b) If anyone asks for the position of the removed field by name, it will > still give the position. > Now, there is only one place in hcat code where we remove a field, and that > is called from HCatOutputFormat.setSchema, where we try to detect if the user > specified partition column names in the schema when they shouldn't have, and > if they did, we remove it. Normally, people do not specify this, and this > check tends to be superfluous. > Once we do this, we wind up serializing that new object (after performing > some validations), and this does appear to stay through the serialization > (and eventual deserialization) which is very worrying. > However, we are luckily saved by the fact that we do not append that field to > it at any time(all appends in hcat code are done on newly initialized > HCatSchema objects which have had no removes done on them), and we don't ask > for the position of something we do not expect to be there(harder to verify > for certain, but seems to be the case on inspection). > The main part that gives me worry is that HCatSchema is part of our public > interface for HCat, in that M/R programs that use HCat can use it, and thus, > they might have more interesting usage patterns that are hitting this bug. > I can't think of any currently open bugs that is caused by this because of > the rarity of the situation, but nevertheless, something we should fix > immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
