[jira] [Created] (HIVE-4268) Beeline should support the -f option

[Rob Weltman (JIRA)]

Rob Weltman created HIVE-4268:
-

 Summary: Beeline should support the -f option
 Key: HIVE-4268
 URL: https://issues.apache.org/jira/browse/HIVE-4268
 Project: Hive
  Issue Type: Improvement
  Components: CLI
Affects Versions: 0.10.0
Reporter: Rob Weltman
 Fix For: 0.11.0


Beeline should support the -f option (pass in a script to execute) for 
compatibility with the Hive CLI.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive 896, Windowing and Table Functions: merging ptf-windowing into trunk

[Butani, Harish]

Hi,

Just uploaded a patch for HIVE 896. This is for Windowing and Partitioned Table 
Function support in HQL.
The Windowing support is inline with the SQL Standard. See HIVE 4197 for 
details. We have over
100 tests for Windowing and about 50 tests for PTFs. The work done can be 
guaged by the Jiras in the PTF-Windowing
component. Just ran the entire testsuite, all tests pass.

Given that a branch for hive 0.11 is being planned soon I would like to propose 
merging the ptf-windowing
branch to trunk now. So that this feature can be made available in hive 0.11.

regards,
Harish Butani

[jira] [Commented] (HIVE-896) Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617954#comment-13617954
 ] 

Harish Butani commented on HIVE-896:


uploaded .5

> Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.
> ---
>
> Key: HIVE-896
> URL: https://issues.apache.org/jira/browse/HIVE-896
> Project: Hive
>  Issue Type: New Feature
>  Components: OLAP, UDF
>Reporter: Amr Awadallah
>Priority: Minor
> Attachments: DataStructs.pdf, HIVE-896.1.patch.txt, 
> Hive-896.2.patch.txt, hive-896.3.patch.txt, HIVE-896.4.patch, 
> HIVE-896.5.patch.txt
>
>
> Windowing functions are very useful for click stream processing and similar 
> time-series/sliding-window analytics.
> More details at:
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1006709
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007059
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007032
> -- amr

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-896) Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-896:
---

Attachment: HIVE-896.5.patch.txt

> Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive.
> ---
>
> Key: HIVE-896
> URL: https://issues.apache.org/jira/browse/HIVE-896
> Project: Hive
>  Issue Type: New Feature
>  Components: OLAP, UDF
>Reporter: Amr Awadallah
>Priority: Minor
> Attachments: DataStructs.pdf, HIVE-896.1.patch.txt, 
> Hive-896.2.patch.txt, hive-896.3.patch.txt, HIVE-896.4.patch, 
> HIVE-896.5.patch.txt
>
>
> Windowing functions are very useful for click stream processing and similar 
> time-series/sliding-window analytics.
> More details at:
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1006709
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007059
> http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007032
> -- amr

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4262) fix last_value UDAF behavior

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4262:
--

Attachment: HIVE-4262.D9837.1.patch

hbutani requested code review of "HIVE-4262 [jira] fix last_value UDAF 
behavior".

Reviewers: JIRA, ashutoshc

fix last_value

When an Order Spec is present in an Over clause
last_value should return the value from the last row that matches the current 
row's Sort Key.
See egs here http://msdn.microsoft.com/en-us/library/hh231517.aspx

TEST PLAN
  added new test to window_navfn.q

REVISION DETAIL
  https://reviews.facebook.net/D9837

AFFECTED FILES
  data/files/flights_tiny.txt
  data/files/part.rc
  data/files/part.seq
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java
  ql/src/test/queries/clientpositive/windowing_navfn.q
  ql/src/test/results/clientpositive/windowing_navfn.q.out

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/23463/

To: JIRA, ashutoshc, hbutani


> fix last_value UDAF behavior
> 
>
> Key: HIVE-4262
> URL: https://issues.apache.org/jira/browse/HIVE-4262
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4262.D9837.1.patch
>
>
> When an Order Spec is present in an Over clause
> last_value should return the value from the last row that matches the current 
> row's Sort Key.
> See egs here http://msdn.microsoft.com/en-us/library/hh231517.aspx

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4107) Update Hive 0.10.0 RELEASE_NOTES.txt

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617910#comment-13617910
 ] 

Thejas M Nair commented on HIVE-4107:
-

Patch can be committed to trunk and 0.10 branch.


> Update Hive 0.10.0 RELEASE_NOTES.txt
> 
>
> Key: HIVE-4107
> URL: https://issues.apache.org/jira/browse/HIVE-4107
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 0.10.0
>Reporter: Lefty Leverenz
>Assignee: Thejas M Nair
> Fix For: 0.11.0
>
> Attachments: HIVE-4107.1.patch
>
>
> Hive release 0.10.0 includes a RELEASE_NOTES.txt file left over from release 
> 0.8.1 (branch-0.8-r2).
> It needs to be updated to match the JIRA change log here:  
> [https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745&styleName=Text&projectId=12310843].
> Thanks to Eric Chu for drawing attention to this problem on 
> u...@hive.apache.org.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4107) Update Hive 0.10.0 RELEASE_NOTES.txt

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4107:


Fix Version/s: 0.11.0
 Assignee: Thejas M Nair
   Status: Patch Available  (was: Open)

Updating the release notes up to 0.10 version.


> Update Hive 0.10.0 RELEASE_NOTES.txt
> 
>
> Key: HIVE-4107
> URL: https://issues.apache.org/jira/browse/HIVE-4107
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 0.10.0
>Reporter: Lefty Leverenz
>Assignee: Thejas M Nair
> Fix For: 0.11.0
>
> Attachments: HIVE-4107.1.patch
>
>
> Hive release 0.10.0 includes a RELEASE_NOTES.txt file left over from release 
> 0.8.1 (branch-0.8-r2).
> It needs to be updated to match the JIRA change log here:  
> [https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745&styleName=Text&projectId=12310843].
> Thanks to Eric Chu for drawing attention to this problem on 
> u...@hive.apache.org.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4107) Update Hive 0.10.0 RELEASE_NOTES.txt

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4107:


Attachment: HIVE-4107.1.patch

> Update Hive 0.10.0 RELEASE_NOTES.txt
> 
>
> Key: HIVE-4107
> URL: https://issues.apache.org/jira/browse/HIVE-4107
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 0.10.0
>Reporter: Lefty Leverenz
> Attachments: HIVE-4107.1.patch
>
>
> Hive release 0.10.0 includes a RELEASE_NOTES.txt file left over from release 
> 0.8.1 (branch-0.8-r2).
> It needs to be updated to match the JIRA change log here:  
> [https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745&styleName=Text&projectId=12310843].
> Thanks to Eric Chu for drawing attention to this problem on 
> u...@hive.apache.org.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4264) Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-4264:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Checked in.

> Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog
> -
>
> Key: HIVE-4264
> URL: https://issues.apache.org/jira/browse/HIVE-4264
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4264.patch
>
>
> The trunk HCatalog code needs to be moved into the right place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4263) Adjust build.xml package command to move all hcat jars and binaries into build

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-4263:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Alan!

> Adjust build.xml package command to move all hcat jars and binaries into build
> --
>
> Key: HIVE-4263
> URL: https://issues.apache.org/jira/browse/HIVE-4263
> Project: Hive
>  Issue Type: Sub-task
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4263.patch
>
>
> HIVE-4198 created a line in build.xml so that as part of the package target   
> hive-hcatalog-${version}.jar is moved into the top level build directory.  
> However, hcatalog produces several jars and several scripts.  The package 
> target needs to be changed to move all of these up into build.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.10.0-SNAPSHOT-h0.20.1 #107

[Apache Jenkins Server]

See 

--
[...truncated 6752 lines...]
[ivy:resolve]   [SUCCESSFUL ] javax.xml.bind#jaxb-api;2.2.2!jaxb-api.jar (100ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/javax/activation/activation/1.1/activation-1.1.jar
 ...
[ivy:resolve]  (61kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] javax.activation#activation;1.1!activation.jar 
(99ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-digester/commons-digester/1.8/commons-digester-1.8.jar
 ...
[ivy:resolve] .. (140kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
commons-digester#commons-digester;1.8!commons-digester.jar (124ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar
 ...
[ivy:resolve] .. (201kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
commons-beanutils#commons-beanutils-core;1.8.0!commons-beanutils-core.jar 
(126ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar
 ...
[ivy:resolve] . (184kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
commons-beanutils#commons-beanutils;1.7.0!commons-beanutils.jar (126ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar
 ...
[ivy:resolve] .. (28kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.thoughtworks.paranamer#paranamer;2.3!paranamer.jar (100ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/xerial/snappy/snappy-java/1.0.3.2/snappy-java-1.0.3.2.jar
 ...
[ivy:resolve] ... 
(972kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.xerial.snappy#snappy-java;1.0.3.2!snappy-java.jar(bundle) (117ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/jline/jline/0.9.94/jline-0.9.94.jar ...
[ivy:resolve]  (85kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] jline#jline;0.9.94!jline.jar (101ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/jboss/netty/netty/3.2.3.Final/netty-3.2.3.Final.jar
 ...
[ivy:resolve] .. (767kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.jboss.netty#netty;3.2.3.Final!netty.jar(bundle) (115ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-common/2.0.0-alpha/hadoop-yarn-common-2.0.0-alpha.jar
 ...
[ivy:resolve] 
.
 (2070kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-yarn-common;2.0.0-alpha!hadoop-yarn-common.jar (152ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-annotations/2.0.0-alpha/hadoop-annotations-2.0.0-alpha.jar
 ...
[ivy:resolve] .. (16kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-annotations;2.0.0-alpha!hadoop-annotations.jar (103ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/google/inject/extensions/guice-servlet/3.0/guice-servlet-3.0.jar
 ...
[ivy:resolve] . (63kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.google.inject.extensions#guice-servlet;3.0!guice-servlet.jar (101ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.0.0-alpha/hadoop-yarn-api-2.0.0-alpha.jar
 ...
[ivy:resolve] 
 (881kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-yarn-api;2.0.0-alpha!hadoop-yarn-api.jar (119ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/google/inject/guice/3.0/guice-3.0.jar ...
[ivy:resolve] ... (693kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] com.google.inject#guice;3.0!guice.jar (111ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.8/jersey-test-framework-grizzly2-1.8.jar
 ...
[ivy:resolve] .. (12kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.sun.jersey.jersey-test-framework#jersey-test-framework-grizzly2;1.8!jersey-test-framework-grizzly2.jar
 (99ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/contribs/jersey-guice/1.8/jersey-guice-1.8.jar
 ...
[ivy:resolve] .. (14kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.sun.jersey.contribs#jersey-guice;1.8!jersey-guice.jar (279ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/jdiff/jdiff/1.0.9/jdiff-1.0.9.jar ...
[ivy:resolve] . (126kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] jdiff#jdiff;1.0.9!jdiff.jar (102ms)
[ivy:resolve] download

[jira] [Created] (HIVE-4267) fix ResultSet.getString in hiveserver2 jdbc driver for binary, timestamp

[Thejas M Nair (JIRA)]

Thejas M Nair created HIVE-4267:
---

 Summary: fix ResultSet.getString in hiveserver2 jdbc driver for 
binary, timestamp
 Key: HIVE-4267
 URL: https://issues.apache.org/jira/browse/HIVE-4267
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, JDBC
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Fix For: 0.11.0


The jdbc driver for hive server2 is printing binary using byte[].toString() 
instead of first converting it string by using it as a string constructor 
argument. In case of timestamp field, it is printed with a .0 at the end, which 
is inconsistent with hive cli.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4263) Adjust build.xml package command to move all hcat jars and binaries into build

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617751#comment-13617751
 ] 

Ashutosh Chauhan commented on HIVE-4263:


+1 will commit soon.

> Adjust build.xml package command to move all hcat jars and binaries into build
> --
>
> Key: HIVE-4263
> URL: https://issues.apache.org/jira/browse/HIVE-4263
> Project: Hive
>  Issue Type: Sub-task
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4263.patch
>
>
> HIVE-4198 created a line in build.xml so that as part of the package target   
> hive-hcatalog-${version}.jar is moved into the top level build directory.  
> However, hcatalog produces several jars and several scripts.  The package 
> target needs to be changed to move all of these up into build.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4242) Predicate push down should also be provided to InputFormats

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4242:
--

Attachment: HIVE-4242.D9831.1.patch

omalley requested code review of "HIVE-4242 [jira] Predicate push down should 
also be provided to InputFormats".

Reviewers: JIRA

HIVE-4242 predicate pushdown for all input formats

Currently, the push down predicate is only provided to native tables if the 
hive.optimize.index.filter configuration variable is set. There is no reason to 
prevent InputFormats from getting the required information to do predicate push 
down.

Obviously, this will be very useful for ORC.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D9831

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/23457/

To: JIRA, omalley


> Predicate push down should also be provided to InputFormats
> ---
>
> Key: HIVE-4242
> URL: https://issues.apache.org/jira/browse/HIVE-4242
> Project: Hive
>  Issue Type: Bug
>  Components: StorageHandler
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: HIVE-4242.D9831.1.patch
>
>
> Currently, the push down predicate is only provided to native tables if the 
> hive.optimize.index.filter configuration variable is set. There is no reason 
> to prevent InputFormats from getting the required information to do predicate 
> push down.
> Obviously, this will be very useful for ORC.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3874) Create a new Optimized Row Columnar file format for Hive

[Owen O'Malley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Owen O'Malley updated HIVE-3874:


Fix Version/s: 0.11.0

> Create a new Optimized Row Columnar file format for Hive
> 
>
> Key: HIVE-3874
> URL: https://issues.apache.org/jira/browse/HIVE-3874
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.11.0
>
> Attachments: hive.3874.2.patch, HIVE-3874.D8529.1.patch, 
> HIVE-3874.D8529.2.patch, HIVE-3874.D8529.3.patch, HIVE-3874.D8529.4.patch, 
> HIVE-3874.D8871.1.patch, OrcFileIntro.pptx, orc.tgz
>
>
> There are several limitations of the current RC File format that I'd like to 
> address by creating a new format:
> * each column value is stored as a binary blob, which means:
> ** the entire column value must be read, decompressed, and deserialized
> ** the file format can't use smarter type-specific compression
> ** push down filters can't be evaluated
> * the start of each row group needs to be found by scanning
> * user metadata can only be added to the file when the file is created
> * the file doesn't store the number of rows per a file or row group
> * there is no mechanism for seeking to a particular row number, which is 
> required for external indexes.
> * there is no mechanism for storing light weight indexes within the file to 
> enable push-down filters to skip entire row groups.
> * the type of the rows aren't stored in the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4265) HCatalog branches need to move out of trunk/hcatalog/historical

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617716#comment-13617716
 ] 

Ashutosh Chauhan commented on HIVE-4265:


I am fine with either choice, though 2 appeals a bit more to me. 

> HCatalog branches need to move out of trunk/hcatalog/historical
> ---
>
> Key: HIVE-4265
> URL: https://issues.apache.org/jira/browse/HIVE-4265
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
>
> When I moved the HCatalog code into Hive I put it all under trunk/hcatalog 
> (as we had discussed).  However this is causing checkouts and 'ant tar' both 
> to pick up the old hcatalog branches, which is not what we want.  We need to 
> move this branch code somewhere else.  Two options have been suggested:
> # Put it in branches, prefixing 'hcatalog-' to the branch name.  So 
> trunk/hcatalog/historical/branches/branch-0.5 would move to 
> branches/hcatalog-branch-0.5
> # Put it in a top level hcatalog-historical directory.
> Either is fine, but we need to do this soon to avoid plugging checkouts and 
> builds with dead code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4238) Integrate HCatalog site into Hive site

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-4238:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to hive/site. Thanks, Alan!

> Integrate HCatalog site into Hive site
> --
>
> Key: HIVE-4238
> URL: https://issues.apache.org/jira/browse/HIVE-4238
> Project: Hive
>  Issue Type: Sub-task
>  Components: Website
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4238.patch
>
>
> Content from HCatalog's site needs to be integrated with Hive's website.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4138) ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4138:
--

Attachment: HIVE-4138.D9219.2.patch

omalley updated the revision "HIVE-4138 [jira] ORC's union object inspector 
returns a type name that isn't parseable by TypeInfoUtils".

  Updated testcase I missed and realized that I needed to make it "uniontype"
  instead of "union"

Reviewers: JIRA

REVISION DETAIL
  https://reviews.facebook.net/D9219

CHANGE SINCE LAST DIFF
  https://reviews.facebook.net/D9219?vs=29397&id=30783#toc

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcUnion.java
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcStruct.java

To: JIRA, omalley


> ORC's union object inspector returns a type name that isn't parseable by 
> TypeInfoUtils
> --
>
> Key: HIVE-4138
> URL: https://issues.apache.org/jira/browse/HIVE-4138
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: h-4138.patch, HIVE-4138.D9219.1.patch, 
> HIVE-4138.D9219.2.patch
>
>
> Currently the typename returned by ORC's union object inspector isn't 
> parseable by TypeInfoUtils. The format needs to be union.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4238) Integrate HCatalog site into Hive site

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617684#comment-13617684
 ] 

Ashutosh Chauhan commented on HIVE-4238:


+1. Will commit soon.

> Integrate HCatalog site into Hive site
> --
>
> Key: HIVE-4238
> URL: https://issues.apache.org/jira/browse/HIVE-4238
> Project: Hive
>  Issue Type: Sub-task
>  Components: Website
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4238.patch
>
>
> Content from HCatalog's site needs to be integrated with Hive's website.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 2035 - Still Failing

[Apache Jenkins Server]

Changes for Build #2032
[namit] HIVE-4219 explain dependency does not capture the input table
(Namit via Gang Tim Liu)


Changes for Build #2033
[gates] Removing old branches to limit size of Hive downloads.

[gates] Removing tags directory as we no longer need them and they're in the 
history.

[gates] Moving HCatalog into Hive.

[gates] Test that perms work for hcatalog

[hashutosh] HIVE-4007 : Create abstract classes for serializer and deserializer 
(Namit Jain via Ashutosh Chauhan)

[hashutosh] HIVE-3381 : Result of outer join is not valid (Navis via Ashutosh 
Chauhan)

[hashutosh] HIVE-3980 : Cleanup after 3403 (Namit Jain via Ashutosh Chauhan)

[hashutosh] HIVE-4042 : ignore mapjoin hint (Namit Jain via Ashutosh Chauhan)

[namit] HIVE-3348 semi-colon in comments in .q file does not work
(Nick Collins via namit)

[namit] HIVE-4212 sort merge join should work for outer joins for more than 8 
inputs
(Namit via Gang Tim Liu)


Changes for Build #2034
[namit] HIVE-3958 support partial scan for analyze command - RCFile
(Gang Tim Liu via namit)


Changes for Build #2035
[kevinwilfong] HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to 
check if table exists. (Gang Tim Liu via kevinwilfong)

[gangtimliu] HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi 
Gang Tim Liu)

[gangtimliu] HIVE-4155: Expose ORC's FileDump as a service

[gangtimliu] HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin 
Wilfong vi Gang Tim Liu)

[namit] HIVE-4149 wrong results big outer joins with array of ints
(Navis via namit)




1 tests failed.
FAILED:  org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform1

Error Message:
Timeout occurred. Please note the time in the report does not reflect the time 
until the timeout.

Stack Trace:
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in 
the report does not reflect the time until the timeout.
at 
net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:299)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #2035)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/2035/ to 
view the results.

[jira] [Commented] (HIVE-4155) Expose ORC's FileDump as a service

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617653#comment-13617653
 ] 

Hudson commented on HIVE-4155:
--

Integrated in Hive-trunk-h0.21 #2035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2035/])
HIVE-4155: Expose ORC's FileDump as a service (Revision 1462352)

 Result = FAILURE
gangtimliu : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462352
Files : 
* /hive/trunk/bin/ext/orcfiledump.sh
* /hive/trunk/bin/hive


> Expose ORC's FileDump as a service
> --
>
> Key: HIVE-4155
> URL: https://issues.apache.org/jira/browse/HIVE-4155
> Project: Hive
>  Issue Type: New Feature
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4155.1.patch.txt
>
>
> Expose ORC's FileDump class as a service similar to RC File Cat
> e.g.
> hive --orcfiledump 
> Should run FileDump on the file.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4235) CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617654#comment-13617654
 ] 

Hudson commented on HIVE-4235:
--

Integrated in Hive-trunk-h0.21 #2035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2035/])
HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to check if 
table exists. (Gang Tim Liu via kevinwilfong) (Revision 1462373)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462373
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java


> CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists
> 
>
> Key: HIVE-4235
> URL: https://issues.apache.org/jira/browse/HIVE-4235
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC, Query Processor, SQL
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Fix For: 0.11.0
>
> Attachments: HIVE-4235.patch.1
>
>
> CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists.
> It uses Hive.java's getTablesByPattern(...) to check if table exists. It 
> involves regular expression and eventually database join. Very efficient. It 
> can cause database lock time increase and hurt db performance if a lot of 
> such commands hit database.
> The suggested approach is to use getTable(...) since we know tablename already

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4159) RetryingHMSHandler doesn't retry in enough cases

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617652#comment-13617652
 ] 

Hudson commented on HIVE-4159:
--

Integrated in Hive-trunk-h0.21 #2035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2035/])
HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin Wilfong 
vi Gang Tim Liu) (Revision 1462350)

 Result = FAILURE
gangtimliu : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462350
Files : 
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingHMSHandler.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/AlternateFailurePreListener.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRetryingHMSHandler.java


> RetryingHMSHandler doesn't retry in enough cases
> 
>
> Key: HIVE-4159
> URL: https://issues.apache.org/jira/browse/HIVE-4159
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4159.1.patch.txt
>
>
> HIVE-3524 introduced a change which caused JDOExceptions to be wrapped in 
> MetaExceptions.  This caused the RetryingHMSHandler to not retry on these 
> exceptions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4157) ORC runs out of heap when writing

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617651#comment-13617651
 ] 

Hudson commented on HIVE-4157:
--

Integrated in Hive-trunk-h0.21 #2035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2035/])
HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi Gang Tim 
Liu) (Revision 1462363)

 Result = FAILURE
gangtimliu : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462363
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OutStream.java


> ORC runs out of heap when writing
> -
>
> Key: HIVE-4157
> URL: https://issues.apache.org/jira/browse/HIVE-4157
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4157.1.patch.txt
>
>
> The OutStream class used by the ORC file format seems to aggressively 
> allocate memory for ByteBuffers and doesn't seem too eager to give it back.
> This causes issues with heap space, particularly when a wide tables/dynamic 
> partitions are involved.
> As a first step to resolving this problem, the OutStream class can be 
> modified to lazily allocate memory, and more actively make it available for 
> garbage collection.
> Follow ups could include checking the amount of free memory as part of 
> determining if a spill is needed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4149) wrong results big outer joins with array of ints

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617650#comment-13617650
 ] 

Hudson commented on HIVE-4149:
--

Integrated in Hive-trunk-h0.21 #2035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2035/])
HIVE-4149 wrong results big outer joins with array of ints
(Navis via namit) (Revision 1461959)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461959
Files : 
* /hive/trunk/data/files/tiny_a.txt
* /hive/trunk/data/files/tiny_b.txt
* /hive/trunk/ql/src/test/queries/clientpositive/join_array.q
* /hive/trunk/ql/src/test/results/clientpositive/join_array.q.out
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyArray.java
* 
/hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java


> wrong results big outer joins with array of ints
> 
>
> Key: HIVE-4149
> URL: https://issues.apache.org/jira/browse/HIVE-4149
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Navis
>Priority: Critical
> Fix For: 0.11.0
>
> Attachments: hive.4149.1.patch, HIVE-4149.D9747.1.patch, 
> HIVE-4149.D9747.2.patch
>
>
> Consider the following query:
> create table tinyA(a bigint, b bigint) stored as textfile;
> create table tinyB(a bigint, bList array) stored as textfile;
> load data local inpath '../data/files/tiny_a' into table tinyA;
> load data local inpath '../data/files/tiny_b' into table tinyB;
> select * from tinyA;
> select * from tinyB;
> select tinyB.a, tinyB.bList from tinyB full outer join tinyA on tinyB.a = 
> tinyA.a;
> The results are wrong

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4266) Refactor HCatalog code to org.apache.hive.hcatalog

[Alan Gates (JIRA)]

Alan Gates created HIVE-4266:


 Summary: Refactor HCatalog code to org.apache.hive.hcatalog
 Key: HIVE-4266
 URL: https://issues.apache.org/jira/browse/HIVE-4266
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.11.0
Reporter: Alan Gates
Assignee: Alan Gates
 Fix For: 0.11.0


Currently HCatalog code is in packages org.apache.hcatalog.  It needs to now 
move to org.apache.hive.hcatalog.  Shell classes/interface need to be created 
for public facing classes so that user's code does not break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4265) HCatalog branches need to move out of trunk/hcatalog/historical

[Alan Gates (JIRA)]

Alan Gates created HIVE-4265:


 Summary: HCatalog branches need to move out of 
trunk/hcatalog/historical
 Key: HIVE-4265
 URL: https://issues.apache.org/jira/browse/HIVE-4265
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.11.0
Reporter: Alan Gates
Assignee: Alan Gates
 Fix For: 0.11.0


When I moved the HCatalog code into Hive I put it all under trunk/hcatalog (as 
we had discussed).  However this is causing checkouts and 'ant tar' both to 
pick up the old hcatalog branches, which is not what we want.  We need to move 
this branch code somewhere else.  Two options have been suggested:
# Put it in branches, prefixing 'hcatalog-' to the branch name.  So 
trunk/hcatalog/historical/branches/branch-0.5 would move to 
branches/hcatalog-branch-0.5
# Put it in a top level hcatalog-historical directory.

Either is fine, but we need to do this soon to avoid plugging checkouts and 
builds with dead code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4264) Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-4264:
-

Attachment: HIVE-4264.patch

This patch file is for review and historical purposes only.  I need to do the 
actual checkin because it involves a series of svn mv's that this patch does 
not fully capture.  

> Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog
> -
>
> Key: HIVE-4264
> URL: https://issues.apache.org/jira/browse/HIVE-4264
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4264.patch
>
>
> The trunk HCatalog code needs to be moved into the right place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4264) Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-4264:
-

Status: Patch Available  (was: Open)

> Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog
> -
>
> Key: HIVE-4264
> URL: https://issues.apache.org/jira/browse/HIVE-4264
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4264.patch
>
>
> The trunk HCatalog code needs to be moved into the right place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4263) Adjust build.xml package command to move all hcat jars and binaries into build

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-4263:
-

Status: Patch Available  (was: Open)

> Adjust build.xml package command to move all hcat jars and binaries into build
> --
>
> Key: HIVE-4263
> URL: https://issues.apache.org/jira/browse/HIVE-4263
> Project: Hive
>  Issue Type: Sub-task
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4263.patch
>
>
> HIVE-4198 created a line in build.xml so that as part of the package target   
> hive-hcatalog-${version}.jar is moved into the top level build directory.  
> However, hcatalog produces several jars and several scripts.  The package 
> target needs to be changed to move all of these up into build.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4263) Adjust build.xml package command to move all hcat jars and binaries into build

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-4263:
-

Attachment: HIVE-4263.patch

This patch should be checked in in conjunction with HIVE-4264.  I need to check 
in HIVE-4264 because it contains a series 'svn mv' commands that will not be 
properly captured in the patch.  Please coordinate with me if you check this 
patch in.

> Adjust build.xml package command to move all hcat jars and binaries into build
> --
>
> Key: HIVE-4263
> URL: https://issues.apache.org/jira/browse/HIVE-4263
> Project: Hive
>  Issue Type: Sub-task
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.11.0
>
> Attachments: HIVE-4263.patch
>
>
> HIVE-4198 created a line in build.xml so that as part of the package target   
> hive-hcatalog-${version}.jar is moved into the top level build directory.  
> However, hcatalog produces several jars and several scripts.  The package 
> target needs to be changed to move all of these up into build.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4264) Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog

[Alan Gates (JIRA)]

Alan Gates created HIVE-4264:


 Summary: Move HCatalog trunk code from trunk/hcatalog/historical 
to trunk/hcatalog
 Key: HIVE-4264
 URL: https://issues.apache.org/jira/browse/HIVE-4264
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.11.0
Reporter: Alan Gates
Assignee: Alan Gates
 Fix For: 0.11.0


The trunk HCatalog code needs to be moved into the right place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4263) Adjust build.xml package command to move all hcat jars and binaries into build

[Alan Gates (JIRA)]

Alan Gates created HIVE-4263:


 Summary: Adjust build.xml package command to move all hcat jars 
and binaries into build
 Key: HIVE-4263
 URL: https://issues.apache.org/jira/browse/HIVE-4263
 Project: Hive
  Issue Type: Sub-task
  Components: Build Infrastructure
Affects Versions: 0.11.0
Reporter: Alan Gates
Assignee: Alan Gates
 Fix For: 0.11.0


HIVE-4198 created a line in build.xml so that as part of the package target   
hive-hcatalog-${version}.jar is moved into the top level build directory.  
However, hcatalog produces several jars and several scripts.  The package 
target needs to be changed to move all of these up into build.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4259) SEL operator created with missing columnExprMap for unions

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4259:
-

Status: Patch Available  (was: Open)

> SEL operator created with missing columnExprMap for unions
> --
>
> Key: HIVE-4259
> URL: https://issues.apache.org/jira/browse/HIVE-4259
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Attachments: HIVE-4259.1.patch
>
>
> Causes failures in some union_remove*.q testcases

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4259) SEL operator created with missing columnExprMap for unions

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617595#comment-13617595
 ] 

Gunther Hagleitner commented on HIVE-4259:
--

https://reviews.facebook.net/D9819

> SEL operator created with missing columnExprMap for unions
> --
>
> Key: HIVE-4259
> URL: https://issues.apache.org/jira/browse/HIVE-4259
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Attachments: HIVE-4259.1.patch
>
>
> Causes failures in some union_remove*.q testcases

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4259) SEL operator created with missing columnExprMap for unions

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4259:
-

Attachment: HIVE-4259.1.patch

Adds colExprMap to sel operator. Fixes test cases union_remove_[2,14,24,5,8,9].q

> SEL operator created with missing columnExprMap for unions
> --
>
> Key: HIVE-4259
> URL: https://issues.apache.org/jira/browse/HIVE-4259
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Attachments: HIVE-4259.1.patch
>
>
> Causes failures in some union_remove*.q testcases

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe

[Mark Wagner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated HIVE-4251:
--

Summary: Indices can't be built on tables whose schema info comes from 
SerDe  (was: Indices can't be built on tables who's schema info comes from 
SerDe)

> Indices can't be built on tables whose schema info comes from SerDe
> ---
>
> Key: HIVE-4251
> URL: https://issues.apache.org/jira/browse/HIVE-4251
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.10.0, 0.11.0, 0.10.1
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Fix For: 0.11.0, 0.10.1
>
> Attachments: HIVE-4251.1.patch
>
>
> Building indices on tables who get the schema information from the 
> deserializer (e.g. Avro backed tables) doesn't work because when the column 
> is checked to exist, the correct API isn't used.
> {code}
> hive> describe doctors;   
>
> OK
> # col_namedata_type   comment 
>
> numberint from deserializer   
> first_namestring  from deserializer   
> last_name string  from deserializer   
> Time taken: 0.215 seconds, Fetched: 5 row(s)
> hive> create index doctors_index on table doctors(number) as 'compact' with 
> deferred rebuild; 
> FAILED: Error in metadata: java.lang.RuntimeException: Check the index 
> columns, they should appear in the table being indexed.
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4240:
-

Attachment: hive.4240.3.patch

> optimize hive.enforce.bucketing and hive.enforce sorting insert
> ---
>
> Key: HIVE-4240
> URL: https://issues.apache.org/jira/browse/HIVE-4240
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch
>
>
> Consider the following scenario:
> set hive.optimize.bucketmapjoin = true;
> set hive.optimize.bucketmapjoin.sortedmerge = true;
> set hive.input.format = 
> org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> set hive.enforce.bucketing=true;
> set hive.enforce.sorting=true;
> set hive.exec.reducers.max = 1;
> set hive.merge.mapfiles=false;
> set hive.merge.mapredfiles=false;
> -- Create two bucketed and sorted tables
> CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) 
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS;
> CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) 
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS;
> FROM src
> INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *;
> -- Insert data into the bucketed table by selecting from another bucketed 
> table
> -- This should be a map-only operation
> INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1')
> SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1';
> We should not need a reducer to perform the above operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4258) Log logical plan tree for debugging

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4258:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Navis

> Log logical plan tree for debugging
> ---
>
> Key: HIVE-4258
> URL: https://issues.apache.org/jira/browse/HIVE-4258
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4258.D9801.1.patch
>
>
> Debugging or implementing optimizer, knowing the shape of logical plan helps 
> general progress of it.
> For example,
> select count(val) from (select a.key as key, b.value as array_val from src a 
> join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c 
> as val
> {noformat} 
> TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
>  -SEL[8]-UDTF[9]-LVJ[10]
> TS[0]-RS[3]-JOIN[4]
> {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4262) fix last_value UDAF behavior

[Harish Butani (JIRA)]

Harish Butani created HIVE-4262:
---

 Summary: fix last_value UDAF behavior
 Key: HIVE-4262
 URL: https://issues.apache.org/jira/browse/HIVE-4262
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Reporter: Harish Butani
Assignee: Harish Butani


When an Order Spec is present in an Over clause
last_value should return the value from the last row that matches the current 
row's Sort Key.
See egs here http://msdn.microsoft.com/en-us/library/hh231517.aspx

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4156) need to add protobuf classes to hive-exec.jar

[Owen O'Malley (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617457#comment-13617457
 ] 

Owen O'Malley commented on HIVE-4156:
-

Kevin,
   Thanks for the catch. You're right of course that I should have also 
included the snappy jar.

> need to add protobuf classes to hive-exec.jar
> -
>
> Key: HIVE-4156
> URL: https://issues.apache.org/jira/browse/HIVE-4156
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: HIVE-4156.D9375.1.patch, HIVE-4156.D9375.2.patch
>
>
> In some queries, the tasks fail when they can't find classes from the 
> protobuf library.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4156) need to add protobuf classes to hive-exec.jar

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4156:
--

Attachment: HIVE-4156.D9375.2.patch

omalley updated the revision "HIVE-4156 [jira] need to add protobuf classes to 
hive-exec.jar".

  I've added snappy to the exec jar also.

Reviewers: JIRA

REVISION DETAIL
  https://reviews.facebook.net/D9375

CHANGE SINCE LAST DIFF
  https://reviews.facebook.net/D9375?vs=29751&id=30759#toc

AFFECTED FILES
  ql/build.xml

To: JIRA, omalley


> need to add protobuf classes to hive-exec.jar
> -
>
> Key: HIVE-4156
> URL: https://issues.apache.org/jira/browse/HIVE-4156
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: HIVE-4156.D9375.1.patch, HIVE-4156.D9375.2.patch
>
>
> In some queries, the tasks fail when they can't find classes from the 
> protobuf library.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4199) ORC writer doesn't handle non-UTF8 encoded Text properly

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617454#comment-13617454
 ] 

Phabricator commented on HIVE-4199:
---

omalley has commented on the revision "HIVE-4199 [jira] ORC writer doesn't 
handle non-UTF8 encoded Text properly".

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/StringRedBlackTree.java:44 Let's 
just remove the current add method and move exclusively to Text for the 
StringRedBlackTree. I should have known better than using String internally.
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java:717 We're going 
to have the same problem here with the statistics. If we need to support 
non-utf8, we need to update the statistics to use Text also.

REVISION DETAIL
  https://reviews.facebook.net/D9501

To: kevinwilfong, sxyuan
Cc: JIRA, omalley


> ORC writer doesn't handle non-UTF8 encoded Text properly
> 
>
> Key: HIVE-4199
> URL: https://issues.apache.org/jira/browse/HIVE-4199
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Samuel Yuan
>Assignee: Samuel Yuan
>Priority: Minor
> Attachments: HIVE-4199.HIVE-4199.HIVE-4199.D9501.1.patch, 
> HIVE-4199.HIVE-4199.HIVE-4199.D9501.2.patch, 
> HIVE-4199.HIVE-4199.HIVE-4199.D9501.3.patch, 
> HIVE-4199.HIVE-4199.HIVE-4199.D9501.4.patch
>
>
> StringTreeWriter currently converts fields stored as Text objects into 
> Strings. This can lose information (see 
> http://en.wikipedia.org/wiki/Replacement_character#Replacement_character), 
> and is also unnecessary since the dictionary stores Text objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4261) union_remove_10 is failing on hadoop2 with assertion (root task with non-empty set of parents)

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-4261:


 Summary: union_remove_10 is failing on hadoop2 with assertion 
(root task with non-empty set of parents)
 Key: HIVE-4261
 URL: https://issues.apache.org/jira/browse/HIVE-4261
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
Priority: Critical
 Fix For: 0.11.0


Output seems to indicate that the stage plan is broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4260) union_remove_12, union_remove_13 are failing on hadoop2

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-4260:


 Summary: union_remove_12, union_remove_13 are failing on hadoop2
 Key: HIVE-4260
 URL: https://issues.apache.org/jira/browse/HIVE-4260
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
Priority: Critical


Problem goes away if hive.mapjoin.hint is set to true. Need to investivate why 
they are failing without it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4179:
-

Status: Patch Available  (was: Open)

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Fix For: 0.11.0
>
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch, HIVE-4179.3.patch, 
> HIVE-4179.4.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4259) SEL operator created with missing columnExprMap for unions

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-4259:


 Summary: SEL operator created with missing columnExprMap for unions
 Key: HIVE-4259
 URL: https://issues.apache.org/jira/browse/HIVE-4259
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
Priority: Critical


Causes failures in some union_remove*.q testcases

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617440#comment-13617440
 ] 

Gunther Hagleitner commented on HIVE-4179:
--

[~navis] I'd be happy to work on those as well. As I said before, there's 
additional issues with the union_remove testcases. I think those can/should be 
handled in follow up jiras though. This one fixes one particular issue and can 
be committed. Do you agree?

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Fix For: 0.11.0
>
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch, HIVE-4179.3.patch, 
> HIVE-4179.4.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4250) Closing lots of RecordWriters is slow

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4250:
--

Attachment: HIVE-4250.D9807.1.patch

omalley requested code review of "HIVE-4250 [jira] Closing lots of 
RecordWriters is slow".

Reviewers: JIRA

add threadpool for close writers

In FileSinkOperator, all of the RecordWriters are closed sequentially. For 
queries with a lot of dynamic partitions this can add substantially to the task 
time. For one query in particular, after processing all of the records in a few 
minutes the reduces spend 15 minutes closing all of the RC files.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D9807

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/23415/

To: JIRA, omalley


> Closing lots of RecordWriters is slow
> -
>
> Key: HIVE-4250
> URL: https://issues.apache.org/jira/browse/HIVE-4250
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: HIVE-4250.D9807.1.patch
>
>
> In FileSinkOperator, all of the RecordWriters are closed sequentially. For 
> queries with a lot of dynamic partitions this can add substantially to the 
> task time. For one query in particular, after processing all of the records 
> in a few minutes the reduces spend 15 minutes closing all of the RC files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-4109) Partition by column does not have to be in order by

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-4109.


Resolution: Fixed
  Assignee: Ashutosh Chauhan

Committed to branch. 

> Partition by column does not have to be in order by
> ---
>
> Key: HIVE-4109
> URL: https://issues.apache.org/jira/browse/HIVE-4109
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-4109.patch
>
>
> Cam up in the review of HIVE-4093.
> Ashutosh
> {noformat}
> I am not sure if this is illegal query. I tried following two queries in 
> postgres, both of them succeeded.
> select p_mfgr, avg(p_retailprice) over(partition by p_mfgr, p_type order by 
> p_mfgr) from part;
> select p_mfgr, avg(p_retailprice) over(partition by p_mfgr order by 
> p_type,p_mfgr) from part;
> {noformat}
> Harish
> {noformat}
> The first one doesn't make sense, right? Order on a subset of the partition 
> columns
> The second one: Can we do this with the Hive ReduceOp have the orderColumns 
> be in a different order than the key columns?
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-4255) update show_functions.q.out for functions added for windowing

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-4255.


Resolution: Fixed

> update show_functions.q.out for functions added for windowing
> -
>
> Key: HIVE-4255
> URL: https://issues.apache.org/jira/browse/HIVE-4255
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4255.1.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4109) Partition by column does not have to be in order by

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617370#comment-13617370
 ] 

Harish Butani commented on HIVE-4109:
-

+1

> Partition by column does not have to be in order by
> ---
>
> Key: HIVE-4109
> URL: https://issues.apache.org/jira/browse/HIVE-4109
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
> Attachments: HIVE-4109.patch
>
>
> Cam up in the review of HIVE-4093.
> Ashutosh
> {noformat}
> I am not sure if this is illegal query. I tried following two queries in 
> postgres, both of them succeeded.
> select p_mfgr, avg(p_retailprice) over(partition by p_mfgr, p_type order by 
> p_mfgr) from part;
> select p_mfgr, avg(p_retailprice) over(partition by p_mfgr order by 
> p_type,p_mfgr) from part;
> {noformat}
> Harish
> {noformat}
> The first one doesn't make sense, right? Order on a subset of the partition 
> columns
> The second one: Can we do this with the Hive ReduceOp have the orderColumns 
> be in a different order than the key columns?
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617281#comment-13617281
 ] 

Brock Noland commented on HIVE-2264:


Awesome! Thanks guys!

> Hive server is SHUTTING DOWN when invalid queries beeing executed.
> --
>
> Key: HIVE-2264
> URL: https://issues.apache.org/jira/browse/HIVE-2264
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Query Processor
>Affects Versions: 0.9.0
> Environment: SuSE-Linux-11
>Reporter: rohithsharma
>Assignee: Navis
>Priority: Blocker
> Fix For: 0.11.0
>
> Attachments: HIVE-2264.1.patch.txt, HIVE-2264-2.patch, 
> HIVE-2264.D9489.1.patch
>
>
> When invalid query is beeing executed, Hive server is shutting down.
> {noformat}
> "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds 
> string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"
> "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location 
> '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4258) Log logical plan tree for debugging

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617261#comment-13617261
 ] 

Namit Jain commented on HIVE-4258:
--

I agree, I end up doing some similar hack all the time

+1

> Log logical plan tree for debugging
> ---
>
> Key: HIVE-4258
> URL: https://issues.apache.org/jira/browse/HIVE-4258
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-4258.D9801.1.patch
>
>
> Debugging or implementing optimizer, knowing the shape of logical plan helps 
> general progress of it.
> For example,
> select count(val) from (select a.key as key, b.value as array_val from src a 
> join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c 
> as val
> {noformat} 
> TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
>  -SEL[8]-UDTF[9]-LVJ[10]
> TS[0]-RS[3]-JOIN[4]
> {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4240:
-

Attachment: hive.4240.2.patch

> optimize hive.enforce.bucketing and hive.enforce sorting insert
> ---
>
> Key: HIVE-4240
> URL: https://issues.apache.org/jira/browse/HIVE-4240
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4240.1.patch, hive.4240.2.patch
>
>
> Consider the following scenario:
> set hive.optimize.bucketmapjoin = true;
> set hive.optimize.bucketmapjoin.sortedmerge = true;
> set hive.input.format = 
> org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> set hive.enforce.bucketing=true;
> set hive.enforce.sorting=true;
> set hive.exec.reducers.max = 1;
> set hive.merge.mapfiles=false;
> set hive.merge.mapredfiles=false;
> -- Create two bucketed and sorted tables
> CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) 
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS;
> CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) 
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS;
> FROM src
> INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *;
> -- Insert data into the bucketed table by selecting from another bucketed 
> table
> -- This should be a map-only operation
> INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1')
> SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1';
> We should not need a reducer to perform the above operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4258) Log logical plan tree for debugging

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-4258:


Status: Patch Available  (was: Open)

Really trivial

> Log logical plan tree for debugging
> ---
>
> Key: HIVE-4258
> URL: https://issues.apache.org/jira/browse/HIVE-4258
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-4258.D9801.1.patch
>
>
> Debugging or implementing optimizer, knowing the shape of logical plan helps 
> general progress of it.
> For example,
> select count(val) from (select a.key as key, b.value as array_val from src a 
> join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c 
> as val
> {noformat} 
> TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
>  -SEL[8]-UDTF[9]-LVJ[10]
> TS[0]-RS[3]-JOIN[4]
> {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4258) Log logical plan tree for debugging

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4258:
--

Attachment: HIVE-4258.D9801.1.patch

navis requested code review of "HIVE-4258 [jira] Log logical plan tree for 
debugging".

Reviewers: JIRA

HIVE-4258 Log logical plan tree for debugging

Debugging or implementing optimizer, knowing the shape of logical plan helps 
general progress of it.

For example,
select count(val) from (select a.key as key, b.value as array_val from src a 
join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c as 
val

TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
 -SEL[8]-UDTF[9]-LVJ[10]
TS[0]-RS[3]-JOIN[4]

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D9801

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/23403/

To: JIRA, navis


> Log logical plan tree for debugging
> ---
>
> Key: HIVE-4258
> URL: https://issues.apache.org/jira/browse/HIVE-4258
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-4258.D9801.1.patch
>
>
> Debugging or implementing optimizer, knowing the shape of logical plan helps 
> general progress of it.
> For example,
> select count(val) from (select a.key as key, b.value as array_val from src a 
> join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c 
> as val
> {noformat} 
> TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
>  -SEL[8]-UDTF[9]-LVJ[10]
> TS[0]-RS[3]-JOIN[4]
> {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4258) Log logical plan tree for debugging

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-4258:


Description: 
Debugging or implementing optimizer, knowing the shape of logical plan helps 
general progress of it.

For example,
select count(val) from (select a.key as key, b.value as array_val from src a 
join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c as 
val
{noformat} 
TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
 -SEL[8]-UDTF[9]-LVJ[10]
TS[0]-RS[3]-JOIN[4]
{noformat} 

  was:
Debugging or implementing optimizer, knowing the shape of logical plan helps 
general progress of it.

For example,
select count(val) from (select a.key as key, b.value as array_val from src a 
join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c as 
val

TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
 -SEL[8]-UDTF[9]-LVJ[10]
TS[0]-RS[3]-JOIN[4]



> Log logical plan tree for debugging
> ---
>
> Key: HIVE-4258
> URL: https://issues.apache.org/jira/browse/HIVE-4258
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-4258.D9801.1.patch
>
>
> Debugging or implementing optimizer, knowing the shape of logical plan helps 
> general progress of it.
> For example,
> select count(val) from (select a.key as key, b.value as array_val from src a 
> join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c 
> as val
> {noformat} 
> TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
>  -SEL[8]-UDTF[9]-LVJ[10]
> TS[0]-RS[3]-JOIN[4]
> {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4258) Log logical plan tree for debugging

[Navis (JIRA)]

Navis created HIVE-4258:
---

 Summary: Log logical plan tree for debugging
 Key: HIVE-4258
 URL: https://issues.apache.org/jira/browse/HIVE-4258
 Project: Hive
  Issue Type: Improvement
  Components: Diagnosability
Reporter: Navis
Assignee: Navis
Priority: Trivial


Debugging or implementing optimizer, knowing the shape of logical plan helps 
general progress of it.

For example,
select count(val) from (select a.key as key, b.value as array_val from src a 
join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c as 
val

TS[1]-RS[2]-JOIN[4]-SEL[5]-LVF[6]-SEL[7]-LVJ[10]-SEL[11]-GBY[12]-RS[13]-GBY[14]-SEL[15]-FS[16]
 -SEL[8]-UDTF[9]-LVJ[10]
TS[0]-RS[3]-JOIN[4]


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617198#comment-13617198
 ] 

Navis commented on HIVE-4179:
-

And.. failing of 12, 13 is caused by mis-configuration in data/hive-site.xml 
(hive.ignore.mapjoin.hint should be true). I don't know why those are failing 
if it's set false.

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Fix For: 0.11.0
>
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch, HIVE-4179.3.patch, 
> HIVE-4179.4.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-4179:


Status: Open  (was: Patch Available)

I've ran union_remove_* tests and found mapping information is missing for type 
casting SEL operator, which is generated in 
SemanticAnalyzer.genInputSelectForUnion().

Packing needed colExprMap seemed to fix some part of problems. Could you do 
that?

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
>Priority: Critical
> Fix For: 0.11.0
>
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch, HIVE-4179.3.patch, 
> HIVE-4179.4.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-4257:
-

Priority: Minor  (was: Major)

> java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
> --
>
> Key: HIVE-4257
> URL: https://issues.apache.org/jira/browse/HIVE-4257
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.11.0
>Reporter: Teddy Choi
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-4257.1.patch.txt
>
>
> java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator 
> after executing dozens of Hive queries periodically, which inserts thousands 
> of rows. It may have a relation with DERBY-5098. To avoid this error, Hive 
> should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 
> 10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
> {noformat}
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617189#comment-13617189
 ] 

Teddy Choi commented on HIVE-4257:
--

If you get "java.lang.SecurityException: sealing violation" message, please 
remove derby-10.4.2.0.jar file from your target directories. 
(build/ivy/lib/default/, build/ivy/lib/compile/, build/ivy/lib/test/)

> java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
> --
>
> Key: HIVE-4257
> URL: https://issues.apache.org/jira/browse/HIVE-4257
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.11.0
>Reporter: Teddy Choi
> Fix For: 0.11.0
>
> Attachments: HIVE-4257.1.patch.txt
>
>
> java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator 
> after executing dozens of Hive queries periodically, which inserts thousands 
> of rows. It may have a relation with DERBY-5098. To avoid this error, Hive 
> should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 
> 10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
> {noformat}
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617179#comment-13617179
 ] 

Teddy Choi commented on HIVE-4257:
--

I attached a patch that makes Hive to use Derby 10.9.1.0, which is the latest 
official release in March 29, 2013. ( 
http://db.apache.org/derby/derby_downloads.html )

> java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
> --
>
> Key: HIVE-4257
> URL: https://issues.apache.org/jira/browse/HIVE-4257
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.11.0
>Reporter: Teddy Choi
> Fix For: 0.11.0
>
> Attachments: HIVE-4257.1.patch.txt
>
>
> java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator 
> after executing dozens of Hive queries periodically, which inserts thousands 
> of rows. It may have a relation with DERBY-5098. To avoid this error, Hive 
> should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 
> 10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
> {noformat}
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-4257:
-

Fix Version/s: 0.11.0
   Status: Patch Available  (was: Open)

> java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
> --
>
> Key: HIVE-4257
> URL: https://issues.apache.org/jira/browse/HIVE-4257
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.11.0
>Reporter: Teddy Choi
> Fix For: 0.11.0
>
> Attachments: HIVE-4257.1.patch.txt
>
>
> java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator 
> after executing dozens of Hive queries periodically, which inserts thousands 
> of rows. It may have a relation with DERBY-5098. To avoid this error, Hive 
> should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 
> 10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
> {noformat}
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-4257:
-

Attachment: HIVE-4257.1.patch.txt

> java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
> --
>
> Key: HIVE-4257
> URL: https://issues.apache.org/jira/browse/HIVE-4257
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.11.0
>Reporter: Teddy Choi
> Attachments: HIVE-4257.1.patch.txt
>
>
> java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator 
> after executing dozens of Hive queries periodically, which inserts thousands 
> of rows. It may have a relation with DERBY-5098. To avoid this error, Hive 
> should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 
> 10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
> {noformat}
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> 2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
> (JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing 
> aggregation. java.sql.SQLNonTransientConnectionException: No current 
> connection.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-4257:


 Summary: java.sql.SQLNonTransientConnectionException on 
JDBCStatsAggregator
 Key: HIVE-4257
 URL: https://issues.apache.org/jira/browse/HIVE-4257
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.11.0
Reporter: Teddy Choi
 Attachments: HIVE-4257.1.patch.txt

java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator after 
executing dozens of Hive queries periodically, which inserts thousands of rows. 
It may have a relation with DERBY-5098. To avoid this error, Hive should use a 
more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2, 10.9.1.0 or later). 
Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.

{noformat}
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
(JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
aggregation. java.sql.SQLNonTransientConnectionException: No current connection.
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
(JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing 
aggregation. java.sql.SQLNonTransientConnectionException: No current connection.
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator 
(JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing aggregation. 
java.sql.SQLNonTransientConnectionException: No current connection.
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4256) JDBC2 HiveConnection does not use the specified database

[Chris Drome (JIRA)]

Chris Drome created HIVE-4256:
-

 Summary: JDBC2 HiveConnection does not use the specified database
 Key: HIVE-4256
 URL: https://issues.apache.org/jira/browse/HIVE-4256
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, JDBC
Affects Versions: 0.11.0
Reporter: Chris Drome
Assignee: Chris Drome


HiveConnection ignores the database specified in the connection string when 
configuring the connection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira