[jira] [Resolved] (HIVE-7157) skipRows() method in TreeReader subclasses in RecordReaderImpl should check return value from stream.skip()
[
https://issues.apache.org/jira/browse/HIVE-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-7157.
--
Resolution: Later
> skipRows() method in TreeReader subclasses in RecordReaderImpl should check
> return value from stream.skip()
> ---
>
> Key: HIVE-7157
> URL: https://issues.apache.org/jira/browse/HIVE-7157
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Assignee: Swarnim Kulkarni
>Priority: Minor
>
> This is an example from DoubleTreeReader#skipRows():
> {code}
> stream.skip(items * 8);
> {code}
> skip() method returns actual number of bytes returned. The return value
> should be checked against input parameter.
> skipRows() method of BinaryTreeReader and StringDirectTreeReader classes has
> the same issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9014) Upgrade hbase dependency to 0.98.10
[ https://issues.apache.org/jira/browse/HIVE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HIVE-9014: - Summary: Upgrade hbase dependency to 0.98.10 (was: Upgrade hbase dependency to 0.98.8) > Upgrade hbase dependency to 0.98.10 > --- > > Key: HIVE-9014 > URL: https://issues.apache.org/jira/browse/HIVE-9014 > Project: Hive > Issue Type: Task > Reporter: Ted Yu > Assignee: Ted Yu >Priority: Minor > Attachments: HIVE-9014.patch > > > HBase 0.98.3 was an old release. > This JIRA upgrades the dependency to 0.98.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-8493) Access to HiveClientCache$CacheableHiveMetaStoreClient#isClosed should be synchronized
[
https://issues.apache.org/jira/browse/HIVE-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-8493.
--
Resolution: Later
> Access to HiveClientCache$CacheableHiveMetaStoreClient#isClosed should be
> synchronized
> --
>
> Key: HIVE-8493
> URL: https://issues.apache.org/jira/browse/HIVE-8493
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> Update to isClosed is protected by synchronizing on
> CacheableHiveMetaStoreClient.this
> {code}
> public boolean isClosed() {
> return isClosed;
> {code}
> The above access should be synchronized as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-8729) VectorizedRowBatch#selected is not restored upon exception in VectorFilterOperator#processOp()
[
https://issues.apache.org/jira/browse/HIVE-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-8729.
--
Resolution: Later
> VectorizedRowBatch#selected is not restored upon exception in
> VectorFilterOperator#processOp()
> --
>
> Key: HIVE-8729
> URL: https://issues.apache.org/jira/browse/HIVE-8729
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> Here is related code:
> {code}
> int [] selectedBackup = vrg.selected;
> vrg.selected = temporarySelected;
> ...
> if (vrg.size > 0) {
> forward(vrg, null);
> }
> {code}
> If forward() throws exception, selected is not restored.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8806) Potential null dereference in Metrics#incrementCounter()
[
https://issues.apache.org/jira/browse/HIVE-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-8806:
-
Resolution: Later
Status: Resolved (was: Patch Available)
> Potential null dereference in Metrics#incrementCounter()
>
>
> Key: HIVE-8806
> URL: https://issues.apache.org/jira/browse/HIVE-8806
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-8806.1.patch, HIVE-8806.2.patch
>
>
> {code}
> if (!metrics.hasKey(name)) {
> value = Long.valueOf(increment);
> set(name, value);
> } else {
> value = ((Long)get(name)) + increment;
> set(name, value);
> }
> {code}
> In the else block, if get(name) returns null, unboxing null object would lead
> to exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-9151) Checking s against null in TezJobMonitor#getNameWithProgress() should be done earlier
[
https://issues.apache.org/jira/browse/HIVE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-9151.
--
Resolution: Later
> Checking s against null in TezJobMonitor#getNameWithProgress() should be done
> earlier
> -
>
> Key: HIVE-9151
> URL: https://issues.apache.org/jira/browse/HIVE-9151
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> {code}
> int spaceRemaining = COLUMN_1_WIDTH - s.length() - 1;
> String trimmedVName = s;
> // if the vertex name is longer than column 1 width, trim it down
> // "Tez Merge File Work" will become "Tez Merge File.."
> if (s != null && s.length() > COLUMN_1_WIDTH) {
> {code}
> s is dereferenced first, rendering the null check ineffective.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-8927) Potential race condition on modification of dirtyAttributeInfoCache in MetricsMBeanImpl#getMBeanInfo()
[
https://issues.apache.org/jira/browse/HIVE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-8927.
--
Resolution: Later
> Potential race condition on modification of dirtyAttributeInfoCache in
> MetricsMBeanImpl#getMBeanInfo()
> --
>
> Key: HIVE-8927
> URL: https://issues.apache.org/jira/browse/HIVE-8927
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> {code}
> public MBeanInfo getMBeanInfo() {
> if (dirtyAttributeInfoCache) {
> synchronized(metricsMap) {
> attributeInfos = new MBeanAttributeInfo[metricsMap.size()];
> {code}
> The check of dirtyAttributeInfoCache is outside the synchronized block.
> There is possibility that different attributeInfos are returned due to race
> condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-8593) Unintended regex is used in ScriptOperator#blackListed()
[
https://issues.apache.org/jira/browse/HIVE-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-8593.
--
Resolution: Later
> Unintended regex is used in ScriptOperator#blackListed()
>
>
> Key: HIVE-8593
> URL: https://issues.apache.org/jira/browse/HIVE-8593
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> {code}
> for (String b : bls) {
> b.replaceAll(".", "_");
> {code}
> The dot can match any character.
> See
> http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#replaceAll(java.lang.String,%20java.lang.String)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9301) Potential null dereference in MoveTask#createTargetPath()
[
https://issues.apache.org/jira/browse/HIVE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268580#comment-14268580
]
Ted Yu commented on HIVE-9301:
--
I stepped through similar code in debugger - the single ampersand prevents
short circuit evaluation of the expression, leading to NPE.
> Potential null dereference in MoveTask#createTargetPath()
> -
>
> Key: HIVE-9301
> URL: https://issues.apache.org/jira/browse/HIVE-9301
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9301.patch
>
>
> {code}
> if (mkDirPath != null & !fs.exists(mkDirPath)) {
> {code}
> '&&' should be used instead of single ampersand.
> If mkDirPath is null, fs.exists() would still be called - resulting in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9301) Potential null dereference in MoveTask#createTargetPath()
[
https://issues.apache.org/jira/browse/HIVE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9301:
-
Assignee: Ted Yu
Status: Patch Available (was: Open)
> Potential null dereference in MoveTask#createTargetPath()
> -
>
> Key: HIVE-9301
> URL: https://issues.apache.org/jira/browse/HIVE-9301
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: HIVE-9301.patch
>
>
> {code}
> if (mkDirPath != null & !fs.exists(mkDirPath)) {
> {code}
> '&&' should be used instead of single ampersand.
> If mkDirPath is null, fs.exists() would still be called - resulting in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9301) Potential null dereference in MoveTask#createTargetPath()
[
https://issues.apache.org/jira/browse/HIVE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9301:
-
Attachment: HIVE-9301.patch
> Potential null dereference in MoveTask#createTargetPath()
> -
>
> Key: HIVE-9301
> URL: https://issues.apache.org/jira/browse/HIVE-9301
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Attachments: HIVE-9301.patch
>
>
> {code}
> if (mkDirPath != null & !fs.exists(mkDirPath)) {
> {code}
> '&&' should be used instead of single ampersand.
> If mkDirPath is null, fs.exists() would still be called - resulting in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9301) Potential null dereference in MoveTask#createTargetPath()
Ted Yu created HIVE-9301:
Summary: Potential null dereference in MoveTask#createTargetPath()
Key: HIVE-9301
URL: https://issues.apache.org/jira/browse/HIVE-9301
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
{code}
if (mkDirPath != null & !fs.exists(mkDirPath)) {
{code}
'&&' should be used instead of single ampersand.
If mkDirPath is null, fs.exists() would still be called - resulting in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265483#comment-14265483
]
Ted Yu commented on HIVE-9183:
--
I don't think TestMiniTezCliDriver is related to HBase.
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch, HIVE-9183.patch, HIVE-9183.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265347#comment-14265347
]
Ted Yu commented on HIVE-9183:
--
The latest patch would support both 0.98.x and 1.0.0
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch, HIVE-9183.patch, HIVE-9183.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Attachment: HIVE-9183.patch
Here is patch without change to pom.xml
The previous patch was for testing.
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch, HIVE-9183.patch, HIVE-9183.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Fwd: way to add custom udf jar in hadoop 2.x version
Forwarding Niels' question to hive mailing list. On Wed, Dec 31, 2014 at 1:24 AM, Niels Basjes wrote: > Thanks for the pointer. > This seems to work for functions. Is there something similar for CREATE > EXTERNAL TABLE ?? > > Niels > On Dec 31, 2014 8:13 AM, "Ted Yu" wrote: > >> Have you seen this thread ? >> >> http://search-hadoop.com/m/8er9TcALc/Hive+udf+custom+jar&subj=Best+way+to+add+custom+UDF+jar+in+HiveServer2 >> >> On Dec 30, 2014, at 10:56 PM, reena upadhyay wrote: >> >> Hi, >> >> I am using hadoop 2.4.0 version. I have created custom udf jar. I am >> trying to execute a simple select udf query using java hive jdbc client >> program. When hive execute the query using map reduce job, then the query >> execution get fails because the mapper is not able to locate the udf class. >> So I wanted to add the udf jar in hadoop environment permanently. Please >> suggest me a way to add this external jar for single node and multi node >> hadoop cluster. >> >> PS: I am using hive 0.13.1 version and I already have this custom udf jar >> added in HIVE_HOME/lib directory. >> >> >> Thanks >> >>
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Attachment: HIVE-9183.patch
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch, HIVE-9183.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: PreCommit QA not working ?
I wonder why patch attached to HIVE-9183 was not picked up by QA bot. Thanks On Sat, Dec 20, 2014 at 11:53 PM, Szehon Ho wrote: > Hi, > > The precommit builds in use are actually here: > > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/ > . > This is the job that files the HiveQA reports. > > We don't use builds.apache.org actively anymore. I dont know if somebody > is still trying it, as its still running as you point out. If not, maybe > somebody with privilege can disable that build in order to reduce > confusion. > > Thanks > Szehon >
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Attachment: (was: hive-9183-001.patch)
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Attachment: HIVE-9183-001.patch
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HIVE-9183-001.patch, hive-9183-001.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: PreCommit QA not working ?
I disabled the PreCommit QA job on apache Jenkins. Cheers On Dec 20, 2014, at 11:53 PM, Szehon Ho wrote: > Hi, > > The precommit builds in use are actually here: > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/. > This is the job that files the HiveQA reports. > > We don't use builds.apache.org actively anymore. I dont know if somebody > is still trying it, as its still running as you point out. If not, maybe > somebody with privilege can disable that build in order to reduce confusion. > > Thanks > Szehon
[jira] [Commented] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255010#comment-14255010
]
Ted Yu commented on HIVE-9183:
--
Running test suite with the following command:
mvn clean -Phadoop-2,dist package
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: hive-9183-001.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
PreCommit QA not working ?
Hi, Looking at recent QA runs, e.g.: https://builds.apache.org/job/PreCommit-Hive-Build/5295/console Description set: HIVE-9183 Finished: SUCCESS The test suite was not run. Build as early as #5265 had the same problem. Does someone have idea what caused this ? Thanks
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Assignee: Ted Yu
Status: Patch Available (was: Open)
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: hive-9183-001.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Attachment: hive-9183-001.patch
Patch v1 moves loading of high_scale_lib.Counter to runtime.
Change in pom.xml is to show that compilation passes against HBase 1.0.0
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
> Reporter: Ted Yu
> Attachments: hive-9183-001.patch
>
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job copy = new Job(jobConf);
> {code}
> However, loading of high_scale_lib.Counter should be done at runtime so that
> 0.98 and 1.0 releases are both supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
[
https://issues.apache.org/jira/browse/HIVE-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9183:
-
Description:
In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
Compilation against HBase 1.0.0 release would fail due to the missing
dependency.
The following change would allow compilation to pass:
{code}
diff --git
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
index 3218639..6f626f0 100644
---
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
+++
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
@@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
jobConf) {
* only need TableMapReduceUtil.addDependencyJars(jobConf) here.
*/
TableMapReduceUtil.addDependencyJars(
- jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
- org.cliffc.high_scale_lib.Counter.class); // this will be removed
for HBase 1.0
+ jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
+ // org.cliffc.high_scale_lib.Counter.class); this will be removed
for HBase 1.0
Set merged = new
LinkedHashSet(jobConf.getStringCollection("tmpjars"));
Job copy = new Job(jobConf);
{code}
However, loading of high_scale_lib.Counter should be done at runtime so that
0.98 and 1.0 releases are both supported.
was:
In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
Compilation against HBase 1.0.0 release would fail due to the missing
dependency.
The following change would allow compilation to pass:
{code}
diff --git
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
index 3218639..6f626f0 100644
---
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
+++
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
@@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
jobConf) {
* only need TableMapReduceUtil.addDependencyJars(jobConf) here.
*/
TableMapReduceUtil.addDependencyJars(
- jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
- org.cliffc.high_scale_lib.Counter.class); // this will be removed
for HBase 1.0
+ jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
+ // org.cliffc.high_scale_lib.Counter.class); this will be removed
for HBase 1.0
Set merged = new
LinkedHashSet(jobConf.getStringCollection("tmpjars"));
Job copy = new Job(jobConf);
{code}
However, a shim layer may be needed so that 0.98 and 1.0 releases are both
supported.
> Compilation against HBase 1.0.0 fails due to missing dependency on
> high_scale_lib.Counter
> -
>
> Key: HIVE-9183
> URL: https://issues.apache.org/jira/browse/HIVE-9183
> Project: Hive
> Issue Type: Task
>Reporter: Ted Yu
>
> In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
> HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
> Compilation against HBase 1.0.0 release would fail due to the missing
> dependency.
> The following change would allow compilation to pass:
> {code}
> diff --git
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
>
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> index 3218639..6f626f0 100644
> ---
> a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> +++
> b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
> @@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
> jobConf) {
> * only need TableMapReduceUtil.addDependencyJars(jobConf) here.
> */
>TableMapReduceUtil.addDependencyJars(
> - jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
> - org.cliffc.high_scale_lib.Counter.class); // this will be removed
> for HBase 1.0
> + jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
> + // org.cliffc.high_scale_lib.Counter.class); this will be removed
> for HBase 1.0
>Set merged = new
> LinkedHashSet(jobConf.getStringCollection("tmpjars"));
>Job
[jira] [Created] (HIVE-9183) Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter
Ted Yu created HIVE-9183:
Summary: Compilation against HBase 1.0.0 fails due to missing
dependency on high_scale_lib.Counter
Key: HIVE-9183
URL: https://issues.apache.org/jira/browse/HIVE-9183
Project: Hive
Issue Type: Task
Reporter: Ted Yu
In the HBase 1.0.0 release, org.cliffc.high_scale_lib is no longer used.
HBaseStorageHandler.java has compile time dependency on high_scale_lib.Counter
Compilation against HBase 1.0.0 release would fail due to the missing
dependency.
The following change would allow compilation to pass:
{code}
diff --git
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
index 3218639..6f626f0 100644
---
a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
+++
b/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
@@ -499,8 +499,8 @@ public void configureJobConf(TableDesc tableDesc, JobConf
jobConf) {
* only need TableMapReduceUtil.addDependencyJars(jobConf) here.
*/
TableMapReduceUtil.addDependencyJars(
- jobConf, HBaseStorageHandler.class, TableInputFormatBase.class,
- org.cliffc.high_scale_lib.Counter.class); // this will be removed
for HBase 1.0
+ jobConf, HBaseStorageHandler.class, TableInputFormatBase.class);
+ // org.cliffc.high_scale_lib.Counter.class); this will be removed
for HBase 1.0
Set merged = new
LinkedHashSet(jobConf.getStringCollection("tmpjars"));
Job copy = new Job(jobConf);
{code}
However, a shim layer may be needed so that 0.98 and 1.0 releases are both
supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9151) Checking s against null in TezJobMonitor#getNameWithProgress() should be done earlier
Ted Yu created HIVE-9151:
Summary: Checking s against null in
TezJobMonitor#getNameWithProgress() should be done earlier
Key: HIVE-9151
URL: https://issues.apache.org/jira/browse/HIVE-9151
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
int spaceRemaining = COLUMN_1_WIDTH - s.length() - 1;
String trimmedVName = s;
// if the vertex name is longer than column 1 width, trim it down
// "Tez Merge File Work" will become "Tez Merge File.."
if (s != null && s.length() > COLUMN_1_WIDTH) {
{code}
s is dereferenced first, rendering the null check ineffective.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9150) Unrelated types are compared in GenTezWork#getFollowingWorkIndex()
Ted Yu created HIVE-9150:
Summary: Unrelated types are compared in
GenTezWork#getFollowingWorkIndex()
Key: HIVE-9150
URL: https://issues.apache.org/jira/browse/HIVE-9150
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is the code:
{code}
if (tezWork.getEdgeProperty(unionWork,
baseWork).equals(TezEdgeProperty.EdgeType.CONTAINS)) {
{code}
getEdgeProperty() returns TezEdgeProperty which is compared with
TezEdgeProperty$EdgeType
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9096) GenericUDF may be left unclosed in PartitionPrune#visitCall()
[
https://issues.apache.org/jira/browse/HIVE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-9096:
-
Status: Patch Available (was: Open)
> GenericUDF may be left unclosed in PartitionPrune#visitCall()
> -
>
> Key: HIVE-9096
> URL: https://issues.apache.org/jira/browse/HIVE-9096
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-9096.patch
>
>
> In
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/PartitionPrune.java
> :
> {code}
> GenericUDF hiveUDF = SqlFunctionConverter.getHiveUDF(call.getOperator(),
> call.getType(), call.operands.size());
> if (hiveUDF != null &&
> !FunctionRegistry.isDeterministic(hiveUDF)) {
> return null;
> }
> {code}
> hiveUDF should be closed upon return.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9096) GenericUDF may be left unclosed in PartitionPrune#visitCall()
Ted Yu created HIVE-9096:
Summary: GenericUDF may be left unclosed in
PartitionPrune#visitCall()
Key: HIVE-9096
URL: https://issues.apache.org/jira/browse/HIVE-9096
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/PartitionPrune.java
:
{code}
GenericUDF hiveUDF = SqlFunctionConverter.getHiveUDF(call.getOperator(),
call.getType(), call.operands.size());
if (hiveUDF != null &&
!FunctionRegistry.isDeterministic(hiveUDF)) {
return null;
}
{code}
hiveUDF should be closed upon return.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9061) BigDecimal is constructed from double in TestFileDump#testDataDump()
Ted Yu created HIVE-9061:
Summary: BigDecimal is constructed from double in
TestFileDump#testDataDump()
Key: HIVE-9061
URL: https://issues.apache.org/jira/browse/HIVE-9061
Project: Hive
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
Here is one example:
{code}
HiveDecimal.create(new BigDecimal(4.)),
{code}
BigDecimal should be constructed from String: new BigDecimal("4.")
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9014) Upgrade hbase dependency to 0.98.8
[ https://issues.apache.org/jira/browse/HIVE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HIVE-9014: - Status: Patch Available (was: Open) > Upgrade hbase dependency to 0.98.8 > -- > > Key: HIVE-9014 > URL: https://issues.apache.org/jira/browse/HIVE-9014 > Project: Hive > Issue Type: Task > Reporter: Ted Yu > Assignee: Ted Yu >Priority: Minor > Attachments: HIVE-9014.patch > > > HBase 0.98.3 was an old release. > This JIRA upgrades the dependency to 0.98.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9014) Upgrade hbase dependency to 0.98.8
[ https://issues.apache.org/jira/browse/HIVE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HIVE-9014: - Attachment: HIVE-9014.patch > Upgrade hbase dependency to 0.98.8 > -- > > Key: HIVE-9014 > URL: https://issues.apache.org/jira/browse/HIVE-9014 > Project: Hive > Issue Type: Task > Reporter: Ted Yu > Assignee: Ted Yu >Priority: Minor > Attachments: HIVE-9014.patch > > > HBase 0.98.3 was an old release. > This JIRA upgrades the dependency to 0.98.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9014) Upgrade hbase dependency to 0.98.8
Ted Yu created HIVE-9014: Summary: Upgrade hbase dependency to 0.98.8 Key: HIVE-9014 URL: https://issues.apache.org/jira/browse/HIVE-9014 Project: Hive Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Minor HBase 0.98.3 was an old release. This JIRA upgrades the dependency to 0.98.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8927) Potential race condition on modification of dirtyAttributeInfoCache in MetricsMBeanImpl#getMBeanInfo()
Ted Yu created HIVE-8927:
Summary: Potential race condition on modification of
dirtyAttributeInfoCache in MetricsMBeanImpl#getMBeanInfo()
Key: HIVE-8927
URL: https://issues.apache.org/jira/browse/HIVE-8927
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
public MBeanInfo getMBeanInfo() {
if (dirtyAttributeInfoCache) {
synchronized(metricsMap) {
attributeInfos = new MBeanAttributeInfo[metricsMap.size()];
{code}
The check of dirtyAttributeInfoCache is outside the synchronized block.
There is possibility that different attributeInfos are returned due to race
condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8594) Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
[
https://issues.apache.org/jira/browse/HIVE-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-8594:
-
Attachment: HIVE-8594.txt
> Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
> ---
>
> Key: HIVE-8594
> URL: https://issues.apache.org/jira/browse/HIVE-8594
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
>Priority: Minor
> Attachments: HIVE-8594.txt, hive-8594.txt
>
>
> {code}
> if(whiteListParamsStr == null && whiteListParamsStr.trim().isEmpty()) {
> {code}
> If whiteListParamsStr is null, the call to trim() would result in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8806) Potential null dereference in Metrics#incrementCounter()
Ted Yu created HIVE-8806:
Summary: Potential null dereference in Metrics#incrementCounter()
Key: HIVE-8806
URL: https://issues.apache.org/jira/browse/HIVE-8806
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
if (!metrics.hasKey(name)) {
value = Long.valueOf(increment);
set(name, value);
} else {
value = ((Long)get(name)) + increment;
set(name, value);
}
{code}
In the else block, if get(name) returns null, unboxing null object would lead
to exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7150) FileInputStream is not closed in HiveConnection#getHttpClient()
[
https://issues.apache.org/jira/browse/HIVE-7150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7150:
-
Labels: jdbc (was: )
> FileInputStream is not closed in HiveConnection#getHttpClient()
> ---
>
> Key: HIVE-7150
> URL: https://issues.apache.org/jira/browse/HIVE-7150
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Labels: jdbc
> Fix For: 0.15.0
>
>
> Here is related code:
> {code}
> sslTrustStore.load(new FileInputStream(sslTrustStorePath),
> sslTrustStorePassword.toCharArray());
> {code}
> The FileInputStream is not closed upon returning from the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8729) VectorizedRowBatch#selected is not restored upon exception in VectorFilterOperator#processOp()
Ted Yu created HIVE-8729:
Summary: VectorizedRowBatch#selected is not restored upon
exception in VectorFilterOperator#processOp()
Key: HIVE-8729
URL: https://issues.apache.org/jira/browse/HIVE-8729
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
int [] selectedBackup = vrg.selected;
vrg.selected = temporarySelected;
...
if (vrg.size > 0) {
forward(vrg, null);
}
{code}
If forward() throws exception, selected is not restored.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-8594) Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
[
https://issues.apache.org/jira/browse/HIVE-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HIVE-8594:
Assignee: Ted Yu
> Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
> ---
>
> Key: HIVE-8594
> URL: https://issues.apache.org/jira/browse/HIVE-8594
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
>Priority: Minor
> Attachments: hive-8594.txt
>
>
> {code}
> if(whiteListParamsStr == null && whiteListParamsStr.trim().isEmpty()) {
> {code}
> If whiteListParamsStr is null, the call to trim() would result in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8594) Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
[
https://issues.apache.org/jira/browse/HIVE-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-8594:
-
Status: Patch Available (was: Open)
> Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
> ---
>
> Key: HIVE-8594
> URL: https://issues.apache.org/jira/browse/HIVE-8594
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
>Priority: Minor
> Attachments: hive-8594.txt
>
>
> {code}
> if(whiteListParamsStr == null && whiteListParamsStr.trim().isEmpty()) {
> {code}
> If whiteListParamsStr is null, the call to trim() would result in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8594) Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
[
https://issues.apache.org/jira/browse/HIVE-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-8594:
-
Attachment: hive-8594.txt
> Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
> ---
>
> Key: HIVE-8594
> URL: https://issues.apache.org/jira/browse/HIVE-8594
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
>Priority: Minor
> Attachments: hive-8594.txt
>
>
> {code}
> if(whiteListParamsStr == null && whiteListParamsStr.trim().isEmpty()) {
> {code}
> If whiteListParamsStr is null, the call to trim() would result in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8645) RecordReaderImpl#TimestampTreeReader#nextVector() should call TreeReader#nextVector()
Ted Yu created HIVE-8645:
Summary: RecordReaderImpl#TimestampTreeReader#nextVector() should
call TreeReader#nextVector()
Key: HIVE-8645
URL: https://issues.apache.org/jira/browse/HIVE-8645
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
Object nextVector(Object previousVector, long batchSize) throws IOException
{
LongColumnVector result = null;
{code}
Call to TreeReader.nextVector() is missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8594) Wrong condition in SettableConfigUpdater#setHiveConfWhiteList()
Ted Yu created HIVE-8594:
Summary: Wrong condition in
SettableConfigUpdater#setHiveConfWhiteList()
Key: HIVE-8594
URL: https://issues.apache.org/jira/browse/HIVE-8594
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
if(whiteListParamsStr == null && whiteListParamsStr.trim().isEmpty()) {
{code}
If whiteListParamsStr is null, the call to trim() would result in NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8593) Unintended regex is used in ScriptOperator#blackListed()
Ted Yu created HIVE-8593:
Summary: Unintended regex is used in ScriptOperator#blackListed()
Key: HIVE-8593
URL: https://issues.apache.org/jira/browse/HIVE-8593
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
for (String b : bls) {
b.replaceAll(".", "_");
{code}
The dot can match any character.
See
http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#replaceAll(java.lang.String,%20java.lang.String)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8523) Potential null dereference in DDLSemanticAnalyzer#addInputsOutputsAlterTable()
Ted Yu created HIVE-8523:
Summary: Potential null dereference in
DDLSemanticAnalyzer#addInputsOutputsAlterTable()
Key: HIVE-8523
URL: https://issues.apache.org/jira/browse/HIVE-8523
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
if (desc == null
||
!AlterTableDesc.doesAlterTableTypeSupportPartialPartitionSpec(desc.getOp())) {
throw new SemanticException(
ErrorMsg.ALTER_TABLE_TYPE_PARTIAL_PARTITION_SPEC_NO_SUPPORTED,
desc.getOp().name());
{code}
If desc is null, the construction of SemanticException would trigger NPE due to
the call to desc.getOp().
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8493) Access to HiveClientCache$CacheableHiveMetaStoreClient#isClosed should be synchronized
Ted Yu created HIVE-8493:
Summary: Access to
HiveClientCache$CacheableHiveMetaStoreClient#isClosed should be synchronized
Key: HIVE-8493
URL: https://issues.apache.org/jira/browse/HIVE-8493
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Update to isClosed is protected by synchronizing on
CacheableHiveMetaStoreClient.this
{code}
public boolean isClosed() {
return isClosed;
{code}
The above access should be synchronized as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8458) Potential null dereference in Utilities#clearWork()
Ted Yu created HIVE-8458:
Summary: Potential null dereference in Utilities#clearWork()
Key: HIVE-8458
URL: https://issues.apache.org/jira/browse/HIVE-8458
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
Path mapPath = getPlanPath(conf, MAP_PLAN_NAME);
Path reducePath = getPlanPath(conf, REDUCE_PLAN_NAME);
// if the plan path hasn't been initialized just return, nothing to clean.
if (mapPath == null && reducePath == null) {
return;
}
try {
FileSystem fs = mapPath.getFileSystem(conf);
{code}
If mapPath is null but reducePath is not null, getFileSystem() call would
produce NPE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8343) Return value from BlockingQueue.offer() is not checked in DynamicPartitionPruner
Ted Yu created HIVE-8343:
Summary: Return value from BlockingQueue.offer() is not checked in
DynamicPartitionPruner
Key: HIVE-8343
URL: https://issues.apache.org/jira/browse/HIVE-8343
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In addEvent() and processVertex(), there is call such as the following:
{code}
queue.offer(event);
{code}
The return value should be checked. If false is returned, event would not have
been queued.
Take a look at line 328 in:
http://fuseyism.com/classpath/doc/java/util/concurrent/LinkedBlockingQueue-source.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8342) Potential null dereference in ColumnTruncateMapper#jobClose()
Ted Yu created HIVE-8342:
Summary: Potential null dereference in
ColumnTruncateMapper#jobClose()
Key: HIVE-8342
URL: https://issues.apache.org/jira/browse/HIVE-8342
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
Utilities.mvFileToFinalPath(outputPath, job, success, LOG, dynPartCtx, null,
reporter);
{code}
Utilities.mvFileToFinalPath() calls createEmptyBuckets() where conf is
dereferenced:
{code}
boolean isCompressed = conf.getCompressed();
TableDesc tableInfo = conf.getTableInfo();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8282) Potential null deference in ConvertJoinMapJoin#convertJoinBucketMapJoin()
[
https://issues.apache.org/jira/browse/HIVE-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153232#comment-14153232
]
Ted Yu commented on HIVE-8282:
--
lgtm
nit: 'bucket map join' was mentioned in the log message @ line 321.
It should appear in the new message as well.
> Potential null deference in ConvertJoinMapJoin#convertJoinBucketMapJoin()
> -
>
> Key: HIVE-8282
> URL: https://issues.apache.org/jira/browse/HIVE-8282
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-8282.patch
>
>
> In convertJoinMapJoin():
> {code}
> for (Operator parentOp :
> joinOp.getParentOperators()) {
> if (parentOp instanceof MuxOperator) {
> return null;
> }
> }
> {code}
> NPE would result if convertJoinMapJoin() returns null:
> {code}
> MapJoinOperator mapJoinOp = convertJoinMapJoin(joinOp, context,
> bigTablePosition);
> MapJoinDesc joinDesc = mapJoinOp.getConf();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8285) Reference equality is used on boolean values in PartitionPruner#removeTruePredciates()
[
https://issues.apache.org/jira/browse/HIVE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153220#comment-14153220
]
Ted Yu commented on HIVE-8285:
--
+1
> Reference equality is used on boolean values in
> PartitionPruner#removeTruePredciates()
> --
>
> Key: HIVE-8285
> URL: https://issues.apache.org/jira/browse/HIVE-8285
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-8285.patch
>
>
> {code}
> if (e.getTypeInfo() == TypeInfoFactory.booleanTypeInfo
> && eC.getValue() == Boolean.TRUE) {
> {code}
> equals() should be used in the above comparison.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8285) Reference equality is used on boolean values in PartitionPruner#removeTruePredciates()
Ted Yu created HIVE-8285:
Summary: Reference equality is used on boolean values in
PartitionPruner#removeTruePredciates()
Key: HIVE-8285
URL: https://issues.apache.org/jira/browse/HIVE-8285
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
if (e.getTypeInfo() == TypeInfoFactory.booleanTypeInfo
&& eC.getValue() == Boolean.TRUE) {
{code}
equals() should be used in the above comparison.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8284) Equality comparison is done between two floating point variables in HiveRelMdUniqueKeys#getUniqueKeys()
Ted Yu created HIVE-8284:
Summary: Equality comparison is done between two floating point
variables in HiveRelMdUniqueKeys#getUniqueKeys()
Key: HIVE-8284
URL: https://issues.apache.org/jira/browse/HIVE-8284
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
double numRows = tScan.getRows();
...
double r = cStat.getRange().maxValue.doubleValue() -
cStat.getRange().minValue.doubleValue() + 1;
isKey = (numRows == r);
{code}
The equality check should use a small epsilon as tolerance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8283) Missing break in FilterSelectivityEstimator#visitCall()
Ted Yu created HIVE-8283:
Summary: Missing break in FilterSelectivityEstimator#visitCall()
Key: HIVE-8283
URL: https://issues.apache.org/jira/browse/HIVE-8283
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
{code}
case NOT_EQUALS: {
selectivity = computeNotEqualitySelectivity(call);
}
case LESS_THAN_OR_EQUAL:
case GREATER_THAN_OR_EQUAL:
case LESS_THAN:
case GREATER_THAN: {
selectivity = ((double) 1 / (double) 3);
break;
}
{code}
break is missing for NOT_EQUALS case. selectivity would be overwritten with 1/3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8282) Potential null deference in ConvertJoinMapJoin#convertJoinBucketMapJoin()
Ted Yu created HIVE-8282:
Summary: Potential null deference in
ConvertJoinMapJoin#convertJoinBucketMapJoin()
Key: HIVE-8282
URL: https://issues.apache.org/jira/browse/HIVE-8282
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In convertJoinMapJoin():
{code}
for (Operator parentOp :
joinOp.getParentOperators()) {
if (parentOp instanceof MuxOperator) {
return null;
}
}
{code}
NPE would result if convertJoinMapJoin() returns null:
{code}
MapJoinOperator mapJoinOp = convertJoinMapJoin(joinOp, context,
bigTablePosition);
MapJoinDesc joinDesc = mapJoinOp.getConf();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8177) Wrong parameter order in ExplainTask#getJSONLogicalPlan()
Ted Yu created HIVE-8177:
Summary: Wrong parameter order in ExplainTask#getJSONLogicalPlan()
Key: HIVE-8177
URL: https://issues.apache.org/jira/browse/HIVE-8177
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
JSONObject jsonPlan = outputMap(work.getParseContext().getTopOps(), true,
out, jsonOutput, work.getExtended(), 0);
{code}
The order of 4th and 5th parameters is reverted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8176) Close of FSDataOutputStream in OrcRecordUpdater ctor should be in finally clause
Ted Yu created HIVE-8176:
Summary: Close of FSDataOutputStream in OrcRecordUpdater ctor
should be in finally clause
Key: HIVE-8176
URL: https://issues.apache.org/jira/browse/HIVE-8176
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
try {
FSDataOutputStream strm = fs.create(new Path(path, ACID_FORMAT), false);
strm.writeInt(ORC_ACID_VERSION);
strm.close();
} catch (IOException ioe) {
{code}
If strm.writeInt() throws IOE, strm would be left unclosed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [ANNOUNCE] New Hive Committer - Eugene Koifman
Congratulations, Eugene.
[jira] [Created] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
Ted Yu created HIVE-8090:
Summary: Potential null pointer reference in
WriterImpl#StreamFactory#createStream()
Key: HIVE-8090
URL: https://issues.apache.org/jira/browse/HIVE-8090
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
{code}
switch (kind) {
...
default:
modifiers = null;
break;
}
BufferedStream result = streams.get(name);
if (result == null) {
result = new BufferedStream(name.toString(), bufferSize,
codec == null ? codec : codec.modify(modifiers));
{code}
In case modifiers is null and codec is ZlibCodec, there would be NPE in
ZlibCodec#modify(EnumSet modifiers) :
{code}
for (Modifier m : modifiers) {
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-7997) Potential null pointer reference in ObjectInspectorUtils#compareTypes()
Ted Yu created HIVE-7997:
Summary: Potential null pointer reference in
ObjectInspectorUtils#compareTypes()
Key: HIVE-7997
URL: https://issues.apache.org/jira/browse/HIVE-7997
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
{code}
if (childFieldsList1 == null && childFieldsList2 == null) {
return true;
}
if (childFieldsList1.size() != childFieldsList2.size()) {
return false;
}
{code}
If either childFieldsList1 or childFieldsList2 is null but not both, the second
if statement would produce NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-7996) Potential resource leak in HiveBurnInClient
Ted Yu created HIVE-7996: Summary: Potential resource leak in HiveBurnInClient Key: HIVE-7996 URL: https://issues.apache.org/jira/browse/HIVE-7996 Project: Hive Issue Type: Bug Reporter: Ted Yu Priority: Minor In createTables() and runQueries(), Statement stmt is not closed upon return. In main(), Connection con is not closed upon exit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7863) Potential null reference in TxnDbUtil#prepareDb()
[
https://issues.apache.org/jira/browse/HIVE-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107773#comment-14107773
]
Ted Yu commented on HIVE-7863:
--
Same pattern appears in cleanDb() as well.
> Potential null reference in TxnDbUtil#prepareDb()
> -
>
> Key: HIVE-7863
> URL: https://issues.apache.org/jira/browse/HIVE-7863
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> Here is related code:
> {code}
> Connection conn = null;
> boolean committed = false;
> try {
> conn = getConnection();
> ...
> } finally {
> if (!committed) conn.rollback();
> conn.close();
> }
> {code}
> If getConnection() throws exception, conn.rollback() would be called -
> leading to NPE.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7863) Potential null reference in TxnDbUtil#prepareDb()
Ted Yu created HIVE-7863:
Summary: Potential null reference in TxnDbUtil#prepareDb()
Key: HIVE-7863
URL: https://issues.apache.org/jira/browse/HIVE-7863
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
Connection conn = null;
boolean committed = false;
try {
conn = getConnection();
...
} finally {
if (!committed) conn.rollback();
conn.close();
}
{code}
If getConnection() throws exception, conn.rollback() would be called - leading
to NPE.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7862) close of InputStream in Utils#copyToZipStream() should be placed in finally block
Ted Yu created HIVE-7862:
Summary: close of InputStream in Utils#copyToZipStream() should be
placed in finally block
Key: HIVE-7862
URL: https://issues.apache.org/jira/browse/HIVE-7862
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/Utils.java , line
278 :
{code}
private static void copyToZipStream(InputStream is, ZipEntry entry,
ZipOutputStream zos)
throws IOException {
zos.putNextEntry(entry);
byte[] arr = new byte[4096];
int read = is.read(arr);
while (read > -1) {
zos.write(arr, 0, read);
read = is.read(arr);
}
is.close();
{code}
If read() throws IOException, is would be left unclosed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7754) Potential null pointer dereference in ColumnTruncateMapper#jobClose()
Ted Yu created HIVE-7754:
Summary: Potential null pointer dereference in
ColumnTruncateMapper#jobClose()
Key: HIVE-7754
URL: https://issues.apache.org/jira/browse/HIVE-7754
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
Utilities.mvFileToFinalPath(outputPath, job, success, LOG, dynPartCtx, null,
reporter);
{code}
null is passed to Utilities.mvFileToFinalPath() which gets passed to
createEmptyBuckets() where:
{code}
boolean isCompressed = conf.getCompressed();
TableDesc tableInfo = conf.getTableInfo();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7753) Same operand appears on both sides of > in DataType#compareByteArray()
[
https://issues.apache.org/jira/browse/HIVE-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7753:
-
Status: Patch Available (was: Open)
> Same operand appears on both sides of > in DataType#compareByteArray()
> --
>
> Key: HIVE-7753
> URL: https://issues.apache.org/jira/browse/HIVE-7753
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: hive-7753-v1.txt
>
>
> Around line 227:
> {code}
> if (o1[i] > o1[i]) {
> return 1;
> {code}
> The above comparison would never be true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7753) Same operand appears on both sides of > in DataType#compareByteArray()
[
https://issues.apache.org/jira/browse/HIVE-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7753:
-
Attachment: hive-7753-v1.txt
> Same operand appears on both sides of > in DataType#compareByteArray()
> --
>
> Key: HIVE-7753
> URL: https://issues.apache.org/jira/browse/HIVE-7753
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: hive-7753-v1.txt
>
>
> Around line 227:
> {code}
> if (o1[i] > o1[i]) {
> return 1;
> {code}
> The above comparison would never be true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7753) Same operand appears on both sides of > in DataType#compareByteArray()
Ted Yu created HIVE-7753:
Summary: Same operand appears on both sides of > in
DataType#compareByteArray()
Key: HIVE-7753
URL: https://issues.apache.org/jira/browse/HIVE-7753
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Around line 227:
{code}
if (o1[i] > o1[i]) {
return 1;
{code}
The above comparison would never be true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7752) isClosed is accessed without holding lock in HiveClientCache#CacheableHiveMetaStoreClient#isClosed()
Ted Yu created HIVE-7752:
Summary: isClosed is accessed without holding lock in
HiveClientCache#CacheableHiveMetaStoreClient#isClosed()
Key: HIVE-7752
URL: https://issues.apache.org/jira/browse/HIVE-7752
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Assignment to isClosed is protected by holding lock
"CacheableHiveMetaStoreClient.this"
{code}
public boolean isClosed() {
return isClosed;
{code}
The above method should carry synchronized keyword.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7672) Potential resource leak in EximUtil#createExportDump()
Ted Yu created HIVE-7672:
Summary: Potential resource leak in EximUtil#createExportDump()
Key: HIVE-7672
URL: https://issues.apache.org/jira/browse/HIVE-7672
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
OutputStream out = fs.create(metadataPath);
out.write(jsonContainer.toString().getBytes("UTF-8"));
out.close();
{code}
If out.write() throws exception, out would be left unclosed.
out.close() should be enclosed in finally block.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7600) ConstantPropagateProcFactory uses reference equality on Boolean
Ted Yu created HIVE-7600:
Summary: ConstantPropagateProcFactory uses reference equality on
Boolean
Key: HIVE-7600
URL: https://issues.apache.org/jira/browse/HIVE-7600
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
shortcutFunction() has the following code:
{code}
if (c.getValue() == Boolean.FALSE) {
{code}
Boolean.FALSE.equals() should be used.
There're a few other occurrences of using reference equality on Boolean in this
class.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7599) NPE in MergeTask#main() when -format is absent
Ted Yu created HIVE-7599:
Summary: NPE in MergeTask#main() when -format is absent
Key: HIVE-7599
URL: https://issues.apache.org/jira/browse/HIVE-7599
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
When '-format' is absent from commandline, the following call would result in
NPE (format is initialized to null):
{code}
if (format.equals("rcfile")) {
mergeWork = new MergeWork(inputPaths, new Path(outputDir),
RCFileInputFormat.class);
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7598) Potential null pointer dereference in MergeTask#closeJob()
Ted Yu created HIVE-7598:
Summary: Potential null pointer dereference in MergeTask#closeJob()
Key: HIVE-7598
URL: https://issues.apache.org/jira/browse/HIVE-7598
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Call to Utilities.mvFileToFinalPath() passes null as second last parameter,
conf.
null gets passed to createEmptyBuckets() which dereferences conf directly:
{code}
boolean isCompressed = conf.getCompressed();
TableDesc tableInfo = conf.getTableInfo();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7521) Reference equality is used on Boolean in NullScanOptimizer#WhereFalseProcessor#process()
Ted Yu created HIVE-7521:
Summary: Reference equality is used on Boolean in
NullScanOptimizer#WhereFalseProcessor#process()
Key: HIVE-7521
URL: https://issues.apache.org/jira/browse/HIVE-7521
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
ExprNodeConstantDesc c = (ExprNodeConstantDesc) condition;
if (c.getValue() != Boolean.FALSE) {
return null;
}
{code}
equals() should be called instead.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7452) Boolean comparison is done through reference equality rather than using equals
Ted Yu created HIVE-7452:
Summary: Boolean comparison is done through reference equality
rather than using equals
Key: HIVE-7452
URL: https://issues.apache.org/jira/browse/HIVE-7452
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In Driver#doAuthorization():
{code}
if (tbl != null && !tableAuthChecked.contains(tbl.getTableName()) &&
!(tableUsePartLevelAuth.get(tbl.getTableName()) == Boolean.TRUE)) {
{code}
The above comparison should be done using .equals() method.
The comparison below doesn't evaluate to true:
{code}
Boolean b = new Boolean(true);
if (b == Boolean.TRUE) {
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7419) Missing break in SemanticAnalyzer#getTableDescFromSerDe()
[
https://issues.apache.org/jira/browse/HIVE-7419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066003#comment-14066003
]
Ted Yu commented on HIVE-7419:
--
Thanks for taking care of this.
> Missing break in SemanticAnalyzer#getTableDescFromSerDe()
> -
>
> Key: HIVE-7419
> URL: https://issues.apache.org/jira/browse/HIVE-7419
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-7419.1.patch.txt
>
>
> {code}
> case HiveParser.TOK_TABLEROWFORMATLINES:
> String lineDelim =
> unescapeSQLString(rowChild.getChild(0).getText());
> tblDesc.getProperties().setProperty(serdeConstants.LINE_DELIM,
> lineDelim);
> if (!lineDelim.equals("\n") && !lineDelim.equals("10")) {
> throw new SemanticException(generateErrorMessage(rowChild,
> ErrorMsg.LINES_TERMINATED_BY_NON_NEWLINE.getMsg()));
> }
> case HiveParser.TOK_TABLEROWFORMATNULL:
> String nullFormat =
> unescapeSQLString(rowChild.getChild(0).getText());
> {code}
> break seems to be missing for TOK_TABLEROWFORMATLINES case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-7418) Missing null check in HiveSQLException#toStackTrace()
[
https://issues.apache.org/jira/browse/HIVE-7418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-7418.
--
Resolution: Not a Problem
> Missing null check in HiveSQLException#toStackTrace()
> -
>
> Key: HIVE-7418
> URL: https://issues.apache.org/jira/browse/HIVE-7418
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> {code}
> public static Throwable toCause(List details) {
> return toStackTrace(details, null, 0);
> {code}
> null is passed to toStackTrace(as parent) where:
> {code}
> if (common > 0) {
> System.arraycopy(parent, parent.length - common, trace, trace.length -
> common, common);
> }
> {code}
> parent.length may throw NPE.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7419) Missing break in SemanticAnalyzer#getTableDescFromSerDe()
Ted Yu created HIVE-7419:
Summary: Missing break in SemanticAnalyzer#getTableDescFromSerDe()
Key: HIVE-7419
URL: https://issues.apache.org/jira/browse/HIVE-7419
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
case HiveParser.TOK_TABLEROWFORMATLINES:
String lineDelim = unescapeSQLString(rowChild.getChild(0).getText());
tblDesc.getProperties().setProperty(serdeConstants.LINE_DELIM,
lineDelim);
if (!lineDelim.equals("\n") && !lineDelim.equals("10")) {
throw new SemanticException(generateErrorMessage(rowChild,
ErrorMsg.LINES_TERMINATED_BY_NON_NEWLINE.getMsg()));
}
case HiveParser.TOK_TABLEROWFORMATNULL:
String nullFormat = unescapeSQLString(rowChild.getChild(0).getText());
{code}
break seems to be missing for TOK_TABLEROWFORMATLINES case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7418) Missing null check in HiveSQLException#toStackTrace()
Ted Yu created HIVE-7418:
Summary: Missing null check in HiveSQLException#toStackTrace()
Key: HIVE-7418
URL: https://issues.apache.org/jira/browse/HIVE-7418
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
public static Throwable toCause(List details) {
return toStackTrace(details, null, 0);
{code}
null is passed to toStackTrace(as parent) where:
{code}
if (common > 0) {
System.arraycopy(parent, parent.length - common, trace, trace.length -
common, common);
}
{code}
parent.length may throw NPE.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-7307) Lack of synchronization for TxnHandler#getDbConn()
[
https://issues.apache.org/jira/browse/HIVE-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HIVE-7307.
--
Resolution: Later
> Lack of synchronization for TxnHandler#getDbConn()
> --
>
> Key: HIVE-7307
> URL: https://issues.apache.org/jira/browse/HIVE-7307
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> TxnHandler#getDbConn() accesses connPool without holding lock on
> TxnHandler.class
> {code}
> Connection dbConn = connPool.getConnection();
> dbConn.setAutoCommit(false);
> {code}
> null check should be performed on the return value, dbConn.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7307) Lack of synchronization for TxnHandler#getDbConn()
[
https://issues.apache.org/jira/browse/HIVE-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049106#comment-14049106
]
Ted Yu commented on HIVE-7307:
--
That was the initial thought.
Thread safety would be handled by BoneCP.
> Lack of synchronization for TxnHandler#getDbConn()
> --
>
> Key: HIVE-7307
> URL: https://issues.apache.org/jira/browse/HIVE-7307
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> TxnHandler#getDbConn() accesses connPool without holding lock on
> TxnHandler.class
> {code}
> Connection dbConn = connPool.getConnection();
> dbConn.setAutoCommit(false);
> {code}
> null check should be performed on the return value, dbConn.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7307) Lack of synchronization for TxnHandler#getDbConn()
[
https://issues.apache.org/jira/browse/HIVE-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7307:
-
Description:
TxnHandler#getDbConn() accesses connPool without holding lock on
TxnHandler.class
{code}
Connection dbConn = connPool.getConnection();
dbConn.setAutoCommit(false);
{code}
null check should be performed on the return value, dbConn.
was:TxnHandler#getDbConn() accesses connPool without holding lock on
TxnHandler.class
> Lack of synchronization for TxnHandler#getDbConn()
> --
>
> Key: HIVE-7307
> URL: https://issues.apache.org/jira/browse/HIVE-7307
> Project: Hive
> Issue Type: Bug
> Reporter: Ted Yu
>Priority: Minor
>
> TxnHandler#getDbConn() accesses connPool without holding lock on
> TxnHandler.class
> {code}
> Connection dbConn = connPool.getConnection();
> dbConn.setAutoCommit(false);
> {code}
> null check should be performed on the return value, dbConn.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7307) Lack of synchronization for TxnHandler#getDbConn()
Ted Yu created HIVE-7307: Summary: Lack of synchronization for TxnHandler#getDbConn() Key: HIVE-7307 URL: https://issues.apache.org/jira/browse/HIVE-7307 Project: Hive Issue Type: Bug Reporter: Ted Yu Priority: Minor TxnHandler#getDbConn() accesses connPool without holding lock on TxnHandler.class -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7306) Ineffective null check in GenericUDAFAverage#GenericUDAFAverageEvaluatorDouble#getNextResult()
[
https://issues.apache.org/jira/browse/HIVE-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046215#comment-14046215
]
Ted Yu commented on HIVE-7306:
--
Similar problem exists in GenericUDAFAverage#GenericUDAFAverageEvaluatorDecimal
as well.
> Ineffective null check in
> GenericUDAFAverage#GenericUDAFAverageEvaluatorDouble#getNextResult()
> --
>
> Key: HIVE-7306
> URL: https://issues.apache.org/jira/browse/HIVE-7306
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> Object[] o = ss.intermediateVals.remove(0);
> Double d = o == null ? 0.0 : (Double) o[0];
> r = r == null ? null : r - d;
> cnt = cnt - ((Long) o[1]);
> {code}
> Array o is accessed without null check in the last line above.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7306) Ineffective null check in GenericUDAFAverage#GenericUDAFAverageEvaluatorDouble#getNextResult()
Ted Yu created HIVE-7306:
Summary: Ineffective null check in
GenericUDAFAverage#GenericUDAFAverageEvaluatorDouble#getNextResult()
Key: HIVE-7306
URL: https://issues.apache.org/jira/browse/HIVE-7306
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
Object[] o = ss.intermediateVals.remove(0);
Double d = o == null ? 0.0 : (Double) o[0];
r = r == null ? null : r - d;
cnt = cnt - ((Long) o[1]);
{code}
Array o is accessed without null check in the last line above.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7305) Return value from in.read() is ignored in SerializationUtils#readLongLE()
Ted Yu created HIVE-7305:
Summary: Return value from in.read() is ignored in
SerializationUtils#readLongLE()
Key: HIVE-7305
URL: https://issues.apache.org/jira/browse/HIVE-7305
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
long readLongLE(InputStream in) throws IOException {
in.read(readBuffer, 0, 8);
return (((readBuffer[0] & 0xff) << 0)
+ ((readBuffer[1] & 0xff) << 8)
{code}
Return value from read() may indicate fewer than 8 bytes read.
The return value should be checked.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7229) String is compared using equal in HiveMetaStore#HMSHandler#init()
[
https://issues.apache.org/jira/browse/HIVE-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042190#comment-14042190
]
Ted Yu commented on HIVE-7229:
--
Looks like QA bot had some issue.
> String is compared using equal in HiveMetaStore#HMSHandler#init()
> -
>
> Key: HIVE-7229
> URL: https://issues.apache.org/jira/browse/HIVE-7229
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
> Attachments: HIVE-7229.1.patch, HIVE-7229.patch
>
>
> Around line 423:
> {code}
> if (partitionValidationRegex != null && partitionValidationRegex != "")
> {
> partitionValidationPattern =
> Pattern.compile(partitionValidationRegex);
> {code}
> partitionValidationRegex.isEmpty() can be used instead.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7225) Unclosed Statement's in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7225:
-
Attachment: hive-7225.3.patch
Patch v3 addresses Alan's comments above.
> Unclosed Statement's in TxnHandler
> --
>
> Key: HIVE-7225
> URL: https://issues.apache.org/jira/browse/HIVE-7225
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: steve, Oh
> Attachments: HIVE-7225.1.patch, hive-7225.3.patch
>
>
> There're several methods in TxnHandler where Statement (local to the method)
> is not closed upon return.
> Here're a few examples:
> In compact():
> {code}
> stmt.executeUpdate(s);
> LOG.debug("Going to commit");
> dbConn.commit();
> {code}
> In showCompact():
> {code}
> Statement stmt = dbConn.createStatement();
> String s = "select cq_database, cq_table, cq_partition, cq_state,
> cq_type, cq_worker_id, " +
> "cq_start, cq_run_as from COMPACTION_QUEUE";
> LOG.debug("Going to execute query <" + s + ">");
> ResultSet rs = stmt.executeQuery(s);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7225) Unclosed Statement's in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HIVE-7225:
-
Status: Patch Available (was: Open)
> Unclosed Statement's in TxnHandler
> --
>
> Key: HIVE-7225
> URL: https://issues.apache.org/jira/browse/HIVE-7225
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: steve, Oh
> Attachments: HIVE-7225.1.patch, hive-7225.3.patch
>
>
> There're several methods in TxnHandler where Statement (local to the method)
> is not closed upon return.
> Here're a few examples:
> In compact():
> {code}
> stmt.executeUpdate(s);
> LOG.debug("Going to commit");
> dbConn.commit();
> {code}
> In showCompact():
> {code}
> Statement stmt = dbConn.createStatement();
> String s = "select cq_database, cq_table, cq_partition, cq_state,
> cq_type, cq_worker_id, " +
> "cq_start, cq_run_as from COMPACTION_QUEUE";
> LOG.debug("Going to execute query <" + s + ">");
> ResultSet rs = stmt.executeQuery(s);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7225) Unclosed Statement's in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036388#comment-14036388
]
Ted Yu commented on HIVE-7225:
--
bq. whether the default is to close the open statements
As far as I can tell from the source code, default value for
closeOpenStatements is false. This is understandable: tracking open statements
would consume more resources. For well designed / implemented client, this
shouldn't be necessary.
bq. make sure we close the statements
Do you have time to take a look at Steve's patch ?
Thanks
> Unclosed Statement's in TxnHandler
> --
>
> Key: HIVE-7225
> URL: https://issues.apache.org/jira/browse/HIVE-7225
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: steve, Oh
> Attachments: HIVE-7225.1.patch
>
>
> There're several methods in TxnHandler where Statement (local to the method)
> is not closed upon return.
> Here're a few examples:
> In compact():
> {code}
> stmt.executeUpdate(s);
> LOG.debug("Going to commit");
> dbConn.commit();
> {code}
> In showCompact():
> {code}
> Statement stmt = dbConn.createStatement();
> String s = "select cq_database, cq_table, cq_partition, cq_state,
> cq_type, cq_worker_id, " +
> "cq_start, cq_run_as from COMPACTION_QUEUE";
> LOG.debug("Going to execute query <" + s + ">");
> ResultSet rs = stmt.executeQuery(s);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7225) Unclosed Statement's in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034400#comment-14034400
]
Ted Yu commented on HIVE-7225:
--
Hive depends on BoneCP 0.8.0
I cloned BoneCP locally and looked at their code.
{code}
public class ConnectionHandle implements Connection,Serializable{
{code}
In ConnectionHandle ctor, I see:
{code}
this.detectUnclosedStatements =
pool.getConfig().isDetectUnclosedStatements();
this.closeOpenStatements = pool.getConfig().isCloseOpenStatements();
{code}
In close() method:
{code}
if (this.closeOpenStatements){
for (Entry statementEntry:
this.trackedStatement.entrySet()){
statementEntry.getKey().close();
if (this.detectUnclosedStatements){
logger.warn(String.format(UNCLOSED_LOG_ERROR_MESSAGE,
statementEntry.getValue()));
}
}
this.trackedStatement.clear();
}
{code}
In Hive codebase, I don't see call that turns on closeOpenStatements.
> Unclosed Statement's in TxnHandler
> --
>
> Key: HIVE-7225
> URL: https://issues.apache.org/jira/browse/HIVE-7225
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: steve, Oh
> Attachments: HIVE-7225.1.patch
>
>
> There're several methods in TxnHandler where Statement (local to the method)
> is not closed upon return.
> Here're a few examples:
> In compact():
> {code}
> stmt.executeUpdate(s);
> LOG.debug("Going to commit");
> dbConn.commit();
> {code}
> In showCompact():
> {code}
> Statement stmt = dbConn.createStatement();
> String s = "select cq_database, cq_table, cq_partition, cq_state,
> cq_type, cq_worker_id, " +
> "cq_start, cq_run_as from COMPACTION_QUEUE";
> LOG.debug("Going to execute query <" + s + ">");
> ResultSet rs = stmt.executeQuery(s);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7225) Unclosed Statement's in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034148#comment-14034148
]
Ted Yu commented on HIVE-7225:
--
Does this answer your question ?
http://stackoverflow.com/questions/2708689/impact-of-java-sql-connection-close-on-java-sql-statement-objects-and-the-like
> Unclosed Statement's in TxnHandler
> --
>
> Key: HIVE-7225
> URL: https://issues.apache.org/jira/browse/HIVE-7225
> Project: Hive
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: steve, Oh
> Attachments: HIVE-7225.1.patch
>
>
> There're several methods in TxnHandler where Statement (local to the method)
> is not closed upon return.
> Here're a few examples:
> In compact():
> {code}
> stmt.executeUpdate(s);
> LOG.debug("Going to commit");
> dbConn.commit();
> {code}
> In showCompact():
> {code}
> Statement stmt = dbConn.createStatement();
> String s = "select cq_database, cq_table, cq_partition, cq_state,
> cq_type, cq_worker_id, " +
> "cq_start, cq_run_as from COMPACTION_QUEUE";
> LOG.debug("Going to execute query <" + s + ">");
> ResultSet rs = stmt.executeQuery(s);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7229) String is compared using equal in HiveMetaStore#HMSHandler#init()
Ted Yu created HIVE-7229:
Summary: String is compared using equal in
HiveMetaStore#HMSHandler#init()
Key: HIVE-7229
URL: https://issues.apache.org/jira/browse/HIVE-7229
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Around line 423:
{code}
if (partitionValidationRegex != null && partitionValidationRegex != "") {
partitionValidationPattern = Pattern.compile(partitionValidationRegex);
{code}
partitionValidationRegex.isEmpty() can be used instead.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7225) Unclosed Statement's in TxnHandler
Ted Yu created HIVE-7225:
Summary: Unclosed Statement's in TxnHandler
Key: HIVE-7225
URL: https://issues.apache.org/jira/browse/HIVE-7225
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
There're several methods in TxnHandler where Statement (local to the method) is
not closed upon return.
Here're a few examples:
In compact():
{code}
stmt.executeUpdate(s);
LOG.debug("Going to commit");
dbConn.commit();
{code}
In showCompact():
{code}
Statement stmt = dbConn.createStatement();
String s = "select cq_database, cq_table, cq_partition, cq_state,
cq_type, cq_worker_id, " +
"cq_start, cq_run_as from COMPACTION_QUEUE";
LOG.debug("Going to execute query <" + s + ">");
ResultSet rs = stmt.executeQuery(s);
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7185) KeyWrapperFactory#TextKeyWrapper#equals() extracts Text incorrectly when isCopy is false
Ted Yu created HIVE-7185:
Summary: KeyWrapperFactory#TextKeyWrapper#equals() extracts Text
incorrectly when isCopy is false
Key: HIVE-7185
URL: https://issues.apache.org/jira/browse/HIVE-7185
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
} else {
t1 = soi_new.getPrimitiveWritableObject(key);
t2 = soi_copy.getPrimitiveWritableObject(obj);
{code}
t2 should be assigned soi_new.getPrimitiveWritableObject(obj)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7183) Size of partColumnGrants should be checked in ObjectStore#removeRole()
Ted Yu created HIVE-7183:
Summary: Size of partColumnGrants should be checked in
ObjectStore#removeRole()
Key: HIVE-7183
URL: https://issues.apache.org/jira/browse/HIVE-7183
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
List partColumnGrants =
listPrincipalAllPartitionColumnGrants(
mRol.getRoleName(), PrincipalType.ROLE);
if (tblColumnGrants.size() > 0) {
pm.deletePersistentAll(partColumnGrants);
{code}
Size of tblColumnGrants is currently checked.
Size of partColumnGrants should be checked instead.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7182) ResultSet is not closed in JDBCStatsPublisher#init()
Ted Yu created HIVE-7182:
Summary: ResultSet is not closed in JDBCStatsPublisher#init()
Key: HIVE-7182
URL: https://issues.apache.org/jira/browse/HIVE-7182
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
{code}
ResultSet rs = dbm.getTables(null, null,
JDBCStatsUtils.getStatTableName(), null);
boolean tblExists = rs.next();
{code}
rs is not closed upon return from init()
If stmt.executeUpdate() throws exception, stmt.close() would be skipped - the
close() call should be placed in finally block.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7180) BufferedReader is not closed in MetaStoreSchemaInfo ctor
Ted Yu created HIVE-7180:
Summary: BufferedReader is not closed in MetaStoreSchemaInfo ctor
Key: HIVE-7180
URL: https://issues.apache.org/jira/browse/HIVE-7180
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Here is related code:
{code}
BufferedReader bfReader =
new BufferedReader(new FileReader(upgradeListFile));
String currSchemaVersion;
while ((currSchemaVersion = bfReader.readLine()) != null) {
upgradeOrderList.add(currSchemaVersion.trim());
{code}
BufferedReader / FileReader should be closed upon return from ctor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
