[jira] [Commented] (HIVE-6593) Create a maven assembly for hive-jdbc

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924769#comment-13924769
 ] 

Konstantin Boudnik commented on HIVE-6593:
--

Actually, we don't ask to create a JDBC tarball. What we need for successful 
integration of hive-jdbc is a build produce an assembly with the things that 
should be given to a JDBC user. Hopefully, that outlines the difference.

> Create a maven assembly for hive-jdbc
> -
>
> Key: HIVE-6593
> URL: https://issues.apache.org/jira/browse/HIVE-6593
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Affects Versions: 0.12.0
>Reporter: Mark Grover
>Assignee: Szehon Ho
>
> Currently in Apache Bigtop we bundle and distribute Hive. In particular, for 
> users to not have to install the entirety of Hive on machines that are just 
> jdbc clients, we have a special package which is a subset of hive, called 
> hive-jdbc that bundles only the jdbc driver jar and it's dependencies.
> However, because Hive doesn't have an assembly for the jdbc jar, we have to 
> hack and hardcode the list of jdbc jars and it's dependencies:
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hive/SPECS/hive.spec#L361
> As Hive moves to Maven, it would be pretty fantastic if Hive could leverage 
> the maven-assembly-plugin and generate a .tar.gz assembly for what's required 
> for jdbc gateway machines. That we can simply take that distribution and 
> build a jdbc package from it without having to hard code jar names and 
> dependencies. That would make the process much less error prone.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13782632#comment-13782632
 ] 

Konstantin Boudnik commented on HIVE-4501:
--

bq. This is not the case, same thread can get used for another session
Are you saying that the same FS ref. can be used across UGIs? If so, It looks 
like a nice big hole to me ;) The call for the close of FS is done at the point 
where the particular UGI's session is getting finished and all its resources 
are let go, e.g. operation handlers, metastore, etc. I am not sure if I follow 
your comment. Could you please elaborate?

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch, 
> HIVE-4501.trunk.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-3694) Generate test jars and publish them to Maven

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-3694:
-

Affects Version/s: 0.9.0

> Generate test jars and publish them to Maven
> 
>
> Key: HIVE-3694
> URL: https://issues.apache.org/jira/browse/HIVE-3694
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
>Priority: Minor
> Attachments: D6843.1.patch, D6843.2.patch, D6843.3.patch, 
> D6843.4.patch
>
>
> It should be possible to generate Hive test jars and publish them to Maven so 
> that other projects that rely on Hive or extend it could reuse its test 
> library.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-3930) Generate and publish source jars

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781187#comment-13781187
 ] 

Konstantin Boudnik commented on HIVE-3930:
--

Any chance to have this fix on 0.12 (or trunk?)

> Generate and publish source jars
> 
>
> Key: HIVE-3930
> URL: https://issues.apache.org/jira/browse/HIVE-3930
> Project: Hive
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>
> Hive should generate and publish source jars to Maven.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Status: Patch Available  (was: Open)

Here's the trunk patch. The original one wasn't applicable for the trunk 
version and should have been clearly marked as such. Apologies.

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch, 
> HIVE-4501.trunk.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Status: Open  (was: Patch Available)

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch, 
> HIVE-4501.trunk.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Attachment: HIVE-4501.trunk.patch

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch, 
> HIVE-4501.trunk.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Status: Patch Available  (was: Open)

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Status: Open  (was: Patch Available)

Wrong patch structure - needs to be regenerated with p0

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Attachment: HIVE-4501.1.patch

Regenerating patch with -p0

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780531#comment-13780531
 ] 

Konstantin Boudnik commented on HIVE-4501:
--

bq. I believe the lease checker threads also go away when GC kicks in

We haven't observed that behavior in our tests, not with explicit GC at least.

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780343#comment-13780343
 ] 

Konstantin Boudnik commented on HIVE-4501:
--

So, shall we have committed it to the trunk and 0.12?

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-4501:
-

Attachment: HIVE-4501.1.patch

Posting the patch for his fix on JIRA.

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch, HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778256#comment-13778256
 ] 

Konstantin Boudnik commented on HIVE-4501:
--

But it essentially prevents auth mechanism for HS2, right?

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration. 
> As a workaround disable impersonation by setting hive.server2.enable.doAs to 
> false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777816#comment-13777816
 ] 

Konstantin Boudnik commented on HIVE-4501:
--

Provided patch doesn't solve the problem though, to my understanding. It makes 
it less pronounced for sure, but it doesn't go away.

> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> 
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.11.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in 
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and 
> fs.file.impl.disable.cache to false.
> Users should not have to bother with this extra configuration. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3694) Generate test jars and publish them to Maven

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775779#comment-13775779
 ] 

Konstantin Boudnik commented on HIVE-3694:
--

It seems that this ticket has been committed (with a different synopsis) back 
in Dec/2012 (SHA1 614c640b2eecdd2d8dcea67af7a0a57300597aea)

> Generate test jars and publish them to Maven
> 
>
> Key: HIVE-3694
> URL: https://issues.apache.org/jira/browse/HIVE-3694
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Mikhail Bautin
>Priority: Minor
> Attachments: D6843.1.patch, D6843.2.patch, D6843.3.patch, 
> D6843.4.patch
>
>
> It should be possible to generate Hive test jars and publish them to Maven so 
> that other projects that rely on Hive or extend it could reuse its test 
> library.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Interesting claims that seem untrue

[Konstantin Boudnik]

Carter,

what you are doing is essentially contradict ASF policy of "community over
code".

Perhaps, your intentions are good. However, LOC calculations or other silly
contests are essentially driving a wedge between developers who happen to draw
their paycheck from different commercial entities. Hadoop community passed
through this already and it caused nothing but despair and bitterness between
the people.

Unlike some other popular contests, the number of lines contributed doesn't
matter for most. Seriously.

Regards,
  Cos

On Mon, Sep 16, 2013 at 01:58PM, Carter Shanklin wrote:
> Ed,
> 
> If nothing else I'm glad it was interesting enough to generate some
> discussion. These sorts of stats are always subjects of a lot of
> controversy. I have seen a lot of these sorts of charts float around in
> confidential slide decks and I think it's good to have them out in the open
> where anyone can critique and correct them.
> 
> In this case Ed, you've pointed out a legitimate flaw in my analysis. Doing
> the analysis again I found that previously, due to a bug in my scripts,
> JIRAs that didn't have Hudson comments in them were not counted (this was
> one way it was identifying SVN commit IDs which I have since removed due to
> flakiness). Brock's patch was the single largest victim of this bug but not
> the only one, there were some from Cloudera, NexR, Hortonworks, Facebook
> even 2 from you Ed. The interested can see a full list of exclusions here:
> https://docs.google.com/spreadsheet/ccc?key=0ArmXd5zzNQm5dDJTMkFtaUk2d0dyU3hnWGJCcUczbXc#gid=0.
> I apologize to those under-represented, there wasn't any intent on my part
> to minimize anyone's work. The impact in final totals is Cloudera +5.4%,
> NexR +0.8%, Facebook -2.7%, Hortonworks -3.3%. I will be updating the blog
> later today with relevant corrections.
> 
> There is going to be continued interest in seeing charts like these, for
> example when Hive 12 is officially done. Sanjay suggested that LoC counts
> may not be the best way to represent true contribution. I agree that not
> all lines of code are created equal, for example a few monster patches
> recently went in re-arranging HCatalog namespaces and I think also
> indentation style. This (hopefully) mechanical work is not on the same
> footing as adding new query language features. Still it is work and
> wouldn't be fair to pretend it didn't happen. If anyone has ideas on better
> ways to fairly capture contribution I'm open to suggestions.
> 
> 
> 
> On Thu, Sep 12, 2013 at 7:19 AM, Edward Capriolo wrote:
> 
> > I was reading the horton-works blog and found an interesting article.
> >
> > http://hortonworks.com/blog/stinger-phase-2-the-journey-to-100x-faster-hive/#comment-160753
> >
> > There is a very interesting graphic which attempts to demonstrate lines of
> > code in the 12 release.
> > http://hortonworks.com/wp-content/uploads/2013/09/hive4.png
> >
> > Although I do not know how they are calculated, they are probably counting
> > code generated by tests output, but besides that they are wrong.
> >
> > One claim is that Cloudera contributed 4,244 lines of code.
> >
> > So to debunk that claim:
> >
> > In https://issues.apache.org/jira/browse/HIVE-4675 Brock Noland from
> > cloudera, created the ptest2 testing framework. He did all the work for
> > ptest2 in hive 12, and it is clearly more then 4,244
> >
> > This consists of 84 java files
> > [edward@desksandra ptest2]$ find . -name "*.java" | wc -l
> > 84
> > and by itself is 8001 lines of code.
> > [edward@desksandra ptest2]$ find . -name "*.java" | xargs cat | wc -l
> > 8001
> >
> > [edward@desksandra hive-trunk]$ wc -l HIVE-4675.patch
> > 7902 HIVE-4675.patch
> >
> > This is not the only feature from cloudera in hive 12.
> >
> > There is also a section of the article that talks of a "ROAD MAP" for hive
> > features. I did not know we (hive) had a road map. I have advocated
> > switching to feature based release and having a road map before, but it was
> > suggested that might limit people from itch-scratching.
> >
> >
> >
> >
> >
> 
> 
> -- 
> Carter Shanklin
> Director, Product Management
> Hortonworks
> (M): +1.650.644.8795 (T): @cshanklin 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

[jira] [Updated] (HIVE-5218) datanucleus does not work with MS SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-5218:
-

Summary: datanucleus does not work with MS SQLServer in Hive metastore  
(was: datanucleus does not work with SQLServer in Hive metastore)

> datanucleus does not work with MS SQLServer in Hive metastore
> -
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
> Attachments: 
> 0001-HIVE-5218-datanucleus-does-not-work-with-SQLServer-i.patch, 
> HIVE-5218.patch
>
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.
> This is the detailed exception I'm getting when using datanucleus 3.2.x with 
> SQL Server:
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTa
> sk. MetaException(message:javax.jdo.JDOException: Exception thrown calling 
> table
> .exists() for a2ee36af45e9f46c19e995bfd2d9b5fd1hivemetastore..SEQUENCE_TABLE
> at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc
> eption(NucleusJDOHelper.java:596)
> at 
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPe
> rsistenceManager.java:732)
> …
> at 
> org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawS
> tore.java:111)
> at $Proxy0.createTable(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_tabl
> e_core(HiveMetaStore.java:1071)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_tabl
> e_with_environment_context(HiveMetaStore.java:1104)
> …
> at $Proxy11.create_table_with_environment_context(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$cr
> eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6417)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$cr
> eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6401)
> NestedThrowablesStackTrace:
> com.microsoft.sqlserver.jdbc.SQLServerException: There is already an object 
> name
> d 'SEQUENCE_TABLE' in the database.
> at 
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
> (SQLServerException.java:197)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServ
> erStatement.java:1493)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQ
> LServerStatement.java:775)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute
> (SQLServerStatement.java:676)
> at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4615)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
> rverConnection.java:1400)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLSer
> verStatement.java:179)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLS
> erverStatement.java:154)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStat
> ement.java:649)
> at com.jolbox.bonecp.StatementHandle.execute(StatementHandle.java:300)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(A
> bstractTable.java:760)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatementLi
> st(AbstractTable.java:711)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.create(AbstractTable.
> java:425)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.
> java:488)
> at 
> org.datanucleus.store.rdbms.valuegenerator.TableGenerator.repositoryE
> xists(TableGenerator.java:242)
> at 
> org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obt
> ainGenerationBlock(AbstractRDBMSGenerator.java:86)
> at 
> org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerati
> onBlock(AbstractGenerator.java:197)
&g

[jira] [Commented] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767249#comment-13767249
 ] 

Konstantin Boudnik commented on HIVE-5218:
--

Let's make this ticket clear - this is MS SQLServer, not _any_ SQL server.

> datanucleus does not work with SQLServer in Hive metastore
> --
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
> Attachments: 
> 0001-HIVE-5218-datanucleus-does-not-work-with-SQLServer-i.patch, 
> HIVE-5218.patch
>
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.
> This is the detailed exception I'm getting when using datanucleus 3.2.x with 
> SQL Server:
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTa
> sk. MetaException(message:javax.jdo.JDOException: Exception thrown calling 
> table
> .exists() for a2ee36af45e9f46c19e995bfd2d9b5fd1hivemetastore..SEQUENCE_TABLE
> at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc
> eption(NucleusJDOHelper.java:596)
> at 
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPe
> rsistenceManager.java:732)
> …
> at 
> org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawS
> tore.java:111)
> at $Proxy0.createTable(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_tabl
> e_core(HiveMetaStore.java:1071)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_tabl
> e_with_environment_context(HiveMetaStore.java:1104)
> …
> at $Proxy11.create_table_with_environment_context(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$cr
> eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6417)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$cr
> eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6401)
> NestedThrowablesStackTrace:
> com.microsoft.sqlserver.jdbc.SQLServerException: There is already an object 
> name
> d 'SEQUENCE_TABLE' in the database.
> at 
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
> (SQLServerException.java:197)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServ
> erStatement.java:1493)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQ
> LServerStatement.java:775)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute
> (SQLServerStatement.java:676)
> at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4615)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
> rverConnection.java:1400)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLSer
> verStatement.java:179)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLS
> erverStatement.java:154)
> at 
> com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStat
> ement.java:649)
> at com.jolbox.bonecp.StatementHandle.execute(StatementHandle.java:300)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(A
> bstractTable.java:760)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatementLi
> st(AbstractTable.java:711)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.create(AbstractTable.
> java:425)
> at 
> org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.
> java:488)
> at 
> org.datanucleus.store.rdbms.valuegenerator.TableGenerator.repositoryE
> xists(TableGenerator.java:242)
> at 
> org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obt
> ainGenerationBlock(AbstractRDBMSGenerator.java:86)
> at 
> org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerati
> onBlock(AbstractGenerator.jav

[jira] [Updated] (HIVE-5087) Rename npath UDF to matchpath

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-5087:
-

Summary: Rename npath UDF to matchpath  (was: Rename npath UDF)

> Rename npath UDF to matchpath
> -
>
> Key: HIVE-5087
> URL: https://issues.apache.org/jira/browse/HIVE-5087
> Project: Hive
>  Issue Type: Bug
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Blocker
> Fix For: 0.12.0
>
> Attachments: HIVE-5087.1.patch.txt, HIVE-5087.99.patch.txt, 
> HIVE-5087-matchpath.1.patch.txt, HIVE-5087.patch.txt, HIVE-5087.patch.txt, 
> regex_path.diff
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Interesting claims that seem untrue

[Konstantin Boudnik]

"My Hadoop is bigger than yours. Part 2" in the movie theater near you... 

  /me Buying pop-corn

On Thu, Sep 12, 2013 at 10:19AM, Edward Capriolo wrote:
> I was reading the horton-works blog and found an interesting article.
> http://hortonworks.com/blog/stinger-phase-2-the-journey-to-100x-faster-hive/#comment-160753
> 
> There is a very interesting graphic which attempts to demonstrate lines of
> code in the 12 release.
> http://hortonworks.com/wp-content/uploads/2013/09/hive4.png
> 
> Although I do not know how they are calculated, they are probably counting
> code generated by tests output, but besides that they are wrong.
> 
> One claim is that Cloudera contributed 4,244 lines of code.
> 
> So to debunk that claim:
> 
> In https://issues.apache.org/jira/browse/HIVE-4675 Brock Noland from
> cloudera, created the ptest2 testing framework. He did all the work for
> ptest2 in hive 12, and it is clearly more then 4,244
> 
> This consists of 84 java files
> [edward@desksandra ptest2]$ find . -name "*.java" | wc -l
> 84
> and by itself is 8001 lines of code.
> [edward@desksandra ptest2]$ find . -name "*.java" | xargs cat | wc -l
> 8001
> 
> [edward@desksandra hive-trunk]$ wc -l HIVE-4675.patch
> 7902 HIVE-4675.patch
> 
> This is not the only feature from cloudera in hive 12.
> 
> There is also a section of the article that talks of a "ROAD MAP" for hive
> features. I did not know we (hive) had a road map. I have advocated
> switching to feature based release and having a road map before, but it was
> suggested that might limit people from itch-scratching.

[jira] [Commented] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763359#comment-13763359
 ] 

Konstantin Boudnik commented on HIVE-5205:
--

I'd argue that grouping of issues is only loosely coupled with the versions of 
a product. Grouping is logical; where's versioning is functional. In this 
particular case, parent issue is about JDK7 support. This ticket is exactly 
about it. But I really don't care one way or another how they are going to be 
connected to each other, until they are linked in some way.

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
>  Labels: target-version_0.11
> Fix For: 0.11.1
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763285#comment-13763285
 ] 

Konstantin Boudnik commented on HIVE-5205:
--

Shall we target it for 0.11.1 instead of simply closing it?

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>    Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
>  Labels: target-version_0.11
> Fix For: 0.11.1
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-5205:
-

Fix Version/s: (was: 0.12.0)
   0.11.1

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>        Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
>  Labels: target-version_0.11
> Fix For: 0.11.1
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-5205:
-

Labels: target-version_0.11  (was: )

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>        Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
>  Labels: target-version_0.11
> Fix For: 0.12.0
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762707#comment-13762707
 ] 

Konstantin Boudnik commented on HIVE-5218:
--

Hmm, this is interesting... I have tried this patch on top of HIVE-4900 and 
seems to solve the problem with the version in datanucleus used by Hive atm. 
Running patched version of Hive with Postgres 9.2 as metastore shows no problem.

> datanucleus does not work with SQLServer in Hive metastore
> --
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
> Attachments: 
> 0001-HIVE-5218-datanucleus-does-not-work-with-SQLServer-i.patch
>
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5089) Non query PreparedStatements are always failing on remote HiveServer2

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760795#comment-13760795
 ] 

Konstantin Boudnik commented on HIVE-5089:
--

Do you mind linking in the JIRA that addresses this issue on trunk?

> Non query PreparedStatements are always failing on remote HiveServer2
> -
>
> Key: HIVE-5089
> URL: https://issues.apache.org/jira/browse/HIVE-5089
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.11.0
>Reporter: Julien Letrouit
> Fix For: 0.12.0
>
>
> This is reproducing the issue systematically:
> {noformat}
> import org.apache.hive.jdbc.HiveDriver;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.PreparedStatement;
> public class Main {
>   public static void main(String[] args) throws Exception {
> DriverManager.registerDriver(new HiveDriver());
> Connection conn = DriverManager.getConnection("jdbc:hive2://someserver");
> PreparedStatement smt = conn.prepareStatement("SET hivevar:test=1");
> smt.execute(); // Exception here
> conn.close();
>   }
> }
> {noformat}
> It is producing the following stacktrace:
> {noformat}
> Exception in thread "main" java.sql.SQLException: Could not create ResultSet: 
> null
>   at 
> org.apache.hive.jdbc.HiveQueryResultSet.retrieveSchema(HiveQueryResultSet.java:183)
>   at 
> org.apache.hive.jdbc.HiveQueryResultSet.(HiveQueryResultSet.java:134)
>   at 
> org.apache.hive.jdbc.HiveQueryResultSet$Builder.build(HiveQueryResultSet.java:122)
>   at 
> org.apache.hive.jdbc.HivePreparedStatement.executeImmediate(HivePreparedStatement.java:194)
>   at 
> org.apache.hive.jdbc.HivePreparedStatement.execute(HivePreparedStatement.java:137)
>   at Main.main(Main.java:12)
> Caused by: org.apache.thrift.transport.TTransportException
>   at 
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>   at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>   at 
> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:346)
>   at 
> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:423)
>   at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:405)
>   at 
> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
>   at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>   at 
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
>   at 
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
>   at 
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
>   at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>   at 
> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_GetResultSetMetadata(TCLIService.java:466)
>   at 
> org.apache.hive.service.cli.thrift.TCLIService$Client.GetResultSetMetadata(TCLIService.java:453)
>   at 
> org.apache.hive.jdbc.HiveQueryResultSet.retrieveSchema(HiveQueryResultSet.java:154)
>   ... 5 more
> {noformat}
> I tried to fix it, unfortunately, the standalone server used in unit tests do 
> not reproduce the issue. The following test added to TestJdbcDriver2 is 
> passing:
> {noformat}
>   public void testNonQueryPrepareStatement() throws Exception {
> try {
>   PreparedStatement ps = con.prepareStatement("SET hivevar:test=1");
>   boolean hasResultSet = ps.execute();
>   assertTrue(hasResultSet);
>   ps.close();
> } catch (Exception e) {
>   e.printStackTrace();
>   fail(e.toString());
> }
>   }
> {noformat}
> Any guidance on how to reproduce it in tests would be appreciated.
> Impact: the data analysis tools we are using are performing 
> PreparedStatements. The use of custom UDF is forcing us to add 'ADD JAR ...' 
> and 'CREATE TEMPORARY FUNCTION ...' statement to our query. Those statements 
> are failing when executed as PreparedStatements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760580#comment-13760580
 ] 

Konstantin Boudnik commented on HIVE-5218:
--

I meant HIVE-4900

> datanucleus does not work with SQLServer in Hive metastore
> --
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760579#comment-13760579
 ] 

Konstantin Boudnik commented on HIVE-5218:
--

Actually, it looks like 4900 isn't complete.

> datanucleus does not work with SQLServer in Hive metastore
> --
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760550#comment-13760550
 ] 

Konstantin Boudnik commented on HIVE-5218:
--

With downgrade - as in HIVE-2084 - JDK7 stops working, actually. 

> datanucleus does not work with SQLServer in Hive metastore
> --
>
> Key: HIVE-5218
> URL: https://issues.apache.org/jira/browse/HIVE-5218
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
>
> HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of 
> datanucleus doesn't work with SQLServer as the metastore. The problem is that 
> datanucleus tries to use fully qualified object name to find a table in the 
> database but couldn't find it.
> If I downgrade the version to HIVE-2084, SQLServer works fine.
> It could be a bug in datanucleus.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758235#comment-13758235
 ] 

Konstantin Boudnik commented on HIVE-5205:
--

looks like it has been been fixed in the trunk as a part of 
  0129f3642660e8696646ac3d4aa414ef062e2118
Hence, the patch is only good for Hive 0.11

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>    Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
> Fix For: 0.12.0
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-5205:
-

Attachment: HIVE-5205.patch

That's do the trick.

> Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7
> --
>
> Key: HIVE-5205
> URL: https://issues.apache.org/jira/browse/HIVE-5205
> Project: Hive
>  Issue Type: Sub-task
>  Components: HCatalog
>Affects Versions: 0.11.0
>    Reporter: Konstantin Boudnik
> Fix For: 0.12.0
>
> Attachments: HIVE-5205.patch
>
>
> when building Hive with OpenJDK7 the following warning message makes the 
> build fail:
>   [javadoc] 
> /var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
>  warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-5205) Javadoc warnings in HCatalog prevent Hive from building under OpenJDK7

[Konstantin Boudnik (JIRA)]

Konstantin Boudnik created HIVE-5205:


 Summary: Javadoc warnings in HCatalog prevent Hive from building 
under OpenJDK7
 Key: HIVE-5205
 URL: https://issues.apache.org/jira/browse/HIVE-5205
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.11.0
Reporter: Konstantin Boudnik
 Fix For: 0.12.0


when building Hive with OpenJDK7 the following warning message makes the build 
fail:

  [javadoc] 
/var/lib/jenkins/workspace/Shark-Hive-0.11-OJDK7/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerFactory.java:81:
 warning - @return tag has no arguments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: custom Hive artifacts for Shark project

[Konstantin Boudnik]

Hi Edward,

Shark is using two jar files from Hive - hive-common and hive-cli. But Shark
community puts a few patches on top of the stock Hive to fix blocking issues
in the latter. The changes aren't proprietary and are either backports from
the newer releases or fixes that weren't committed yet (HIVE-3772 is good
example of this).

Taking into example Hive 0.9 which Shark 0.7 uses. Shark backports a few
bugfixes that were committed into Hive 0.10 or Hive 0.11, but never made it
into Hive 0.9. I believe this is a side effect of Hive always moving forward
and (almost) never making maintenance releases.

Changes and especially massive rewrites bring instability into the software.
It needs to be gradually ironed out with consequent releases. A good example
of such a project would be HBase, that does quite a number of minor releases
to provide their users with stable and robust server-side software. In the
absence of maintenance releases downstream projects tend to find ways to work
around such an obstacle. Hence my earlier email.

As of 0.11.1: Shark currently doesn't support Hive 0.11 because of significant
changes in the APIs of the latter. The support is coming in the next a couple
of months. So, publishing artifacts improving on top of Hive 0.9 might be more
a pressing issue.

Hope it clarifies the situation,
  Cos

On Sun, Aug 25, 2013 at 11:54PM, Edward Capriolo wrote:
> I think we plan on doing an 11.1 or just a 12.0. How does shark use hive?
> Do you just include hive components from maven or does the project somehow
> encorportate our build infrastructure.
> 
> 
> On Sun, Aug 25, 2013 at 7:42 PM, Konstantin Boudnik  wrote:
> 
> > Guys,
> >
> > considering the absence of the input, I take it that it really doesn't
> > matter
> > which way the custom artifact will be published. Is it a correct
> > impression?
> >
> > My first choice would be
> > org.apache.hive.hive-common;0.9-shark0.7
> > org.apache.hive.hive-cli;0.9-shark0.7
> > artifacts.
> > If this meets the objections from the community here, then I'd like to
> > proceed
> > with
> > org.shark-project.hive-common;0.9.0
> > org.shark-project.hive-cli;0.9.0
> >
> > Any of the artifacts are better be published at Maven central to make it
> > readily available for development community.
> >
> > Thoughts?
> > Regards,
> >   Cos
> >
> > On Sat, Aug 10, 2013 at 10:08PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > I am trying to help Spark/Shark community (spark-project.org and now
> > > http://incubator.apache.org/projects/spark) with a predicament. Shark -
> > that's
> > > also known as Hive on Spark - is using some parts of Hive, ie HQL parser,
> > > query optimizer, serdes, and codecs.
> > >
> > > In order to improve some known issues with performance and/or concurrency
> > > Shark developers need to apply a couple of patches on top of the stock
> > Hive:
> > >https://issues.apache.org/jira/browse/HIVE-2891
> > >https://issues.apache.org/jira/browse/HIVE-3772 (just committed to
> > trunk)
> > > (as per https://github.com/amplab/shark/wiki/Hive-Patches)
> > >
> > > The issue here is that latest Shark is working on top if Hive 0.9 (Hive
> > 0.11
> > > work is underway) and having developers to apply the patches and build
> > > their own version of the Hive is an extra step that can be avoided.
> > >
> > > One way to address it is to publish Shark specific versions of Hive
> > artifacts
> > > that would have all needed patches applied to stock release.  This way
> > > downstream projects can simply reference the version org.apache.hive with
> > > version 0.9.0-shark-0.7 instead of building Hive locally every time.
> > >
> > > Perhaps this approach is a little overkill, so perhaps if Hive community
> > is
> > > willing to consider a maintenance release of Hive 0.9.1 and perhaps
> > 0.11.1
> > > to include fixes needed by Shark project?
> > >
> > > I am willing to step up and produce Hive release bits if any of the
> > committers
> > > here can help with publishing.
> > >
> > > --
> > > Thanks in advance,
> > >   Cos
> > >
> >
> >
> >


signature.asc
Description: Digital signature

Re: custom Hive artifacts for Shark project

[Konstantin Boudnik]

Guys,

considering the absence of the input, I take it that it really doesn't matter
which way the custom artifact will be published. Is it a correct impression?

My first choice would be
org.apache.hive.hive-common;0.9-shark0.7
org.apache.hive.hive-cli;0.9-shark0.7
artifacts.
If this meets the objections from the community here, then I'd like to proceed
with 
org.shark-project.hive-common;0.9.0
org.shark-project.hive-cli;0.9.0

Any of the artifacts are better be published at Maven central to make it
readily available for development community.

Thoughts?
Regards,
  Cos

On Sat, Aug 10, 2013 at 10:08PM, Konstantin Boudnik wrote:
> Guys,
> 
> I am trying to help Spark/Shark community (spark-project.org and now
> http://incubator.apache.org/projects/spark) with a predicament. Shark - that's
> also known as Hive on Spark - is using some parts of Hive, ie HQL parser,
> query optimizer, serdes, and codecs. 
> 
> In order to improve some known issues with performance and/or concurrency
> Shark developers need to apply a couple of patches on top of the stock Hive:
>https://issues.apache.org/jira/browse/HIVE-2891
>https://issues.apache.org/jira/browse/HIVE-3772 (just committed to trunk)
> (as per https://github.com/amplab/shark/wiki/Hive-Patches)
> 
> The issue here is that latest Shark is working on top if Hive 0.9 (Hive 0.11
> work is underway) and having developers to apply the patches and build
> their own version of the Hive is an extra step that can be avoided. 
> 
> One way to address it is to publish Shark specific versions of Hive artifacts
> that would have all needed patches applied to stock release.  This way
> downstream projects can simply reference the version org.apache.hive with
> version 0.9.0-shark-0.7 instead of building Hive locally every time.
> 
> Perhaps this approach is a little overkill, so perhaps if Hive community is
> willing to consider a maintenance release of Hive 0.9.1 and perhaps 0.11.1
> to include fixes needed by Shark project?
> 
> I am willing to step up and produce Hive release bits if any of the committers
> here can help with publishing.
> 
> -- 
> Thanks in advance,
>   Cos
> 




signature.asc
Description: Digital signature

Re: Proposing a 0.11.1

[Konstantin Boudnik]

Hi.

Can I suggest to include 
  HIVE-3772 (performance fix; committed to trunk)
  HIVE-4583 (OpenJDK7 support; all subtasks seem to be done)

Thanks,
  Cos

On Tue, Aug 13, 2013 at 10:02AM, Owen O'Malley wrote:
> All,
>I'd like to create an 0.11.1 with some fixes in it. I plan to put
> together a release candidate over the next week. I'm in the process of
> putting together the list of bugs that I want to include, but I wanted to
> solicit the jiras that others though would be important for an 0.11.1.
> 
> Thanks,
>Owen

[jira] [Updated] (HIVE-3619) Hive JDBC driver should return a proper update-count of rows affected by query

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-3619:
-

Attachment: (was: HIVE-3619.patch)

> Hive JDBC driver should return a proper update-count of rows affected by query
> --
>
> Key: HIVE-3619
> URL: https://issues.apache.org/jira/browse/HIVE-3619
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Harsh J
>Priority: Minor
> Attachments: HIVE-3619.patch
>
>
> HiveStatement.java currently has an explicit 0 return:
> public int getUpdateCount() throws SQLException { return 0; }
> Ideally we ought to emit the exact number of rows affected by the query 
> statement itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3619) Hive JDBC driver should return a proper update-count of rows affected by query

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-3619:
-

Attachment: HIVE-3619.patch

> Hive JDBC driver should return a proper update-count of rows affected by query
> --
>
> Key: HIVE-3619
> URL: https://issues.apache.org/jira/browse/HIVE-3619
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Harsh J
>Priority: Minor
> Attachments: HIVE-3619.patch
>
>
> HiveStatement.java currently has an explicit 0 return:
> public int getUpdateCount() throws SQLException { return 0; }
> Ideally we ought to emit the exact number of rows affected by the query 
> statement itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3619) Hive JDBC driver should return a proper update-count of rows affected by query

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-3619:
-

Attachment: HIVE-3619.patch

Returning -1 as suggested above would at least give a correct indication to the 
client software about the expectations of this implementation.

> Hive JDBC driver should return a proper update-count of rows affected by query
> --
>
> Key: HIVE-3619
> URL: https://issues.apache.org/jira/browse/HIVE-3619
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Harsh J
>Priority: Minor
> Attachments: HIVE-3619.patch
>
>
> HiveStatement.java currently has an explicit 0 return:
> public int getUpdateCount() throws SQLException { return 0; }
> Ideally we ought to emit the exact number of rows affected by the query 
> statement itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

custom Hive artifacts for Shark project

[Konstantin Boudnik]

Guys,

I am trying to help Spark/Shark community (spark-project.org and now
http://incubator.apache.org/projects/spark) with a predicament. Shark - that's
also known as Hive on Spark - is using some parts of Hive, ie HQL parser,
query optimizer, serdes, and codecs. 

In order to improve some known issues with performance and/or concurrency
Shark developers need to apply a couple of patches on top of the stock Hive:
   https://issues.apache.org/jira/browse/HIVE-2891
   https://issues.apache.org/jira/browse/HIVE-3772 (just committed to trunk)
(as per https://github.com/amplab/shark/wiki/Hive-Patches)

The issue here is that latest Shark is working on top if Hive 0.9 (Hive 0.11
work is underway) and having developers to apply the patches and build
their own version of the Hive is an extra step that can be avoided. 

One way to address it is to publish Shark specific versions of Hive artifacts
that would have all needed patches applied to stock release.  This way
downstream projects can simply reference the version org.apache.hive with
version 0.9.0-shark-0.7 instead of building Hive locally every time.

Perhaps this approach is a little overkill, so perhaps if Hive community is
willing to consider a maintenance release of Hive 0.9.1 and perhaps 0.11.1
to include fixes needed by Shark project?

I am willing to step up and produce Hive release bits if any of the committers
here can help with publishing.

-- 
Thanks in advance,
Cos



signature.asc
Description: Digital signature

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736107#comment-13736107
 ] 

Konstantin Boudnik commented on HIVE-3772:
--

Got it, thanks for the explanation.

> Fix a concurrency bug in LazyBinaryUtils due to a static field
> --
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Fix For: 0.12.0
>
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772.1.patch.txt, 
> HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735981#comment-13735981
 ] 

Konstantin Boudnik commented on HIVE-3772:
--

Edward, any chance it can be also backported into 0.11.1 ?

> Fix a concurrency bug in LazyBinaryUtils due to a static field
> --
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Fix For: 0.12.0
>
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772.1.patch.txt, 
> HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735980#comment-13735980
 ] 

Konstantin Boudnik commented on HIVE-3772:
--

Thank you so much, Edward.

> Fix a concurrency bug in LazyBinaryUtils due to a static field
> --
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Fix For: 0.12.0
>
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772.1.patch.txt, 
> HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735537#comment-13735537
 ] 

Konstantin Boudnik commented on HIVE-3772:
--

The patch applies cleanly to the current 0.11 branch HEAD so it seems like a 
good candidate for maintenance 0.11.1 release?

> Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by 
> Reynold Xin)
> -
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3619) Hive JDBC driver should return a proper update-count of rows affected by query

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732658#comment-13732658
 ] 

Konstantin Boudnik commented on HIVE-3619:
--

At least returning {{-1}} in the interim would be good, no?

> Hive JDBC driver should return a proper update-count of rows affected by query
> --
>
> Key: HIVE-3619
> URL: https://issues.apache.org/jira/browse/HIVE-3619
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Harsh J
>Priority: Minor
>
> HiveStatement.java currently has an explicit 0 return:
> public int getUpdateCount() throws SQLException { return 0; }
> Ideally we ought to emit the exact number of rows affected by the query 
> statement itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3607) Set mapreduce.task.classpath.user.precedence to true by default

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726925#comment-13726925
 ] 

Konstantin Boudnik commented on HIVE-3607:
--

Are there any plans to commit this and HIVE-3606 in the bugfix release for Hive 
0.11 (if any?)

> Set mapreduce.task.classpath.user.precedence to true by default
> ---
>
> Key: HIVE-3607
> URL: https://issues.apache.org/jira/browse/HIVE-3607
> Project: Hive
>  Issue Type: Improvement
>  Components: Configuration
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>
> When queries are actually run in a Hadoop cluster, Hive's jars are appended 
> to Hadoop's classpath.  However, when we test/run jobs locally Hive's 
> classpath comes first.  This leads to issues like the one brought up here 
> after the patch was committed HIVE-3581 where a change depended on a jar Hive 
> includes which conflicted with one provided by Hadoop which is an older 
> version in 0.20
> It's possible that more of the jars we include are getting preceded by older 
> jars in Hadoop, and we haven't noticed yet.
> If we add Hive jars to the beginning of Hadoop's classpath we will be in 
> control in such situations where the jars are backwards compatible.  We will 
> be able to update the jars in Hive and these will be used at run time, 
> instead of just compile time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)

[Konstantin Boudnik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik updated HIVE-3772:
-

  Component/s: Serializers/Deserializers
Affects Version/s: 0.9.0

> Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by 
> Reynold Xin)
> -
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.9.0
>Reporter: Mikhail Bautin
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13651287#comment-13651287
 ] 

Konstantin Boudnik commented on HIVE-3772:
--

Is there any change of this get fixed in 0.11?

> Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by 
> Reynold Xin)
> -
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>Reporter: Mikhail Bautin
> Attachments: D7155.1.patch, D7155.2.patch, HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4114) hive-metastore.jar depends on jdo2-api:jar:2.3-ec, which is missing in maven central

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614667#comment-13614667
 ] 

Konstantin Boudnik commented on HIVE-4114:
--

Can we wrap it into a pom file and deploy to, perhaps, apache maven? 

> hive-metastore.jar depends on jdo2-api:jar:2.3-ec, which is missing in maven 
> central
> 
>
> Key: HIVE-4114
> URL: https://issues.apache.org/jira/browse/HIVE-4114
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Gopal V
>Priority: Trivial
>
> Adding hive-exec-0.10.0 to an independent pom.xml results in the following 
> error
> {code}
> Failed to retrieve javax.jdo:jdo2-api-2.3-ec
> Caused by: Could not find artifact javax.jdo:jdo2-api:jar:2.3-ec in central 
> (http://repo1.maven.org/maven2)
> ...
> Path to dependency: 
>   1) org.notmysock.hive:plan-viewer:jar:1.0-SNAPSHOT
>   2) org.apache.hive:hive-exec:jar:0.10.0
>   3) org.apache.hive:hive-metastore:jar:0.10.0
>   4) javax.jdo:jdo2-api:jar:2.3-ec
> {code}
> From the best I could tell, in the hive build ant+ivy pulls this file from 
> the datanucleus repo
> http://www.datanucleus.org/downloads/maven2/javax/jdo/jdo2-api/2.3-ec/
> For completeness sake, the dependency needs to be pulled to maven central.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1517) ability to select across a database

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444597#comment-13444597
 ] 

Konstantin Boudnik commented on HIVE-1517:
--

Do I understand correctly, that this fix has never been extended for the views?

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.10.patch, HIVE-1517.11.patch, 
> HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt, HIVE-1517.3.patch, 
> HIVE-1517.4.patch, HIVE-1517.5.patch, HIVE-1517.6.patch, HIVE-1517.7.patch, 
> HIVE-1517.8.patch, HIVE-1517.9.patch
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-80) Add testcases for concurrent query execution

[Konstantin Boudnik (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150193#comment-13150193
 ] 

Konstantin Boudnik commented on HIVE-80:


Perhaps my question is bigger than this issue or shall be asked somewhere else, 
but looking at Hive code I couldn't help noticing that it is essentially build 
as a singleton i.e. only a single instance of Hive object is allowed to exist. 

What was/is a design motivation behind of this? Why Hive can't be instantiated 
at will by the client so different instances can be independently for query 
analysis, job submissions, etc.? This will make Hive much more flexible and 
extendable from Hive client applications perspective, won't it?

> Add testcases for concurrent query execution
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hive
>  Issue Type: Test
>  Components: Query Processor, Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Carl Steinbach
>Priority: Critical
>  Labels: concurrency
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira