[jira] [Commented] (HIVE-6587) allow specifying additional Hive classpath for Hadoop
[ https://issues.apache.org/jira/browse/HIVE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931463#comment-13931463 ] Lefty Leverenz commented on HIVE-6587: -- bq. Allow users to add jars to hive's Hadoop classpath without explicitly modifying their Hadoop classpath This needs userdoc somewhere in the wiki. How about starting with a release note? allow specifying additional Hive classpath for Hadoop - Key: HIVE-6587 URL: https://issues.apache.org/jira/browse/HIVE-6587 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Trivial Fix For: 0.13.0 Attachments: HIVE-6587.patch Allow users to add jars to hive's Hadoop classpath without explicitly modifying their Hadoop classpath -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6616) Document ORC file format to enable development of external converters to/from ORC/text files
[ https://issues.apache.org/jira/browse/HIVE-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931467#comment-13931467 ] Lefty Leverenz commented on HIVE-6616: -- Does HIVE-5728 address your needs, or is that a separate issue? Document ORC file format to enable development of external converters to/from ORC/text files Key: HIVE-6616 URL: https://issues.apache.org/jira/browse/HIVE-6616 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.11.0, 0.12.0 Reporter: Michael Please document the structure of ORC file in a way that it allow writing and reading such a file by external software. I would like to be able to create ORC files myself without help of Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931470#comment-13931470 ] Vaibhav Gumashta commented on HIVE-5155: [~prasadm] Actually one thing I noticed we missed: bumping up the version number to V7 (since we changed the thrift interface)... Sorry for pointing out so late. But we can take that up in a follow up patch. Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6512) HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test
[ https://issues.apache.org/jira/browse/HIVE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6512: --- Status: Patch Available (was: Open) HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test Key: HIVE-6512 URL: https://issues.apache.org/jira/browse/HIVE-6512 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6512.1.patch Basically the test tries to test a kerberos doAs which is invalid since it doesn't do a kerberos login and it's not possible to unit test a kerberos setup. Surprisingly it has been hanging around for a while. Needs to be removed from the test suite. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6512) HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test
[ https://issues.apache.org/jira/browse/HIVE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6512: --- Status: Open (was: Patch Available) HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test Key: HIVE-6512 URL: https://issues.apache.org/jira/browse/HIVE-6512 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6512.1.patch Basically the test tries to test a kerberos doAs which is invalid since it doesn't do a kerberos login and it's not possible to unit test a kerberos setup. Surprisingly it has been hanging around for a while. Needs to be removed from the test suite. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931479#comment-13931479
]
Lefty Leverenz commented on HIVE-6359:
--
Should this bug fix be mentioned in the wiki, with version information?
* [HiveServer2 Clients: Beeline Command Options
|https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineCommandOptions]
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931482#comment-13931482
]
Thejas M Nair commented on HIVE-6359:
-
Yes, I think we can put a note for the -f option about this bug in 0.12 and
earlier, as this is not something that is easy to debug, and people could have
tabs in their script files.
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6571) query id should be available for logging during query compilation
[ https://issues.apache.org/jira/browse/HIVE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931486#comment-13931486 ] Lefty Leverenz commented on HIVE-6571: -- Q1: Why doesn't this jira have a fix version number? Q2: Should this change be mentioned in the wiki? For example, here: * [Getting Started: Error Logs |https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs] query id should be available for logging during query compilation - Key: HIVE-6571 URL: https://issues.apache.org/jira/browse/HIVE-6571 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Attachments: HIVE-6571.1.patch Would be nice to have the query id set during compilation to tie logs together etc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931502#comment-13931502
]
Lefty Leverenz commented on HIVE-6359:
--
Done, please review -f option in Beeline wikidoc (link 2 comments back).
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6455) Scalable dynamic partitioning and bucketing optimization
[
https://issues.apache.org/jira/browse/HIVE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931518#comment-13931518
]
Hive QA commented on HIVE-6455:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633838/HIVE-6455.14.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 5381 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_extended
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_dyn_part
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input9
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1707/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1707/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633838
Scalable dynamic partitioning and bucketing optimization
Key: HIVE-6455
URL: https://issues.apache.org/jira/browse/HIVE-6455
Project: Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Prasanth J
Assignee: Prasanth J
Labels: optimization
Attachments: HIVE-6455.1.patch, HIVE-6455.1.patch,
HIVE-6455.10.patch, HIVE-6455.10.patch, HIVE-6455.11.patch,
HIVE-6455.12.patch, HIVE-6455.13.patch, HIVE-6455.13.patch,
HIVE-6455.14.patch, HIVE-6455.2.patch, HIVE-6455.3.patch, HIVE-6455.4.patch,
HIVE-6455.4.patch, HIVE-6455.5.patch, HIVE-6455.6.patch, HIVE-6455.7.patch,
HIVE-6455.8.patch, HIVE-6455.9.patch, HIVE-6455.9.patch
The current implementation of dynamic partition works by keeping at least one
record writer open per dynamic partition directory. In case of bucketing
there can be multispray file writers which further adds up to the number of
open record writers. The record writers of column oriented file format (like
ORC, RCFile etc.) keeps some sort of in-memory buffers (value buffer or
compression buffers) open all the time to buffer up the rows and compress
them before flushing it to disk. Since these buffers are maintained per
column basis the amount of constant memory that will required at runtime
increases as the number of partitions and number of columns per partition
increases. This often leads to OutOfMemory (OOM) exception in mappers or
reducers depending on the number of open record writers. Users often tune the
JVM heapsize (runtime memory) to get over such OOM issues.
With this optimization, the dynamic partition columns and bucketing columns
(in case of bucketed tables) are sorted before being fed to the reducers.
Since the partitioning and bucketing columns are sorted, each reducers can
keep only one record writer open at any time thereby reducing the memory
pressure on the reducers. This optimization is highly scalable as the number
of partition and number of columns per partition increases at the cost of
sorting the columns.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931545#comment-13931545 ] Prasad Mujumdar commented on HIVE-5155: --- [~vaibhavgumashta] Good catch. Given that we are changing the version on every major interface change, this should cover that as well. It's a trivial change, I do have a patch ready. [~thejas] let me know if you want to review another update for this ticket or update that in a follow up patch. Thanks! Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931548#comment-13931548
]
Thejas M Nair commented on HIVE-6359:
-
The wikidoc update LGTM
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931550#comment-13931550 ] Vaibhav Gumashta commented on HIVE-5155: [~prasadm] I agree. It would be awesome if you can post the new patch. I have one more comment (I'll add to rb), but we can definitely take that up in a follow up jira :) Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931560#comment-13931560 ] Thejas M Nair commented on HIVE-5155: - I am planning to commit this soon, lets increment the version number in a followup jira. Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6625) HiveServer2 running in http mode should support trusted proxy access
Vaibhav Gumashta created HIVE-6625: -- Summary: HiveServer2 running in http mode should support trusted proxy access Key: HIVE-6625 URL: https://issues.apache.org/jira/browse/HIVE-6625 Project: Hive Issue Type: Improvement Environment: HIVE-5155 adds trusted proxy access to HiveServer2. This patch a minor change to have it used when running HiveServer2 in http mode. Patch to be applied on top of HIVE-4764 HIVE-5155. Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6625) HiveServer2 running in http mode should support trusted proxy access
[ https://issues.apache.org/jira/browse/HIVE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6625: --- Environment: (was: HIVE-5155 adds trusted proxy access to HiveServer2. This patch a minor change to have it used when running HiveServer2 in http mode. Patch to be applied on top of HIVE-4764 HIVE-5155.) HiveServer2 running in http mode should support trusted proxy access Key: HIVE-6625 URL: https://issues.apache.org/jira/browse/HIVE-6625 Project: Hive Issue Type: Improvement Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6625) HiveServer2 running in http mode should support trusted proxy access
[ https://issues.apache.org/jira/browse/HIVE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6625: --- Description: HIVE-5155 adds trusted proxy access to HiveServer2. This patch a minor change to have it used when running HiveServer2 in http mode. Patch to be applied on top of HIVE-4764 HIVE-5155. HiveServer2 running in http mode should support trusted proxy access Key: HIVE-6625 URL: https://issues.apache.org/jira/browse/HIVE-6625 Project: Hive Issue Type: Improvement Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta HIVE-5155 adds trusted proxy access to HiveServer2. This patch a minor change to have it used when running HiveServer2 in http mode. Patch to be applied on top of HIVE-4764 HIVE-5155. -- This message was sent by Atlassian JIRA (v6.2#6252)
Dot syntax for DESCRIBE in wiki
The wiki gives this syntax for DESCRIBEhttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column table/view/column: DESCRIBE [EXTENDED|FORMATTED] [db_name.]table_name[DOT col_name ( [DOT field_name] | [DOT '$elem$'] | [DOT '$key$'] | [DOT '$value$'] )* ] This is the only syntax with DOT, but apparently DOT just means . or perhaps it's a valid alternative to an actual dot. What's the story here? Is this something everybody knows except me? Note that [db_name.] shows an actual dot, so it's inconsistent with the rest of the syntax. Lars Francke added that part to the syntax in August 2013 (thanks, Lars) but if dots work then shouldn't the syntax show them? And if DOT is a keyword which also works, shouldn't both forms be shown? Furthermore, dots are now optional. So the doc needs an overhaul. Examples would help. Related jiras: - HIVE-1977 https://issues.apache.org/jira/browse/HIVE-1977: DESCRIBE TABLE syntax doesn't support specifying a database qualified table name -- *dots optional* - HIVE-5931https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924779#comment-13924779: SQL std auth - add metastore get_role_participants api - to support DESCRIBE ROLE -- *changed to SHOW PRINCIPALS role_name, but Thejas brought up the syntax issue for DESCRIBE* -- Lefty
[jira] [Commented] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931570#comment-13931570
]
Lefty Leverenz commented on HIVE-6359:
--
Thanks, night owl.
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6625) HiveServer2 running in http mode should support trusted proxy access
[ https://issues.apache.org/jira/browse/HIVE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6625: --- Attachment: HIVE-6625.1.patch Will create rb request / upload rebased patch when blocking jiras are committed. cc [~thejas] [~prasadm] HiveServer2 running in http mode should support trusted proxy access Key: HIVE-6625 URL: https://issues.apache.org/jira/browse/HIVE-6625 Project: Hive Issue Type: Improvement Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Attachments: HIVE-6625.1.patch HIVE-5155 adds trusted proxy access to HiveServer2. This patch a minor change to have it used when running HiveServer2 in http mode. Patch to be applied on top of HIVE-4764 HIVE-5155. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6626) HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR path
[
https://issues.apache.org/jira/browse/HIVE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-6626:
---
Description:
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, hive.session.id does not get expanded.
was:
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, the ${hive.session.id} is not expanded.
HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR path
-
Key: HIVE-6626
URL: https://issues.apache.org/jira/browse/HIVE-6626
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Fix For: 0.13.0
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, hive.session.id does not get expanded.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6626) HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR path
[
https://issues.apache.org/jira/browse/HIVE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-6626:
---
Description:
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, the ${hive.session.id} is not expanded.
was:
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, the ${hive.session.id} is not expanded.
HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR path
-
Key: HIVE-6626
URL: https://issues.apache.org/jira/browse/HIVE-6626
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Fix For: 0.13.0
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, the ${hive.session.id} is not expanded.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6626) HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR path
Vaibhav Gumashta created HIVE-6626:
--
Summary: HiveServer2 does not expand the DOWNLOADED_RESOURCES_DIR
path
Key: HIVE-6626
URL: https://issues.apache.org/jira/browse/HIVE-6626
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Fix For: 0.13.0
The downloaded scratch dir is specified in HiveConf as:
{code}
DOWNLOADED_RESOURCES_DIR(hive.downloaded.resources.dir,
System.getProperty(java.io.tmpdir) + File.separator +
${hive.session.id}_resources),
{code}
However, the ${hive.session.id} is not expanded.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6627) HiveServer2 should handle scratch dir permissions / errors in a better way
Vaibhav Gumashta created HIVE-6627: -- Summary: HiveServer2 should handle scratch dir permissions / errors in a better way Key: HIVE-6627 URL: https://issues.apache.org/jira/browse/HIVE-6627 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Should do the following: if permission is 777 , we don't need to try changing permissions 1. If owner change, the permissions to 777 for all 3 scratch dirs (if they don't exist create). 2. Else throw a meaningful permission denied error and exit. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Dot syntax for DESCRIBE in wiki
On Wed, Mar 12, 2014 at 2:16 AM, Lefty Leverenz leftylever...@gmail.comwrote: The wiki gives this syntax for DESCRIBE https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column table/view/column: DESCRIBE [EXTENDED|FORMATTED] [db_name.]table_name[DOT col_name ( [DOT field_name] | [DOT '$elem$'] | [DOT '$key$'] | [DOT '$value$'] )* ] This is the only syntax with DOT, but apparently DOT just means . or perhaps it's a valid alternative to an actual dot. What's the story here? Is this something everybody knows except me? DOT is just . . We should certainly make it consistent to avoid confusion. Yes, DOT is also optional now, but I am not a fan of that syntax. It has made it almost impossible to extend describe statements to other commands, as you can see from HIVE-5931https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924779#comment-13924779 . Lets just document the syntax with DOT for now. That way it be easier to fix the syntax later if we choose to. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6602) Multi-user HiveServer2 throws error
[ https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931584#comment-13931584 ] Vaibhav Gumashta commented on HIVE-6602: [~rhbutani] I think HIVE-6627 and HIVE-6626 should also get in if we're keeping this for 13. Or else this should be deferred. [~thejas]: What do you say? Multi-user HiveServer2 throws error --- Key: HIVE-6602 URL: https://issues.apache.org/jira/browse/HIVE-6602 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 Attachments: HIVE-6602.1.patch Error thrown: Error while processing statement: FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x For hive query execution, a scratch directory specified by hive.exec.scratchdir is created with default permission 700. In HiveServer2, during the CLIService startup, we check for the presence of scratch directories (local + dfs) and if they don't exist, create them with permission 777. However, we should also change the permission from the default 700 to 777 in case the dfs scratch directory already exists. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6488) Investigate TestBeeLineWithArgs
[
https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931592#comment-13931592
]
Jason Dere commented on HIVE-6488:
--
Looks like HS2 is continually trying/failing to connect to a zookeeper instance
at a default port 2181. Not sure when this started occurring or whether the
root cause of this needs to be fixed. Creating a zookeeper instance and
configuring the tests to use it allows this test to pass in under a minute.
{noformat}
2014-03-12 02:34:25,167 INFO zookeeper.ClientCnxn
(ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable
to locate a login configuration)
2014-03-12 02:34:25,177 WARN zookeeper.ClientCnxn (ClientCnxn.java:run(1089))
- Session 0x0 for server null, unexpected error, closing socket connection and
attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2014-03-12 02:34:25,285 WARN ZooKeeperHiveLockManager
(ZooKeeperHiveLockManager.java:setContext(121)) - Unexpected ZK exception when
creating parent node /hive_zookeeper_namespace
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode =
ConnectionLoss for /hive_zookeeper_namespace
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.setContext(ZooKeeperHiveLockManager.java:117)
at
org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager.getLockManager(DummyTxnManager.java:74)
at
org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager.acquireLocks(DummyTxnManager.java:103)
at
org.apache.hadoop.hive.ql.Driver.acquireReadWriteLocks(Driver.java:840)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1046)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:884)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:879)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:139)
at
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:62)
at
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:168)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
{noformat}
Investigate TestBeeLineWithArgs
---
Key: HIVE-6488
URL: https://issues.apache.org/jira/browse/HIVE-6488
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Blocker
TestBeeLineWithArgs started taking many, many hours and eventually timing out
which is one cause of precommit runs taking a long time. For now I have
skipped it in for precommit tests so we should figure out what is going on so
we can re-enable the test.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 13845: HIVE-5155: Support secure proxy user access to HiveServer2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/#review36906 --- service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java https://reviews.apache.org/r/13845/#comment68096 In case of a non-kerberos setup, this will do a doAs twice: one time using TUGIContainingProcessor and the second time at the session level. Actually getting rid of doAs at thrift processor level is a good idea since it ensures proper cleanup, but it might involve more work. HIVE-6312 aims to do that (patch available). I'm not sure if doing doAs twice will lead to any new issues (I don't think so). - Vaibhav Gumashta On March 10, 2014, 6:39 p.m., Prasad Mujumdar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/ --- (Updated March 10, 2014, 6:39 p.m.) Review request for hive, Brock Noland, Carl Steinbach, and Thejas Nair. Bugs: HIVE-5155 https://issues.apache.org/jira/browse/HIVE-5155 Repository: hive-git Description --- Delegation token support - Enable delegation token connection for HiveServer2 Enhance the TCLIService interface to support delegation token requests Support passing the delegation token connection type via JDBC URL and Beeline option Direct proxy access - Define new proxy user property Shim interfaces to validate proxy access for a given user Note that the diff doesn't include thrift generated code. Diffs - beeline/pom.xml 02bfaaa beeline/src/java/org/apache/hive/beeline/BeeLine.java e63a3b0 beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 91e20ec beeline/src/java/org/apache/hive/beeline/Commands.java d2d7fd3 beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java 94178ef beeline/src/test/org/apache/hive/beeline/ProxyAuthTest.java PRE-CREATION common/src/java/org/apache/hadoop/hive/conf/HiveConf.java edc3d38 conf/hive-default.xml.template a5a85b4 data/files/ProxyAuth.res PRE-CREATION itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 8210e75 jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java d08e05b jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 607fc7a jdbc/src/java/org/apache/hive/jdbc/Utils.java 608837e service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java d8ba3aa service/src/java/org/apache/hive/service/auth/KerberosSaslHelper.java 71dc592 service/src/java/org/apache/hive/service/auth/PlainSaslHelper.java 15b1675 service/src/java/org/apache/hive/service/cli/CLIService.java 2b1e712 service/src/java/org/apache/hive/service/cli/CLIServiceClient.java b9d1489 service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java a31ea94 service/src/java/org/apache/hive/service/cli/ICLIService.java 621d689 service/src/java/org/apache/hive/service/cli/session/HiveSession.java c8fb8ec service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java d6d0d27 service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java b934ebe service/src/java/org/apache/hive/service/cli/session/SessionManager.java cec3b04 service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 26bda5a service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java 3675e86 service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 8fa4afd service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java 2fac800 shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 51c8051 shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java e205caa shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 29114f0 shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java dc89de1 shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java e15ab4e shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 03f4e51 Diff: https://reviews.apache.org/r/13845/diff/ Testing --- Since this requires kerberos setup, its tested by a standalone test program that runs various existing and new secure connection scenarios. The test code is attached to the ticket at https://issues.apache.org/jira/secure/attachment/12600119/ProxyAuth.java Thanks, Prasad Mujumdar
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931595#comment-13931595 ] Vaibhav Gumashta commented on HIVE-5155: [~prasadm] Left a comment on rb, but I think if HIVE-6312 (by [~navis]) gets in, it will address the issue. Thanks! Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
Sure, we can defer 6068. Actually 6602 too, since there are a couple of related jiras that need to be addressed. However, if possible can we have https://issues.apache.org/jira/browse/HIVE-6625 in 13? Its big brother https://issues.apache.org/jira/browse/HIVE-5155 is ready to be committed and HIVE-6625 is a minor patch on top of it. Thanks a lot! On Tue, Mar 11, 2014 at 1:17 PM, Harish Butani hbut...@hortonworks.comwrote: ok with the first 3. HIVE-6068 doesn't have a patch yet. Can this be deferred? On Mar 11, 2014, at 12:06 PM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Can you please consider the following: https://issues.apache.org/jira/browse/HIVE-6602 (committed to trunk), https://issues.apache.org/jira/browse/HIVE-6512, https://issues.apache.org/jira/browse/HIVE-6068, https://issues.apache.org/jira/browse/HIVE-6580. Most of them are bug fixes. Thanks, --Vaibhav On Tue, Mar 11, 2014 at 8:39 AM, Harish Butani hbut...@hortonworks.com wrote: yes sure. On Mar 10, 2014, at 3:55 PM, Gopal V gop...@apache.org wrote: Can I add HIVE-6518 as well to the merge queue on https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status It is a relatively simple OOM safety patch to vectorized group-by. Tests pass locally for vec group-by, but the pre-commit tests haven't fired eventhough it's been PA for a while now. Cheers, Gopal -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Assigned] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reassigned HIVE-5931: --- Assignee: Thejas M Nair SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.2.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.nothrifgen.2.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h Support command for listing all members of a role. A new metastore api call also needs to be added for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to trunk and 0.13 branch. I didn't realize that I had not checked with Harish on including this in 0.13. Sorry, about that. I have now included it the wiki page. cc [~rhbutani] SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.2.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.nothrifgen.2.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h Support command for listing all members of a role. A new metastore api call also needs to be added for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Fix Version/s: 0.13.0 SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.13.0 Attachments: HIVE-5931.1.patch, HIVE-5931.2.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.nothrifgen.2.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h Support command for listing all members of a role. A new metastore api call also needs to be added for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 17437: doAs with plain sasl auth should be session aware
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17437/#review36907 --- service/src/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java https://reviews.apache.org/r/17437/#comment68098 I think we should access thread locals through the SessionManager. service/src/java/org/apache/hive/service/cli/session/SessionManager.java https://reviews.apache.org/r/17437/#comment68097 I think accessing thread locals through SessionManager is a good idea. A number of other patches seem to rely on this. - Vaibhav Gumashta On Jan. 28, 2014, 1:21 a.m., Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17437/ --- (Updated Jan. 28, 2014, 1:21 a.m.) Review request for hive. Bugs: HIVE-6312 https://issues.apache.org/jira/browse/HIVE-6312 Repository: hive-git Description --- TUGIContainingProcessor creates new Subject for each invocation which induces FileSystem leakage when cache is enable(true by default). Diffs - service/src/java/org/apache/hive/service/auth/PlainSaslHelper.java 15b1675 service/src/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java 0bf34ce service/src/java/org/apache/hive/service/cli/session/HiveSession.java c8fb8ec service/src/java/org/apache/hive/service/cli/session/HiveSessionBase.java PRE-CREATION service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 445c858 service/src/java/org/apache/hive/service/cli/session/HiveSessionProxy.java 76f18a9 service/src/java/org/apache/hive/service/cli/session/SessionManager.java bfe0e7b service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java b5a6138 service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 8fa4afd Diff: https://reviews.apache.org/r/17437/diff/ Testing --- Thanks, Navis Ryu
[jira] [Commented] (HIVE-6312) doAs with plain sasl auth should be session aware
[ https://issues.apache.org/jira/browse/HIVE-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931622#comment-13931622 ] Vaibhav Gumashta commented on HIVE-6312: [~navis] Left some minor comments on rb. How about we get rid of TUGIContainingProcessor class too since it won't be used anymore? doAs with plain sasl auth should be session aware - Key: HIVE-6312 URL: https://issues.apache.org/jira/browse/HIVE-6312 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Navis Assignee: Navis Attachments: HIVE-6312.1.patch.txt, HIVE-6312.2.patch.txt TUGIContainingProcessor creates new Subject for each invocation which induces FileSystem leakage when cache is enable(true by default). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6616) Document ORC file format to enable development of external converters to/from ORC/text files
[ https://issues.apache.org/jira/browse/HIVE-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931623#comment-13931623 ] Michael commented on HIVE-6616: --- Yes, it is the same issue but from a non-Hive developer. As such, I do not want to dive into Hive source code and look for appropriate classes and methods. I need two things: 1. Clear documentation with fields, sizes, types, offsets, etc. that allows creation of an ORC file without using Hive code 2. Clear and comprehensive user guide (preferably with usage examples) for any Hive API that can help in creating an ORC file. I can be a tester for such documentation (and any other if it takes reasonable time). Document ORC file format to enable development of external converters to/from ORC/text files Key: HIVE-6616 URL: https://issues.apache.org/jira/browse/HIVE-6616 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.11.0, 0.12.0 Reporter: Michael Please document the structure of ORC file in a way that it allow writing and reading such a file by external software. I would like to be able to create ORC files myself without help of Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5155: Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Patch committed to trunk and 0.13 branch (this is included in the list for 0.13 maintained by Harish). Thanks for the contribution [~prasadm]. Thanks for reviews Vaibhav, Brock. Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.13.0 Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, HIVE-5155.5.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6512) HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test
[ https://issues.apache.org/jira/browse/HIVE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6512: Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to trunk and 0.13 branch. This is a blocker for HIVE-4764 which is tracked in the 0.13 cwiki, which needs this test case fix. cc [~rhbutani]. Thanks for the contribution Vaibhav ! HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test Key: HIVE-6512 URL: https://issues.apache.org/jira/browse/HIVE-6512 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6512.1.patch Basically the test tries to test a kerberos doAs which is invalid since it doesn't do a kerberos login and it's not possible to unit test a kerberos setup. Surprisingly it has been hanging around for a while. Needs to be removed from the test suite. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931673#comment-13931673 ] Thejas M Nair commented on HIVE-4764: - This does not apply to trunk after HIVE-5155 went in. [~vgumashta] Can you please rebase ? Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931689#comment-13931689
]
Hive QA commented on HIVE-6012:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633756/HIVE-6012.6.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5391 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1708/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1708/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633756
restore backward compatibility of arithmetic operations
---
Key: HIVE-6012
URL: https://issues.apache.org/jira/browse/HIVE-6012
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Jason Dere
Attachments: HIVE-6012.1.patch, HIVE-6012.2.patch, HIVE-6012.3.patch,
HIVE-6012.4.patch, HIVE-6012.5.patch, HIVE-6012.6.patch
HIVE-5356 changed the behavior of some of the arithmetic operations, and the
change is not backward compatible, as pointed out in this [jira
comment|https://issues.apache.org/jira/browse/HIVE-5356?focusedCommentId=13813398page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13813398]
{code}
int / int = decimal
float / float = double
float * float = double
float + float = double
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5099) Some partition publish operation cause OOM in metastore backed by SQL Server
[
https://issues.apache.org/jira/browse/HIVE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-5099:
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Patch committed to trunk and 0.13 branch (already in Harish's list for 0.13).
Thanks Daniel!
Some partition publish operation cause OOM in metastore backed by SQL Server
Key: HIVE-5099
URL: https://issues.apache.org/jira/browse/HIVE-5099
Project: Hive
Issue Type: Bug
Components: Metastore, Windows
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.13.0
Attachments: HIVE-5099-1.patch, HIVE-5099-2.patch, HIVE-5099.3.patch
For certain metastore operation combination, metastore operation hangs and
metastore server eventually fail due to OOM. This happens when metastore is
backed by SQL Server. Here is a testcase to reproduce:
{code}
CREATE TABLE tbl_repro_oom1 (a STRING, b INT) PARTITIONED BY (c STRING, d
STRING);
CREATE TABLE tbl_repro_oom_2 (a STRING ) PARTITIONED BY (e STRING);
ALTER TABLE tbl_repro_oom1 ADD PARTITION (c='France', d=4);
ALTER TABLE tbl_repro_oom1 ADD PARTITION (c='Russia', d=3);
ALTER TABLE tbl_repro_oom_2 ADD PARTITION (e='Russia');
ALTER TABLE tbl_repro_oom1 DROP PARTITION (c = 'India'); --failure
{code}
The code cause the issue is in ExpressionTree.java:
{code}
valString = partitionName.substring(partitionName.indexOf(\ + keyEqual +
\)+ + keyEqualLength + ).substring(0,
partitionName.substring(partitionName.indexOf(\ + keyEqual + \)+ +
keyEqualLength + ).indexOf(\/\));
{code}
The snapshot of table partition before the drop partition statement is:
{code}
PART_ID CREATE_TIMELAST_ACCESS_TIME PART_NAMESD_ID
TBL_ID
931376526718 0c=France/d=4 127 33
941376526718 0c=Russia/d=3 128 33
951376526718 0e=Russia 129 34
{code}
Datanucleus query try to find the value of a particular key by locating
$key= as the start, / as the end. For example, value of c in
c=France/d=4 by locating c= as the start, / following as the end.
However, this query fail if we try to find value e in e=Russia since
there is no tailing /.
Other database works since the query plan first filter out the partition not
belonging to tbl_repro_oom1. Whether this error surface or not depends on the
query optimizer.
When this exception happens, metastore keep trying and throw exception. The
memory image of metastore contains a large number of exception objects:
{code}
com.microsoft.sqlserver.jdbc.SQLServerException: Invalid length parameter
passed to the LEFT or SUBSTRING function.
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:4762)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1682)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:955)
at
org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at
org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at
org.datanucleus.store.rdbms.query.ForwardQueryResult.init(ForwardQueryResult.java:90)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:686)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1791)
at org.datanucleus.store.query.Query.executeWithMap(Query.java:1694)
at org.datanucleus.api.jdo.JDOQuery.executeWithMap(JDOQuery.java:334)
at
org.apache.hadoop.hive.metastore.ObjectStore.listMPartitionsByFilter(ObjectStore.java:1715)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1590)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy4.getPartitionsByFilter(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_filter(HiveMetaStore.java:2163)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
[jira] [Updated] (HIVE-6499) Using Metastore-side Auth errors on non-resolvable IF/OF/SerDe
[ https://issues.apache.org/jira/browse/HIVE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6499: Attachment: HIVE-6499.patch HIVE-6499.patch - uploading a copy of the patch, in case the pre-commit tests have marked earlier file as processed. Using Metastore-side Auth errors on non-resolvable IF/OF/SerDe -- Key: HIVE-6499 URL: https://issues.apache.org/jira/browse/HIVE-6499 Project: Hive Issue Type: Bug Components: Metastore, Security Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6499.patch, HIVE-6499.patch In cases where a user needs to use a custom IF/OF/SerDe that is not accessible from the metastore, calls like msc.createTable and msc.dropTable should still work without being able to load the class. This is possible as long as one does not enable MetaStore-side authorization, at which point this becomes impossible, erroring out with a ClassNotFoundException. The reason this happens is that since the AuthorizationProvider interface is defined against a ql.metadata.Table, we wind up needing to instantiate a ql.metadata.Table object, which, in its constructor tries to instantiate IF/OF/SerDe elements in an attempt to pre-load those fields. And if we do not have access to those classes in the metastore, this is when that fails. The constructor/initialize methods of Table and Partition do not really need to pre-initialize these fields, since the fields are accessed only through the accessor, and will be instantiated on first-use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6628) Use UDFs in create table statement
nicolas maillard created HIVE-6628: -- Summary: Use UDFs in create table statement Key: HIVE-6628 URL: https://issues.apache.org/jira/browse/HIVE-6628 Project: Hive Issue Type: New Feature Components: CLI Reporter: nicolas maillard Priority: Trivial It would be nice to be able to use UDFs in a create table statement Say my data is : userid, timestamp utc, list_of_things 123,1386716402,thing1;thing2:thing3 Being able to say create external table mytable( userid int adate string as from_utc_timestamp(tilmestamp,Europe/Paris) listofthings arrayString as split( list_of_things, ;) )ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' this is like a much lighter serde or a simpler view I guess. It would allow to correct the view of certains fields on the fly without needing to do reproscessing. this is a use case we see happening a lot in our inital data collections -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6507) OrcFile table property names are specified as strings
[ https://issues.apache.org/jira/browse/HIVE-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6507: Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Patch committed to trunk and 0.13 branch (in list of 0.13 patches maintained by Harish). Thanks for the contribution Sushanth! OrcFile table property names are specified as strings - Key: HIVE-6507 URL: https://issues.apache.org/jira/browse/HIVE-6507 Project: Hive Issue Type: Bug Components: HCatalog, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Fix For: 0.13.0 Attachments: HIVE-6507.2.patch, HIVE-6507.patch In HIVE-5504, we had to do some special casing in HCatalog to add a particular set of orc table properties from table properties to job properties. In doing so, it's obvious that that is a bit cumbersome, and ideally, the list of all orc file table properties should really be an enum, rather than individual loosely tied constant strings. If we were to clean this up, we can clean up other code that references this to reference the entire enum, and avoid future errors when new table properties are introduced, but other referencing code is not updated. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6629) Discuss SET ROLE NONE
Brock Noland created HIVE-6629: -- Summary: Discuss SET ROLE NONE Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE syntax before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-6629: --- Description: I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. was: I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-6629: --- Description: I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. was: I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE syntax before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931771#comment-13931771 ] Brock Noland commented on HIVE-6629: Hey [~thejas] I created this blocker not because I am trying to be difficult, but because I feel we need to discuss this before the 0.13 release as I find SET ROLE NONE to be counter-intuitive. If we decided to change that behavior to SET ROLE ALL, I'd be more than happy to create the patch myself, today. FYI [~rhbutani] Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE syntax before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931835#comment-13931835 ] Thejas M Nair commented on HIVE-6629: - The sql standard says If NONE is specified, then the current role name is removed. . If you consider that it resets the action of 'set role' command, it should go to the default state. Standard does not define a set role default or set role all. But standard does not allow to have more than one current role, which explains why all/default would not make sense in standard. In addition to the databases you referenced, I also checked Teradata, that also follows the same Oracle style semantics. I am fine changing the action of set role none to be specified by set role all. It might also make sense to reserve the use of default role for future. [~alangates] [~ashutoshc] [~sushanth] Do you guys have an opinion on this ? Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931845#comment-13931845 ] Brock Noland commented on HIVE-6629: bq. I am fine changing the action of set role none to be specified by set role all. It might also make sense to reserve the use of default role for future. This makes sense to me. Unless there is disagreement in the new couple hours I'll go ahead and create a patch to that effect. Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931864#comment-13931864 ] Thejas M Nair commented on HIVE-6629: - We should make the role names NONE, ALL and DEFAULT reserved, ie, users should not be allowed to create roles with those names. Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6629) Discuss SET ROLE NONE
[ https://issues.apache.org/jira/browse/HIVE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931865#comment-13931865 ] Brock Noland commented on HIVE-6629: bq. We should make the role names NONE, ALL and DEFAULT reserved, ie, users should not be allowed to create roles with those names. +1 Discuss SET ROLE NONE - Key: HIVE-6629 URL: https://issues.apache.org/jira/browse/HIVE-6629 Project: Hive Issue Type: Task Affects Versions: 0.13.0 Reporter: Brock Noland Priority: Blocker Fix For: 0.13.0 I understand this is well after it's been implemented. However, I think we need to discuss the counter-intuitive SET ROLE semantics before we release 0.13. The new Hive Authz work implements SET ROLE NONE similar to MySQL meaning that SET ROLE NONE actually sets your role to the default. This is extremely counter-intuitive. * [Oracle|http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10004.htm] * [Informix|http://pic.dhe.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm.sqls.doc%2Fids_sqs_1186.htm] * [Vertica|https://my.vertica.com/docs/5.1.6/HTML/index.htm#15645.htm] * [SAP|http://www.sapdb.org/htmhelp/44/a17998442911d3a98200a0c9449261/content.htm] All of the widely deployed databases above have SET ROLE NONE disable all privileges. Those databases have the intuitive SET ROLE ALL or SET ROLE DEFAULT enable all privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6434) Restrict function create/drop to admin roles
[
https://issues.apache.org/jira/browse/HIVE-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931894#comment-13931894
]
Hive QA commented on HIVE-6434:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633760/HIVE-6434.6.patch
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 5386 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_dboutput
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_lateral_view_explode2
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_avg
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_group_concat
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_max
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_max_n
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_min
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udaf_example_min_n
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_add
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_arraymapstruct
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_format
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_row_sequence
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udtf_explode2
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udtf_output_on_close
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_case_with_row_sequence
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_invalid_row_sequence
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_udtf_explode2
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1709/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1709/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633760
Restrict function create/drop to admin roles
Key: HIVE-6434
URL: https://issues.apache.org/jira/browse/HIVE-6434
Project: Hive
Issue Type: Sub-task
Components: Authorization, UDF
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6434.1.patch, HIVE-6434.2.patch, HIVE-6434.3.patch,
HIVE-6434.4.patch, HIVE-6434.5.patch, HIVE-6434.6.patch
Restrict function create/drop to admin roles, if sql std auth is enabled.
This would include temp/permanent functions, as well as macros.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 19127: Bug fix for fs based stats collection in presence of LB.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19127/ --- Review request for hive and Gunther Hagleitner. Bugs: HIVE-6630 https://issues.apache.org/jira/browse/HIVE-6630 Repository: hive-git Description --- No need to track per dir stats. Tracking per partition suffices. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 2cd8756 ql/src/java/org/apache/hadoop/hive/ql/stats/fs/FSStatsPublisher.java bd6dc62 ql/src/test/queries/clientpositive/lb_fs_stats.q PRE-CREATION ql/src/test/results/clientpositive/lb_fs_stats.q.out PRE-CREATION Diff: https://reviews.apache.org/r/19127/diff/ Testing --- Added new test Thanks, Ashutosh Chauhan
[jira] [Updated] (HIVE-6630) FS based stats collection have issues for list bucketing case
[ https://issues.apache.org/jira/browse/HIVE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6630: --- Attachment: HIVE-6630.patch Simple fix. FS based stats collection have issues for list bucketing case - Key: HIVE-6630 URL: https://issues.apache.org/jira/browse/HIVE-6630 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6630.patch We need not to track per directory stats in FS based stats collection mechanism, which other stats collection mechanism need to do. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6630) FS based stats collection have issues for list bucketing case
[ https://issues.apache.org/jira/browse/HIVE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6630: --- Status: Patch Available (was: Open) FS based stats collection have issues for list bucketing case - Key: HIVE-6630 URL: https://issues.apache.org/jira/browse/HIVE-6630 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6630.patch We need not to track per directory stats in FS based stats collection mechanism, which other stats collection mechanism need to do. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6630) FS based stats collection have issues for list bucketing case
Ashutosh Chauhan created HIVE-6630: -- Summary: FS based stats collection have issues for list bucketing case Key: HIVE-6630 URL: https://issues.apache.org/jira/browse/HIVE-6630 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan We need not to track per directory stats in FS based stats collection mechanism, which other stats collection mechanism need to do. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6631) NPE when select a field of a struct from a table stored by ORC
Yin Huai created HIVE-6631:
--
Summary: NPE when select a field of a struct from a table stored
by ORC
Key: HIVE-6631
URL: https://issues.apache.org/jira/browse/HIVE-6631
Project: Hive
Issue Type: Bug
Reporter: Yin Huai
I have two tables lineitem_orc_cg
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2 STRUCTL_QUANTITY:float,
L_EXTENDEDPRICE:float,
L_DISCOUNT:float,
L_SHIPDATE:STRING
)
row format serde 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
stored as orc tblproperties (orc.compress=NONE);
{code}
When I want to select a field from a struct by using
{code:sql}
select cg1.l_comment from lineitem_orc_cg limit 1;
{code}
I got
{code}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:928)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:954)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:459)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:415)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:409)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
... 22 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6631) NPE when select a field of a struct from a table stored by ORC
[
https://issues.apache.org/jira/browse/HIVE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated HIVE-6631:
---
Component/s: Serializers/Deserializers
Query Processor
NPE when select a field of a struct from a table stored by ORC
--
Key: HIVE-6631
URL: https://issues.apache.org/jira/browse/HIVE-6631
Project: Hive
Issue Type: Bug
Components: Query Processor, Serializers/Deserializers
Affects Versions: 0.13.0, 0.14.0
Reporter: Yin Huai
I have two tables lineitem_orc_cg
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2 STRUCTL_QUANTITY:float,
L_EXTENDEDPRICE:float,
L_DISCOUNT:float,
L_SHIPDATE:STRING
)
row format serde 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
stored as orc tblproperties (orc.compress=NONE);
{code}
When I want to select a field from a struct by using
{code:sql}
select cg1.l_comment from lineitem_orc_cg limit 1;
{code}
I got
{code}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:928)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:954)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:459)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:415)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:409)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
... 22 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6631) NPE when select a field of a struct from a table stored by ORC
[
https://issues.apache.org/jira/browse/HIVE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated HIVE-6631:
---
Affects Version/s: 0.14.0
0.13.0
NPE when select a field of a struct from a table stored by ORC
--
Key: HIVE-6631
URL: https://issues.apache.org/jira/browse/HIVE-6631
Project: Hive
Issue Type: Bug
Components: Query Processor, Serializers/Deserializers
Affects Versions: 0.13.0, 0.14.0
Reporter: Yin Huai
I have two tables lineitem_orc_cg
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2 STRUCTL_QUANTITY:float,
L_EXTENDEDPRICE:float,
L_DISCOUNT:float,
L_SHIPDATE:STRING
)
row format serde 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
stored as orc tblproperties (orc.compress=NONE);
{code}
When I want to select a field from a struct by using
{code:sql}
select cg1.l_comment from lineitem_orc_cg limit 1;
{code}
I got
{code}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:928)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:954)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:459)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:415)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:409)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
... 22 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6631) NPE when select a field of a struct from a table stored by ORC
[
https://issues.apache.org/jira/browse/HIVE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated HIVE-6631:
---
Description:
I have a table like this ...
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2 STRUCTL_QUANTITY:float,
L_EXTENDEDPRICE:float,
L_DISCOUNT:float,
L_SHIPDATE:STRING
)
row format serde 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
stored as orc tblproperties (orc.compress=NONE);
{code}
When I want to select a field from a struct by using
{code:sql}
select cg1.l_comment from lineitem_orc_cg limit 1;
{code}
I got
{code}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:928)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:954)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:459)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:415)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:409)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
... 22 more
{code}
was:
I have two tables lineitem_orc_cg
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2 STRUCTL_QUANTITY:float,
L_EXTENDEDPRICE:float,
L_DISCOUNT:float,
L_SHIPDATE:STRING
)
row format serde 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
stored as orc tblproperties (orc.compress=NONE);
{code}
When I want to select a field from a struct by using
{code:sql}
select cg1.l_comment from lineitem_orc_cg limit 1;
{code}
I got
{code}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:928)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:954)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:459)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:415)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:409)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
... 22 more
{code}
NPE when select a field of a struct from a table stored by ORC
--
Key: HIVE-6631
URL: https://issues.apache.org/jira/browse/HIVE-6631
Project: Hive
Issue Type: Bug
Components: Query Processor, Serializers/Deserializers
Affects Versions: 0.13.0, 0.14.0
Reporter: Yin Huai
I have a table like this ...
{code:sql}
create table lineitem_orc_cg
(
CG1 STRUCTL_PARTKEY:INT,
L_SUPPKEY:INT,
L_COMMITDATE:STRING,
L_RECEIPTDATE:STRING,
L_SHIPINSTRUCT:STRING,
L_SHIPMODE:STRING,
L_COMMENT:STRING,
L_TAX:float,
L_RETURNFLAG:STRING,
L_LINESTATUS:STRING,
L_LINENUMBER:INT,
L_ORDERKEY:INT,
CG2
[jira] [Updated] (HIVE-6609) Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2
[ https://issues.apache.org/jira/browse/HIVE-6609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6609: --- Status: Open (was: Patch Available) Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2 - Key: HIVE-6609 URL: https://issues.apache.org/jira/browse/HIVE-6609 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.12.0, 0.11.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6609.patch This is because url based job killing which we use doesn't work on hadoop2. We need to use java api. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6609) Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2
[ https://issues.apache.org/jira/browse/HIVE-6609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6609: --- Attachment: HIVE-6609.2.patch Addressed Mohammad's comments. Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2 - Key: HIVE-6609 URL: https://issues.apache.org/jira/browse/HIVE-6609 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0, 0.12.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6609.2.patch, HIVE-6609.patch This is because url based job killing which we use doesn't work on hadoop2. We need to use java api. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6632) ORC should be able to only read needed fields in a complex column
Yin Huai created HIVE-6632: -- Summary: ORC should be able to only read needed fields in a complex column Key: HIVE-6632 URL: https://issues.apache.org/jira/browse/HIVE-6632 Project: Hive Issue Type: Improvement Reporter: Yin Huai Currently, we use a string of ids to record needed columns. However, this string cannot record needed fields of a complex column. Although ORC decomposes a complex column to multiple sub-columns, it has to load the entire complex column if only a single field of this complex column is needed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6609) Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2
[ https://issues.apache.org/jira/browse/HIVE-6609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6609: --- Status: Patch Available (was: Open) Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2 - Key: HIVE-6609 URL: https://issues.apache.org/jira/browse/HIVE-6609 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.12.0, 0.11.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6609.2.patch, HIVE-6609.patch This is because url based job killing which we use doesn't work on hadoop2. We need to use java api. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18992: Fix ctrl-C killing MR jobs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18992/ --- (Updated March 12, 2014, 4:49 p.m.) Review request for hive. Changes --- Addressed Mohammad's comments. Bugs: HIVE-6609 https://issues.apache.org/jira/browse/HIVE-6609 Repository: hive-git Description --- Switched to java api to kill jobs instead of url based job killing. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 288da8e ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java a4585de ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 5a6899c ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 4b58d92 ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/truncate/ColumnTruncateTask.java 21b537c ql/src/java/org/apache/hadoop/hive/ql/plan/ReducerTimeStatsPerJob.java 40c27e3 Diff: https://reviews.apache.org/r/18992/diff/ Testing --- Manually tested for both Hadoop-1 Hadoop-2. Working as expected. Thanks, Ashutosh Chauhan
[jira] [Commented] (HIVE-6632) ORC should be able to only read needed fields in a complex column
[ https://issues.apache.org/jira/browse/HIVE-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931978#comment-13931978 ] Yin Huai commented on HIVE-6632: Does Parquet have the same issue? ORC should be able to only read needed fields in a complex column - Key: HIVE-6632 URL: https://issues.apache.org/jira/browse/HIVE-6632 Project: Hive Issue Type: Improvement Reporter: Yin Huai Currently, we use a string of ids to record needed columns. However, this string cannot record needed fields of a complex column. Although ORC decomposes a complex column to multiple sub-columns, it has to load the entire complex column if only a single field of this complex column is needed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-3009) do authorization for all metadata operations
[ https://issues.apache.org/jira/browse/HIVE-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931984#comment-13931984 ] Alex Nastetsky commented on HIVE-3009: -- Hi, are there still plans to fix this ticket? It is creating a security concern in our scenario where we have different databases for each customer but any user can do show tables in any database. Thanks. do authorization for all metadata operations Key: HIVE-3009 URL: https://issues.apache.org/jira/browse/HIVE-3009 Project: Hive Issue Type: Bug Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Vandana Ayyalasomayajula Most of the metadata read operations and some write operations are not checking for authorization. See org.apache.hadoop.hive.ql.plan.HiveOperation . Operations such as DESCTABLE and DROPDATABASE have null for required privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6587) allow specifying additional Hive classpath for Hadoop
[ https://issues.apache.org/jira/browse/HIVE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6587: --- Release Note: HIVE_CLASSPATH environment variable support has been added. This is added to Hadoop classpath for jobs. allow specifying additional Hive classpath for Hadoop - Key: HIVE-6587 URL: https://issues.apache.org/jira/browse/HIVE-6587 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Trivial Fix For: 0.13.0 Attachments: HIVE-6587.patch Allow users to add jars to hive's Hadoop classpath without explicitly modifying their Hadoop classpath -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6587) allow specifying additional Hive classpath for Hadoop
[ https://issues.apache.org/jira/browse/HIVE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932006#comment-13932006 ] Sergey Shelukhin commented on HIVE-6587: Added allow specifying additional Hive classpath for Hadoop - Key: HIVE-6587 URL: https://issues.apache.org/jira/browse/HIVE-6587 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Trivial Fix For: 0.13.0 Attachments: HIVE-6587.patch Allow users to add jars to hive's Hadoop classpath without explicitly modifying their Hadoop classpath -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6575) select * fails on parquet table with map datatype
[
https://issues.apache.org/jira/browse/HIVE-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932014#comment-13932014
]
Szehon Ho commented on HIVE-6575:
-
Hi [~xuefuz] can we commit this when you get a chance? If possible it would be
good to get it in for 0.13. Thanks.
select * fails on parquet table with map datatype
-
Key: HIVE-6575
URL: https://issues.apache.org/jira/browse/HIVE-6575
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6575.2.patch, HIVE-6575.3.patch, HIVE-6575.patch
Create parquet table with map and run select * from parquet_table, returns
following exception:
{noformat}
FAILED: RuntimeException java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.parquet.serde.DeepParquetHiveMapInspector cannot
be cast to
org.apache.hadoop.hive.ql.io.parquet.serde.StandardParquetHiveMapInspector
{noformat}
However select mapcol from parquet_table seems to work, and thus joins will
work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (HIVE-6633) pig -useHCatalog with embedded metastore fails to pass command line args to metastore
[ https://issues.apache.org/jira/browse/HIVE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson reassigned HIVE-6633: - Assignee: Eric Hanson pig -useHCatalog with embedded metastore fails to pass command line args to metastore - Key: HIVE-6633 URL: https://issues.apache.org/jira/browse/HIVE-6633 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0, 0.12.0, 0.13.0, 0.14.0 Reporter: Eric Hanson Assignee: Eric Hanson Fix For: 0.14.0 This fails because the embedded metastore can't connect to the database because the command line -D arguments passed to pig are not getting passed to the metastore when the embedded metastore is created. Using hive.metastore.uris set to the empty string causes creation of an embedded metastore. pig -useHCatalog -Dhive.metastore.uris= -Djavax.jdo.option.ConnectionPassword=AzureSQLDBXYZ The goal is to allow a pig job submitted via WebHCat to specify a metastore to use via job arguments. That is not working because it is not possible to pass Djavax.jdo.option.ConnectionPassword and other necessary arguments to the embedded metastore. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6633) pig -useHCatalog with embedded metastore fails to pass command line args to metastore
Eric Hanson created HIVE-6633: - Summary: pig -useHCatalog with embedded metastore fails to pass command line args to metastore Key: HIVE-6633 URL: https://issues.apache.org/jira/browse/HIVE-6633 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0, 0.11.0, 0.13.0, 0.14.0 Reporter: Eric Hanson Fix For: 0.14.0 This fails because the embedded metastore can't connect to the database because the command line -D arguments passed to pig are not getting passed to the metastore when the embedded metastore is created. Using hive.metastore.uris set to the empty string causes creation of an embedded metastore. pig -useHCatalog -Dhive.metastore.uris= -Djavax.jdo.option.ConnectionPassword=AzureSQLDBXYZ The goal is to allow a pig job submitted via WebHCat to specify a metastore to use via job arguments. That is not working because it is not possible to pass Djavax.jdo.option.ConnectionPassword and other necessary arguments to the embedded metastore. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6575) select * fails on parquet table with map datatype
[
https://issues.apache.org/jira/browse/HIVE-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932057#comment-13932057
]
Xuefu Zhang commented on HIVE-6575:
---
[~szehon] I just notice that there is no test case for this. A test case would
be good to prevent future breakage. I can commit this, but could you please
create and work on a followup JIRA adding the missing test case?
select * fails on parquet table with map datatype
-
Key: HIVE-6575
URL: https://issues.apache.org/jira/browse/HIVE-6575
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6575.2.patch, HIVE-6575.3.patch, HIVE-6575.patch
Create parquet table with map and run select * from parquet_table, returns
following exception:
{noformat}
FAILED: RuntimeException java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.parquet.serde.DeepParquetHiveMapInspector cannot
be cast to
org.apache.hadoop.hive.ql.io.parquet.serde.StandardParquetHiveMapInspector
{noformat}
However select mapcol from parquet_table seems to work, and thus joins will
work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6575) select * fails on parquet table with map datatype
[
https://issues.apache.org/jira/browse/HIVE-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932066#comment-13932066
]
Szehon Ho commented on HIVE-6575:
-
OK I created it, will work on that one next.
select * fails on parquet table with map datatype
-
Key: HIVE-6575
URL: https://issues.apache.org/jira/browse/HIVE-6575
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6575.2.patch, HIVE-6575.3.patch, HIVE-6575.patch
Create parquet table with map and run select * from parquet_table, returns
following exception:
{noformat}
FAILED: RuntimeException java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.parquet.serde.DeepParquetHiveMapInspector cannot
be cast to
org.apache.hadoop.hive.ql.io.parquet.serde.StandardParquetHiveMapInspector
{noformat}
However select mapcol from parquet_table seems to work, and thus joins will
work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6634) Add test case for HIVE-6575
Szehon Ho created HIVE-6634: --- Summary: Add test case for HIVE-6575 Key: HIVE-6634 URL: https://issues.apache.org/jira/browse/HIVE-6634 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Szehon Ho Assignee: Szehon Ho Follow up JIRA to HIVE-6575 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6495) TableDesc.getDeserializer() should use correct classloader when calling Class.forName()
[
https://issues.apache.org/jira/browse/HIVE-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932064#comment-13932064
]
Hive QA commented on HIVE-6495:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633762/HIVE-6495.2.patch
{color:green}SUCCESS:{color} +1 5381 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1710/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1710/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633762
TableDesc.getDeserializer() should use correct classloader when calling
Class.forName()
---
Key: HIVE-6495
URL: https://issues.apache.org/jira/browse/HIVE-6495
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6495.1.patch, HIVE-6495.2.patch
User is getting an error with the following stack trace below. It looks like
when Class.forName() is called, it may not be using the correct class loader
(JavaUtils.getClassLoader() is used in other contexts when the loaded jar may
be required).
{noformat}
FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException:
Failed with exception java.lang.ClassNotFoundException:
my.serde.ColonSerdejava.lang.RuntimeException:
java.lang.ClassNotFoundException: my.serde.ColonSerde
at
org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:68)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRowInspectorFromTable(FetchOperator.java:231)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getOutputObjectInspector(FetchOperator.java:608)
at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:80)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at
org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:66)
... 20 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6635) Heartbeats are not being sent when DbLockMgr is used and an operation holds locks
Alan Gates created HIVE-6635: Summary: Heartbeats are not being sent when DbLockMgr is used and an operation holds locks Key: HIVE-6635 URL: https://issues.apache.org/jira/browse/HIVE-6635 Project: Hive Issue Type: Bug Components: Locking Affects Versions: 0.13.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 The new DbLockManager depends on heartbeats from the client in order to determine that a lock has not timed out. The client is not currently sending those heartbeats. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6575) select * fails on parquet table with map datatype
[
https://issues.apache.org/jira/browse/HIVE-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6575:
--
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Patch committed to trunk. Thanks goes to Szehon for the contribution.
select * fails on parquet table with map datatype
-
Key: HIVE-6575
URL: https://issues.apache.org/jira/browse/HIVE-6575
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Fix For: 0.14.0
Attachments: HIVE-6575.2.patch, HIVE-6575.3.patch, HIVE-6575.patch
Create parquet table with map and run select * from parquet_table, returns
following exception:
{noformat}
FAILED: RuntimeException java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.parquet.serde.DeepParquetHiveMapInspector cannot
be cast to
org.apache.hadoop.hive.ql.io.parquet.serde.StandardParquetHiveMapInspector
{noformat}
However select mapcol from parquet_table seems to work, and thus joins will
work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Timeline for the Hive 0.13 release?
Hi Harish, Could you please add HIVE-6575, which is already resolved in trunk, but bad enough to be fixed for 0.13 as well? Thanks, Xuefu On Wed, Mar 12, 2014 at 3:13 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Sure, we can defer 6068. Actually 6602 too, since there are a couple of related jiras that need to be addressed. However, if possible can we have https://issues.apache.org/jira/browse/HIVE-6625 in 13? Its big brother https://issues.apache.org/jira/browse/HIVE-5155 is ready to be committed and HIVE-6625 is a minor patch on top of it. Thanks a lot! On Tue, Mar 11, 2014 at 1:17 PM, Harish Butani hbut...@hortonworks.com wrote: ok with the first 3. HIVE-6068 doesn't have a patch yet. Can this be deferred? On Mar 11, 2014, at 12:06 PM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Can you please consider the following: https://issues.apache.org/jira/browse/HIVE-6602 (committed to trunk), https://issues.apache.org/jira/browse/HIVE-6512, https://issues.apache.org/jira/browse/HIVE-6068, https://issues.apache.org/jira/browse/HIVE-6580. Most of them are bug fixes. Thanks, --Vaibhav On Tue, Mar 11, 2014 at 8:39 AM, Harish Butani hbut...@hortonworks.com wrote: yes sure. On Mar 10, 2014, at 3:55 PM, Gopal V gop...@apache.org wrote: Can I add HIVE-6518 as well to the merge queue on https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status It is a relatively simple OOM safety patch to vectorized group-by. Tests pass locally for vec group-by, but the pre-commit tests haven't fired eventhough it's been PA for a while now. Cheers, Gopal -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6495) TableDesc.getDeserializer() should use correct classloader when calling Class.forName()
[
https://issues.apache.org/jira/browse/HIVE-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-6495:
---
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Committed to 0.13 trunk. Thanks, Jason!
TableDesc.getDeserializer() should use correct classloader when calling
Class.forName()
---
Key: HIVE-6495
URL: https://issues.apache.org/jira/browse/HIVE-6495
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Jason Dere
Assignee: Jason Dere
Fix For: 0.13.0
Attachments: HIVE-6495.1.patch, HIVE-6495.2.patch
User is getting an error with the following stack trace below. It looks like
when Class.forName() is called, it may not be using the correct class loader
(JavaUtils.getClassLoader() is used in other contexts when the loaded jar may
be required).
{noformat}
FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException:
Failed with exception java.lang.ClassNotFoundException:
my.serde.ColonSerdejava.lang.RuntimeException:
java.lang.ClassNotFoundException: my.serde.ColonSerde
at
org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:68)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRowInspectorFromTable(FetchOperator.java:231)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getOutputObjectInspector(FetchOperator.java:608)
at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:80)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at
org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:66)
... 20 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
Sergey Shelukhin created HIVE-6636: -- Summary: /user/hive is a bad default for HDFS jars path for Tez Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932080#comment-13932080
]
Lefty Leverenz commented on HIVE-6012:
--
Doc questions: Does *hive.compat* accept any Hive release number, or only
some? The default is 0.12 but can it be set to 0.12.0? Would it balk at
0.12.5 or just ignore .5? What about 0.11 and earlier releases -- would they
give the same behavior as 0.12?
restore backward compatibility of arithmetic operations
---
Key: HIVE-6012
URL: https://issues.apache.org/jira/browse/HIVE-6012
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Jason Dere
Attachments: HIVE-6012.1.patch, HIVE-6012.2.patch, HIVE-6012.3.patch,
HIVE-6012.4.patch, HIVE-6012.5.patch, HIVE-6012.6.patch
HIVE-5356 changed the behavior of some of the arithmetic operations, and the
change is not backward compatible, as pointed out in this [jira
comment|https://issues.apache.org/jira/browse/HIVE-5356?focusedCommentId=13813398page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13813398]
{code}
int / int = decimal
float / float = double
float * float = double
float + float = double
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6618) assertion when getting reference key from loader with byte-array mapjoin key
[ https://issues.apache.org/jira/browse/HIVE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932084#comment-13932084 ] Gunther Hagleitner commented on HIVE-6618: -- +1 LGTM. assertion when getting reference key from loader with byte-array mapjoin key Key: HIVE-6618 URL: https://issues.apache.org/jira/browse/HIVE-6618 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6618.patch java.lang.AssertionError: Should be called after loading tables at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.processRow(MapRecordProcessor.java:205) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:171) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:152) This is because tables may have already been loaded. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6637) UDF in_file() doesn't take CHAR or VARCHAR as input
Xuefu Zhang created HIVE-6637:
-
Summary: UDF in_file() doesn't take CHAR or VARCHAR as input
Key: HIVE-6637
URL: https://issues.apache.org/jira/browse/HIVE-6637
Project: Hive
Issue Type: Bug
Components: Types, UDF
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
{code}
hive desc alter_varchar_1;
key string None
value varchar(3) None
key2int None
value2 varchar(10) None
hive select in_file(value, value2) from alter_varchar_1;
FAILED: SemanticException [Error 10016]: Line 1:15 Argument type mismatch
'value': The 1st argument of function IN_FILE must be a string but
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableHiveVarcharObjectInspector@10f1f34a
was given.
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[ https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6636: --- Attachment: HIVE-6636.patch /user/hive is a bad default for HDFS jars path for Tez -- Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6636.patch If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6430) MapJoin hash table has large memory overhead
[ https://issues.apache.org/jira/browse/HIVE-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932094#comment-13932094 ] Sergey Shelukhin commented on HIVE-6430: the test changes were not intentional... merged wrong branch MapJoin hash table has large memory overhead Key: HIVE-6430 URL: https://issues.apache.org/jira/browse/HIVE-6430 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6430.01.patch, HIVE-6430.patch Right now, in some queries, I see that storing e.g. 4 ints (2 for key and 2 for row) can take several hundred bytes, which is ridiculous. I am reducing the size of MJKey and MJRowContainer in other jiras, but in general we don't need to have java hash table there. We can either use primitive-friendly hashtable like the one from HPPC (Apache-licenced), or some variation, to map primitive keys to single row storage structure without an object per row (similar to vectorization). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[
https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-6636:
---
Fix Version/s: 0.13.0
Release Note: Hive on Tez will now use /user/current Hadoop user name,
rather than just /user/hive, as default HDFS directory for jars.
{{hive.jar.directory}} can still be used to set the path explicitly.
Status: Patch Available (was: Open)
/user/hive is a bad default for HDFS jars path for Tez
--
Key: HIVE-6636
URL: https://issues.apache.org/jira/browse/HIVE-6636
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.13.0
Attachments: HIVE-6636.patch
If user runs hive under the user name that is not hive, jobs will fail
until everyone is granted write access to /user/hive, which is not nice.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932099#comment-13932099
]
Jason Dere commented on HIVE-6012:
--
Based on exact string match with the supported release numbers (0.12, 0.13, or
latest to use latest supported level). If it does not recognize number it
will squawk error and use default support level.
restore backward compatibility of arithmetic operations
---
Key: HIVE-6012
URL: https://issues.apache.org/jira/browse/HIVE-6012
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Jason Dere
Attachments: HIVE-6012.1.patch, HIVE-6012.2.patch, HIVE-6012.3.patch,
HIVE-6012.4.patch, HIVE-6012.5.patch, HIVE-6012.6.patch
HIVE-5356 changed the behavior of some of the arithmetic operations, and the
change is not backward compatible, as pointed out in this [jira
comment|https://issues.apache.org/jira/browse/HIVE-5356?focusedCommentId=13813398page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13813398]
{code}
int / int = decimal
float / float = double
float * float = double
float + float = double
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932100#comment-13932100
]
Jason Dere commented on HIVE-6012:
--
Test case Minimr bucketizedhiveinputformat.q passes for me when run locally, on
both Mac/Linux. Flaky test?
restore backward compatibility of arithmetic operations
---
Key: HIVE-6012
URL: https://issues.apache.org/jira/browse/HIVE-6012
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Jason Dere
Attachments: HIVE-6012.1.patch, HIVE-6012.2.patch, HIVE-6012.3.patch,
HIVE-6012.4.patch, HIVE-6012.5.patch, HIVE-6012.6.patch
HIVE-5356 changed the behavior of some of the arithmetic operations, and the
change is not backward compatible, as pointed out in this [jira
comment|https://issues.apache.org/jira/browse/HIVE-5356?focusedCommentId=13813398page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13813398]
{code}
int / int = decimal
float / float = double
float * float = double
float + float = double
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[ https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932108#comment-13932108 ] Gunther Hagleitner commented on HIVE-6636: -- Looks good except: - You're setting the var to , I think that means that your null check won't catch that the var wasn't set and you really catch it only in the fs.status call. Why not set it to null in the first place? - You should update the hive-default.xml.template too /user/hive is a bad default for HDFS jars path for Tez -- Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-6636.patch If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6488) Investigate TestBeeLineWithArgs
[ https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6488: - Attachment: HIVE-6488.1.patch Patch v1 fixes the test by using non-zookeeper lock manager, avoiding the need to connect to zookeeper when creating table. Investigate TestBeeLineWithArgs --- Key: HIVE-6488 URL: https://issues.apache.org/jira/browse/HIVE-6488 Project: Hive Issue Type: Bug Components: Tests Reporter: Brock Noland Priority: Blocker Attachments: HIVE-6488.1.patch TestBeeLineWithArgs started taking many, many hours and eventually timing out which is one cause of precommit runs taking a long time. For now I have skipped it in for precommit tests so we should figure out what is going on so we can re-enable the test. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HIVE-6488) Investigate TestBeeLineWithArgs
[ https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-6488: Assignee: Jason Dere Investigate TestBeeLineWithArgs --- Key: HIVE-6488 URL: https://issues.apache.org/jira/browse/HIVE-6488 Project: Hive Issue Type: Bug Components: Tests Reporter: Brock Noland Assignee: Jason Dere Priority: Blocker Attachments: HIVE-6488.1.patch TestBeeLineWithArgs started taking many, many hours and eventually timing out which is one cause of precommit runs taking a long time. For now I have skipped it in for precommit tests so we should figure out what is going on so we can re-enable the test. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6488) Investigate TestBeeLineWithArgs
[ https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6488: - Component/s: Tests Investigate TestBeeLineWithArgs --- Key: HIVE-6488 URL: https://issues.apache.org/jira/browse/HIVE-6488 Project: Hive Issue Type: Bug Components: Tests Reporter: Brock Noland Assignee: Jason Dere Priority: Blocker Attachments: HIVE-6488.1.patch TestBeeLineWithArgs started taking many, many hours and eventually timing out which is one cause of precommit runs taking a long time. For now I have skipped it in for precommit tests so we should figure out what is going on so we can re-enable the test. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6488) Investigate TestBeeLineWithArgs
[ https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6488: - Status: Patch Available (was: Open) Investigate TestBeeLineWithArgs --- Key: HIVE-6488 URL: https://issues.apache.org/jira/browse/HIVE-6488 Project: Hive Issue Type: Bug Components: Tests Reporter: Brock Noland Assignee: Jason Dere Priority: Blocker Attachments: HIVE-6488.1.patch TestBeeLineWithArgs started taking many, many hours and eventually timing out which is one cause of precommit runs taking a long time. For now I have skipped it in for precommit tests so we should figure out what is going on so we can re-enable the test. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[ https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932115#comment-13932115 ] Sergey Shelukhin commented on HIVE-6636: 1 - last-minute change, will fix; 2 - will do /user/hive is a bad default for HDFS jars path for Tez -- Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-6636.patch If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6037) Synchronize HiveConf with hive-default.xml.template and support show conf
[ https://issues.apache.org/jira/browse/HIVE-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932119#comment-13932119 ] Lefty Leverenz commented on HIVE-6037: -- TLDR (last comment)? I'd like to know if HIVE-6037 will be included in release 0.13.0. Synchronize HiveConf with hive-default.xml.template and support show conf - Key: HIVE-6037 URL: https://issues.apache.org/jira/browse/HIVE-6037 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.13.0 Attachments: CHIVE-6037.3.patch.txt, HIVE-6037.1.patch.txt, HIVE-6037.10.patch.txt, HIVE-6037.11.patch.txt, HIVE-6037.12.patch.txt, HIVE-6037.14.patch.txt, HIVE-6037.15.patch.txt, HIVE-6037.16.patch.txt, HIVE-6037.17.patch, HIVE-6037.2.patch.txt, HIVE-6037.4.patch.txt, HIVE-6037.5.patch.txt, HIVE-6037.6.patch.txt, HIVE-6037.7.patch.txt, HIVE-6037.8.patch.txt, HIVE-6037.9.patch.txt, HIVE-6037.patch see HIVE-5879 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[ https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932121#comment-13932121 ] Sergey Shelukhin commented on HIVE-6636: 1 actually I'm passing the default explicitly: +String hdfsDirPathStr = HiveConf.getVar(conf, HiveConf.ConfVars.HIVE_JAR_DIRECTORY, null); /user/hive is a bad default for HDFS jars path for Tez -- Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-6636.patch If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6488) Investigate TestBeeLineWithArgs
[ https://issues.apache.org/jira/browse/HIVE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932123#comment-13932123 ] Ashutosh Chauhan commented on HIVE-6488: +1 test is about Beeline, not about ZK or lock management, so ok to test with Embedded LM. Investigate TestBeeLineWithArgs --- Key: HIVE-6488 URL: https://issues.apache.org/jira/browse/HIVE-6488 Project: Hive Issue Type: Bug Components: Tests Reporter: Brock Noland Assignee: Jason Dere Priority: Blocker Attachments: HIVE-6488.1.patch TestBeeLineWithArgs started taking many, many hours and eventually timing out which is one cause of precommit runs taking a long time. For now I have skipped it in for precommit tests so we should figure out what is going on so we can re-enable the test. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6636) /user/hive is a bad default for HDFS jars path for Tez
[ https://issues.apache.org/jira/browse/HIVE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6636: --- Attachment: HIVE-6636.01.patch /user/hive is a bad default for HDFS jars path for Tez -- Key: HIVE-6636 URL: https://issues.apache.org/jira/browse/HIVE-6636 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-6636.01.patch, HIVE-6636.patch If user runs hive under the user name that is not hive, jobs will fail until everyone is granted write access to /user/hive, which is not nice. -- This message was sent by Atlassian JIRA (v6.2#6252)
