[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-17 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288086#comment-15288086
 ] 

Lefty Leverenz commented on HIVE-13449:
---

Doc note:  This adds *hive.llap.create.token.locally* to HiveConf.java, so it 
will need to be documented in the wiki for release 2.1.0.

* [Configuration Properties -- LLAP | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-LLAP]

Added a TODOC2.1 label.

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>  Labels: TODOC2.1
> Fix For: 2.1.0
>
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.04.patch, 
> HIVE-13449.05.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-17 Thread Siddharth Seth (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287739#comment-15287739
 ] 

Siddharth Seth commented on HIVE-13449:
---

+1.

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.04.patch, 
> HIVE-13449.05.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-16 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286075#comment-15286075
 ] 

Lefty Leverenz commented on HIVE-13449:
---

+1 for the parameter description in HiveConf.java 
(hive.llap.create.token.locally).

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.04.patch, 
> HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-11 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280936#comment-15280936
 ] 

Sergey Shelukhin commented on HIVE-13449:
-

[~hagleitn] [~vikram.dixit] ping?

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275300#comment-15275300
 ] 

Hive QA commented on HIVE-13449:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12802541/HIVE-13449.03.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 9938 tests 
executed
*Failed tests:*
{noformat}
TestCliDriver-gen_udf_example_add10.q-ppd_join4.q-union27.q-and-12-more - did 
not produce a TEST-*.xml file
TestCliDriver-partition_timestamp.q-ppd_random.q-vector_outer_join5.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-ptf_general_queries.q-unionDistinct_1.q-groupby1_noskew.q-and-12-more
 - did not produce a TEST-*.xml file
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
TestMiniTezCliDriver-enforce_order.q-vector_partition_diff_num_cols.q-unionDistinct_1.q-and-12-more
 - did not produce a TEST-*.xml file
TestMiniTezCliDriver-insert_values_non_partitioned.q-join1.q-schema_evol_orc_nonvec_mapwork_part.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-smb_mapjoin_15.q-mapreduce1.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt3.q-union27.q-multigroupby_singlemr.q-and-12-more 
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_case_with_row_sequence
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf.testGetMetaConfDefault
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveOperationType.checkHiveOperationTypeMatch
org.apache.hive.service.cli.session.TestHiveSessionImpl.testLeakOperationHandle
org.apache.hive.service.cli.session.TestSessionManagerMetrics.testThreadPoolMetrics
{noformat}

Test results: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/205/testReport
Console output: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/205/console
Test logs: 
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-205/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12802541 - PreCommit-HIVE-MASTER-Build

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-05 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273353#comment-15273353
 ] 

Lefty Leverenz commented on HIVE-13449:
---

I left an editorial comment on RB for the description of 
*hive.llap.create.token.locally*.

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-05 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273187#comment-15273187
 ] 

Sergey Shelukhin commented on HIVE-13449:
-

[~hagleitn] [~jdere] [~vikram.dixit] uploaded a new patch; this refactors some 
code around the tokens for Tez session, so I'd like to commit it before some 
other security patches that would otherwise conflict a lot.
SecretManager was moved, and LlapSecurityHelper was also mostly moved to 
LlapTokenClientFactory; the rest are new changes. The RB link is attached.


> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.WIP.patch, 
> HIVE-13449.02.patch, HIVE-13449.03.patch, HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-05-04 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271706#comment-15271706
 ] 

Sergey Shelukhin commented on HIVE-13449:
-

[~hagleitn] [~vikram.dixit] ping?

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.01.patch, HIVE-13449.02.patch, 
> HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-04-24 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255709#comment-15255709
 ] 

Hive QA commented on HIVE-13449:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12800357/HIVE-13449.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 39 failed/errored test(s), 9900 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-enforce_order.q-vector_partition_diff_num_cols.q-unionDistinct_1.q-and-12-more
 - did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_distinct_2.q-tez_joins_explain.q-cte_mat_1.q-and-12-more
 - did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_varchar_4.q-smb_cache.q-tez_join_hash.q-and-8-more 
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorized_parquet.q-vector_decimal_aggregate.q-tez_self_join.q-and-12-more
 - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityPreemption
org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote.org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testAddPartitions
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testFetchingPartitionsWithDifferentSchemas
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hadoop.hive.metastore.TestMetaStoreEndFunctionListener.testEndFunctionListener
org.apache.hadoop.hive.metastore.TestMetaStoreEventListenerOnlyOnCommit.testEventStatus
org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.testMetaStoreInitListener
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAppendPartitionWithCommas
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAppendPartitionWithValidCharacters
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleTable
org.apache.hadoop.hive.ql.security.TestClientSideAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestExtendedAcls.org.apache.hadoop.hive.ql.security.TestExtendedAcls
org.apache.hadoop.hive.ql.security.TestFolderPermissions.org.apache.hadoop.hive.ql.security.TestFolderPermissions
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener.org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropDatabase
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropTable
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropView
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadDbSuccess
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableFailure
org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testSaslWithHiveMetaStore
org.apache.hive.beeline.TestSchemaTool.testSchemaInit
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.org.apache.hive.minikdc.TestJdbcWithDBTokenStore
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.org.apache.hive.service.TestHS2ImpersonationWithRemoteMS
{noformat}

Test results: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/63/testReport
Console output: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/63/console
Test logs: 
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-63/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests ex

[jira] [Commented] (HIVE-13449) LLAP: HS2 should get the token directly, rather than from LLAP

2016-04-22 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254439#comment-15254439
 ] 

Sergey Shelukhin commented on HIVE-13449:
-

[~vikram.dixit] [~hagleitn] perhaps you can review this

> LLAP: HS2 should get the token directly, rather than from LLAP
> --
>
> Key: HIVE-13449
> URL: https://issues.apache.org/jira/browse/HIVE-13449
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13449.patch
>
>
> HS2 doesn't need a roundtrip to LLAP; it can instantiate the SecretManager 
> directly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)