[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS

2016-10-20 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated HIVE-14482:

Attachment: Screen Shot 2016-10-21 at 1.06.33 pm.png

The error seems to be unrelated and there is no existing test coverage for HMS 
audit logs, also it is pretty hard to test it.

Please advise if the patch can be accepted.

Thanks

> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Attachments: HIVE-14482.2.patch, HIVE-14482.patch, Screen Shot 
> 2016-10-21 at 1.06.33 pm.png
>
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,094 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154094 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr : 
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr 
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  start=1470723154095 end=1470723154112 duration=17 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154186 duration=14 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,199 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154199 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO  org.apache.hadoop.hive.metastore.ObjectStore: 
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported 
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache: 
> [pool-4-thread-2]: Could not find uri with key 
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: dropPartition() will move partition-directories to 
> trash-directory.
> 2016-08-08 23:12:34,239 INFO  hive.metastore.hivemetastoressimpl: 
> [pool-4-thread-2]: deleting  
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO

[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS

2016-10-20 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated HIVE-14482:

Attachment: HIVE-14482.2.patch

> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Attachments: HIVE-14482.2.patch, HIVE-14482.patch
>
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,094 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154094 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr : 
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr 
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  start=1470723154095 end=1470723154112 duration=17 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154186 duration=14 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,199 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154199 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO  org.apache.hadoop.hive.metastore.ObjectStore: 
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported 
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache: 
> [pool-4-thread-2]: Could not find uri with key 
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: dropPartition() will move partition-directories to 
> trash-directory.
> 2016-08-08 23:12:34,239 INFO  hive.metastore.hivemetastoressimpl: 
> [pool-4-thread-2]: deleting  
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO  org.apache.hadoop.fs.TrashPolicyDefault: 
> [pool-4-thread-2]: Moved: 
> 'hdfs://:8020/user/hive/warehouse/default/test/b=140' to trash at: 
> hdfs://:8020/user/hive/.Trash/Current/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO

[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS

2016-09-07 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated HIVE-14482:

Status: Patch Available  (was: Open)

Added audit log for add, drop and rename partitions

> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Attachments: HIVE-14482.patch
>
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,094 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154094 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr : 
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr 
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  start=1470723154095 end=1470723154112 duration=17 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154186 duration=14 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,199 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154199 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO  org.apache.hadoop.hive.metastore.ObjectStore: 
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported 
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache: 
> [pool-4-thread-2]: Could not find uri with key 
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: dropPartition() will move partition-directories to 
> trash-directory.
> 2016-08-08 23:12:34,239 INFO  hive.metastore.hivemetastoressimpl: 
> [pool-4-thread-2]: deleting  
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO  org.apache.hadoop.fs.TrashPolicyDefault: 
> [pool-4-thread-2]: Moved: 
> 'hdfs://:8020/user/hive/warehouse/default/test/b=140' to trash at: 
> hdfs://:8020/user/hive/.Trash/Current/user/hive/warehouse/default/test/b=140
>

[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS

2016-09-07 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated HIVE-14482:

Attachment: HIVE-14482.patch

> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Attachments: HIVE-14482.patch
>
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,094 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154094 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr : 
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr 
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  start=1470723154095 end=1470723154112 duration=17 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154186 duration=14 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,199 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154199 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO  org.apache.hadoop.hive.metastore.ObjectStore: 
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported 
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache: 
> [pool-4-thread-2]: Could not find uri with key 
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: dropPartition() will move partition-directories to 
> trash-directory.
> 2016-08-08 23:12:34,239 INFO  hive.metastore.hivemetastoressimpl: 
> [pool-4-thread-2]: deleting  
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO  org.apache.hadoop.fs.TrashPolicyDefault: 
> [pool-4-thread-2]: Moved: 
> 'hdfs://:8020/user/hive/warehouse/default/test/b=140' to trash at: 
> hdfs://:8020/user/hive/.Trash/Current/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO

[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS

2016-08-14 Thread Eric Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Lin updated HIVE-14482:

Summary: Add and drop table partition is not audit logged in HMS  (was: 
Drop table partition is not audit logged in HMS)

> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,094 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154094 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr : 
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr 
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  start=1470723154095 end=1470723154112 duration=17 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154186 duration=14 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO  
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: 
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default 
> tbl=test
> 2016-08-08 23:12:34,199 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  end=1470723154199 duration=13 
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: 
> [pool-4-thread-2]:  from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO  org.apache.hadoop.hive.metastore.ObjectStore: 
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported 
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache: 
> [pool-4-thread-2]: Could not find uri with key 
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: 
> [pool-4-thread-2]: dropPartition() will move partition-directories to 
> trash-directory.
> 2016-08-08 23:12:34,239 INFO  hive.metastore.hivemetastoressimpl: 
> [pool-4-thread-2]: deleting  
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO  org.apache.hadoop.fs.TrashPolicyDefault: 
> [pool-4-thread-2]: Moved: 
> 'hdfs://:8020/user/hive/warehouse/default/test/b=140' to trash at: 
> hdfs://:8020/user/hive/.Trash/Current/user/hive/warehouse/default/test/b=140
> 2016-08-08