[
https://issues.apache.org/jira/browse/RANGER-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kwangwoo Kim updated RANGER-341:
--------------------------------
Description:
If hiveql is too long (exceeds 2000 characters), the access audit is not
inserted into xa_access_audit table properly. In fact, it rarely happens in
ordinary usage of Hive but sometimes when making a partitioned table, which has
many columns, or when query string has a lot of spaces, the issue would occur.
One of the hdfs side access audit example is as below:
{code:json}
{"resource":"foo/bar","resType":"@table","reqData":"create table if not exists
foo.bar \n\t\t ( \n\t\t\tsday
string\n\t\t , \n\t\t\tsday_cd
string\n\t\t , \n\t\t\tsday_nm
string\n\t\t , \n\t\t\tmonth_cd
string\n\t\t , \n\t\t\tmonth_wk
string\n\t\t , \n\t\t\tyear_wk
string\n\t\t , \n\t\t\tyear_wk_ori
string\n\t\t , \n\t\t\tquater_cd
string\n\t\t , \n\t\t\thalf_cd
string\n\t\t , \n\t\t\tyear_cd
string\n\t\t , \n\t\t\tyear_nm
string\n\t\t , \n\t\t\thalf_nm
string\n\t\t , \n\t\t\tquater_nm
string\n\t\t , \n\t\t\tyear_wk_nm
string\n\t\t , \n\t\t\tmonth_nm
string\n\t\t , \n\t\t\tmonth_wk_nm
string\n\t\t , \n\t\t\tday_cd
string\n\t\t , \n\t\t\tday_nm
string\n\t\t , \n\t\t\tweek_nm
string\n\t\t , \n\t\t\tweek_cd
string\n\t\t , \n\t\t\tholiday_yn
string\n\t\t , \n\t\t\tholiday_desc
string\n\t\t , \n\t\t\tworking_day
int\n\t\t , \n\t\t\tpace_day
double\n\t\t , \n\t\t\tpace_month_acc
double\n\t\t , \n\t\t\tpace_year_acc
double\n\t\t , \n\t\t\tpace_month_tot
double\n\t\t , \n\t\t\tpace_year_tot
double\n\t\t , \n\t\t\tevent_yn
string\n\t\t , \n\t\t\tevent_desc
string\n\t\t , \n\t\t\tload_dt
string\n\t\t ) \n\t\t \n\t\t \n\t\t \n\t\tROW FORMAT
SERDE\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcSerde\u0027\n\t\tSTORED
AS
INPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcInputFormat\u0027\n\t\tOUTPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat\u0027\n\t\t
\n\t\t \n\t\t \n\t\tTBLPROPERTIES (\n\t\t\t\u0027orc.compress\u0027 \u003d
\u0027SNAPPY\u0027\n\t\t)","reqUser":"xxxx","evtTime":"2015-03-25
13:35:55.802","policy":0,"access":"CREATE","result":1,"enforcer":"xasecure-acl","repoType":3,"repo":"hive_base_repo","sess":"10086e96-e8fd-467f-8161-3cd7497e7476","cliType":"HIVESERVER2","cliIP":"000.000.000.000","action":"CREATETABLE","agentHost":"hostname.domain.net","logType":"RangerAudit","id":"3676d57f-8d70-40eb-b6ab-d9b4fcdfd923"}
{code}
was:
If hiveql is too long (exceeds 2000 characters), the access audit is not
inserted into xa_access_audit table properly. In fact, it rarely happens in
ordinary usage of Hive but sometimes when making a partitioned table, which has
many columns, or when query string has a lot of spaces, the issue would occur.
One of the hdfs side access audit example is as below:
{"resource":"foo/bar","resType":"@table","reqData":"create table if not exists
foo.bar \n\t\t ( \n\t\t\tsday
string\n\t\t , \n\t\t\tsday_cd
string\n\t\t , \n\t\t\tsday_nm
string\n\t\t , \n\t\t\tmonth_cd
string\n\t\t , \n\t\t\tmonth_wk
string\n\t\t , \n\t\t\tyear_wk
string\n\t\t , \n\t\t\tyear_wk_ori
string\n\t\t , \n\t\t\tquater_cd
string\n\t\t , \n\t\t\thalf_cd
string\n\t\t , \n\t\t\tyear_cd
string\n\t\t , \n\t\t\tyear_nm
string\n\t\t , \n\t\t\thalf_nm
string\n\t\t , \n\t\t\tquater_nm
string\n\t\t , \n\t\t\tyear_wk_nm
string\n\t\t , \n\t\t\tmonth_nm
string\n\t\t , \n\t\t\tmonth_wk_nm
string\n\t\t , \n\t\t\tday_cd
string\n\t\t , \n\t\t\tday_nm
string\n\t\t , \n\t\t\tweek_nm
string\n\t\t , \n\t\t\tweek_cd
string\n\t\t , \n\t\t\tholiday_yn
string\n\t\t , \n\t\t\tholiday_desc
string\n\t\t , \n\t\t\tworking_day
int\n\t\t , \n\t\t\tpace_day
double\n\t\t , \n\t\t\tpace_month_acc
double\n\t\t , \n\t\t\tpace_year_acc
double\n\t\t , \n\t\t\tpace_month_tot
double\n\t\t , \n\t\t\tpace_year_tot
double\n\t\t , \n\t\t\tevent_yn
string\n\t\t , \n\t\t\tevent_desc
string\n\t\t , \n\t\t\tload_dt
string\n\t\t ) \n\t\t \n\t\t \n\t\t \n\t\tROW FORMAT
SERDE\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcSerde\u0027\n\t\tSTORED
AS
INPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcInputFormat\u0027\n\t\tOUTPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat\u0027\n\t\t
\n\t\t \n\t\t \n\t\tTBLPROPERTIES (\n\t\t\t\u0027orc.compress\u0027 \u003d
\u0027SNAPPY\u0027\n\t\t)","reqUser":"xxxx","evtTime":"2015-03-25
13:35:55.802","policy":0,"access":"CREATE","result":1,"enforcer":"xasecure-acl","repoType":3,"repo":"hive_base_repo","sess":"10086e96-e8fd-467f-8161-3cd7497e7476","cliType":"HIVESERVER2","cliIP":"000.000.000.000","action":"CREATETABLE","agentHost":"hostname.domain.net","logType":"RangerAudit","id":"3676d57f-8d70-40eb-b6ab-d9b4fcdfd923"}
> request_data and request_path columns are short
> -----------------------------------------------
>
> Key: RANGER-341
> URL: https://issues.apache.org/jira/browse/RANGER-341
> Project: Ranger
> Issue Type: Bug
> Components: admin
> Affects Versions: 0.4.0, 0.5.0, 0.4.1
> Reporter: Kwangwoo Kim
> Priority: Minor
>
> If hiveql is too long (exceeds 2000 characters), the access audit is not
> inserted into xa_access_audit table properly. In fact, it rarely happens in
> ordinary usage of Hive but sometimes when making a partitioned table, which
> has many columns, or when query string has a lot of spaces, the issue would
> occur.
> One of the hdfs side access audit example is as below:
> {code:json}
> {"resource":"foo/bar","resType":"@table","reqData":"create table if not
> exists foo.bar \n\t\t ( \n\t\t\tsday
>
> string\n\t\t , \n\t\t\tsday_cd
>
> string\n\t\t , \n\t\t\tsday_nm
>
> string\n\t\t , \n\t\t\tmonth_cd
>
> string\n\t\t , \n\t\t\tmonth_wk
>
> string\n\t\t , \n\t\t\tyear_wk
>
> string\n\t\t , \n\t\t\tyear_wk_ori
>
> string\n\t\t , \n\t\t\tquater_cd
>
> string\n\t\t , \n\t\t\thalf_cd
>
> string\n\t\t , \n\t\t\tyear_cd
>
> string\n\t\t , \n\t\t\tyear_nm
>
> string\n\t\t , \n\t\t\thalf_nm
>
> string\n\t\t , \n\t\t\tquater_nm
>
> string\n\t\t , \n\t\t\tyear_wk_nm
>
> string\n\t\t , \n\t\t\tmonth_nm
>
> string\n\t\t , \n\t\t\tmonth_wk_nm
>
> string\n\t\t , \n\t\t\tday_cd
>
> string\n\t\t , \n\t\t\tday_nm
>
> string\n\t\t , \n\t\t\tweek_nm
>
> string\n\t\t , \n\t\t\tweek_cd
>
> string\n\t\t , \n\t\t\tholiday_yn
>
> string\n\t\t , \n\t\t\tholiday_desc
>
> string\n\t\t , \n\t\t\tworking_day
>
> int\n\t\t , \n\t\t\tpace_day
>
> double\n\t\t , \n\t\t\tpace_month_acc
>
> double\n\t\t , \n\t\t\tpace_year_acc
>
> double\n\t\t , \n\t\t\tpace_month_tot
>
> double\n\t\t , \n\t\t\tpace_year_tot
>
> double\n\t\t , \n\t\t\tevent_yn
>
> string\n\t\t , \n\t\t\tevent_desc
>
> string\n\t\t , \n\t\t\tload_dt
>
> string\n\t\t ) \n\t\t \n\t\t \n\t\t \n\t\tROW FORMAT
> SERDE\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcSerde\u0027\n\t\tSTORED
> AS
> INPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcInputFormat\u0027\n\t\tOUTPUTFORMAT\n\t\t\t\u0027org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat\u0027\n\t\t
> \n\t\t \n\t\t \n\t\tTBLPROPERTIES (\n\t\t\t\u0027orc.compress\u0027 \u003d
> \u0027SNAPPY\u0027\n\t\t)","reqUser":"xxxx","evtTime":"2015-03-25
> 13:35:55.802","policy":0,"access":"CREATE","result":1,"enforcer":"xasecure-acl","repoType":3,"repo":"hive_base_repo","sess":"10086e96-e8fd-467f-8161-3cd7497e7476","cliType":"HIVESERVER2","cliIP":"000.000.000.000","action":"CREATETABLE","agentHost":"hostname.domain.net","logType":"RangerAudit","id":"3676d57f-8d70-40eb-b6ab-d9b4fcdfd923"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
