[ https://issues.apache.org/jira/browse/SENTRY-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339478#comment-16339478 ]
Na Li commented on SENTRY-2127: ------------------------------- The actual problem is MapRedTask sometimes fail with more complicated join statement. It usually passes when I ran it locally, but may fail when I ran it in build machine. So it seems to be performance issue for map reduce. I have checked log and all sentry authorization finished successfully. So this issue is not related to sentry, and should not cause sentry unit test to fail. > Fix unstable unit test TestColumnEndToEnd.testCrossDbTableOperations > -------------------------------------------------------------------- > > Key: SENTRY-2127 > URL: https://issues.apache.org/jira/browse/SENTRY-2127 > Project: Sentry > Issue Type: Bug > Components: Sentry > Affects Versions: 2.1.0 > Reporter: Na Li > Assignee: Na Li > Priority: Major > Attachments: SENTRY-2127.001.patch > > > Occasionally, the test TestColumnEndToEnd.testCrossDbTableOperations fails at > statement > statement.execute("CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num > from db_1.tb1,db_2.tb3,db_2.tb2"); > We should make it stable to avoid blocking other updates. > More info: > 1) Failed run with the following statement. The reason it fails is not > related to sentry processing. It is due to the map reduce task fails with the > more complicated join in the unit test > {code:java} > 2018-01-23 19:46:28,766 (5c4ce662-7587-40a5-8b24-b4c31f006b53 > HiveServer2-Handler-Pool: Thread-365) [INFO - > org.apache.hadoop.hive.ql.Driver.compile(Driver.java:460)] Compiling > command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c): > CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num from > db_1.tb1,db_2.tb3,db_2.tb2 > ... > 2018-01-23 19:46:29,266 (HiveServer2-Background-Pool: Thread-373) [INFO - > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1713)] Executing > command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c): > CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num from > db_1.tb1,db_2.tb3,db_2.tb2 > ... > 2018-01-23 19:49:29,229 (HiveServer2-Background-Pool: Thread-373) [INFO - > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2021)] Completed > executing > command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c); > Time taken: 179.962 seconds > 2018-01-23 19:49:29,235 (HiveServer2-Background-Pool: Thread-373) [ERROR - > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:345)] > Error running hive query: > org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.mr.MapRedTask > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:330) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:254) > at > org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:91) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:342) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:354) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > 2) Success run with following statement in unit test > {code:java} > statement.execute("CREATE table db_1.t2 as select tb1.id from db_1.tb1"); > 2018-01-24 10:29:01,071 (c042d8d8-423b-4ddf-a50b-63f1dd3e3f50 > HiveServer2-Handler-Pool: Thread-365) [INFO - > org.apache.hadoop.hive.ql.Driver.compile(Driver.java:460)] Compiling > command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94): > CREATE table db_1.t2 as select tb1.id from db_1.tb1 > 2018-01-24 10:29:01,513 (HiveServer2-Background-Pool: Thread-372) [INFO - > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1713)] Executing > command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94): > CREATE table db_1.t2 as select tb1.id from db_1.tb1 > 2018-01-24 10:29:03,636 (HiveServer2-Background-Pool: Thread-372) [INFO - > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2021)] Completed > executing > command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94); > Time taken: 2.122 seconds > OK{code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)