[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[
https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810954#comment-16810954
]
Hadoop QA commented on OOZIE-3328:
--
Testing JIRA OOZIE-3328
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:red}-1 JAVADOC{color}
.{color:red}-1{color} build with Javadoc generation fails with the patch
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [8] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [8] new bugs found below threshold in
[core] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At HiveCredentials.java:[line 98]: Exception is caught when Exception is
not thrown in
org.apache.oozie.action.hadoop.HiveCredentials.updateCredentials(Credentials,
Configuration, CredentialsProperties, ActionExecutor$Context)
.At BulkJPAExecutor.java:[line 207]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 177]: At BulkJPAExecutor.java:[line 176]
.At BulkJPAExecutor.java:[line 206]: At BulkJPAExecutor.java:[line 200]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 207]
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run test cases
.{color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommand#testNone
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1076/
> Create Hive compatibility action executor to run hive actions using beeline
> ---
>
> Key: OOZIE-3328
> URL: https://issues.apache.org/jira/browse/OOZIE-3328
> Project: Oozie
> Issue Type: Task
> Components: action, core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Labels: features,
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[ https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810850#comment-16810850 ] Hadoop QA commented on OOZIE-3328: -- PreCommit-OOZIE-Build started > Create Hive compatibility action executor to run hive actions using beeline > --- > > Key: OOZIE-3328 > URL: https://issues.apache.org/jira/browse/OOZIE-3328 > Project: Oozie > Issue Type: Task > Components: action, core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Labels: features, usability > Attachments: OOZIE-3328.001.patch > > > If I am correct then Hive will not support HiveCli for long and Oozie may > have to handle this. > A new executor shall be created which can understand the original hive action > format while this executor shall run the action using beeline. > What are your thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[ https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810778#comment-16810778 ] Denes Bodo commented on OOZIE-3328: --- The suggested implementation needs Hive3 as dependency due to the used beeline-site.xml location. I am uploading the diff here and to [RB|https://reviews.apache.org/r/70406/]. > Create Hive compatibility action executor to run hive actions using beeline > --- > > Key: OOZIE-3328 > URL: https://issues.apache.org/jira/browse/OOZIE-3328 > Project: Oozie > Issue Type: Task > Components: action, core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Labels: features, usability > > If I am correct then Hive will not support HiveCli for long and Oozie may > have to handle this. > A new executor shall be created which can understand the original hive action > format while this executor shall run the action using beeline. > What are your thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[ https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809911#comment-16809911 ] Zsombor Gegesy commented on OOZIE-3328: --- The problem is that there are users with hundreds of workflows which use 'hive' action. But, with Hive 3, the HiveCLI is not supported anymore - so if Oozie still want to run as-is - without forcing every user to rewrite their workflow, it needs a translation layer, which accepts the same parameters and flags as the original hive action, but calls the Beeline tool under the hood. AFAIK, the big difference between HiveCLI and Beeline is the different delegation tokens, the use of the beeline-site.xml, and the different JDBC URL handling. > Create Hive compatibility action executor to run hive actions using beeline > --- > > Key: OOZIE-3328 > URL: https://issues.apache.org/jira/browse/OOZIE-3328 > Project: Oozie > Issue Type: Task > Components: action, core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Labels: features, usability > > If I am correct then Hive will not support HiveCli for long and Oozie may > have to handle this. > A new executor shall be created which can understand the original hive action > format while this executor shall run the action using beeline. > What are your thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[ https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579885#comment-16579885 ] Peter Cseh commented on OOZIE-3328: --- What would be the point of that? If I know correctly there are incompatibilities between the two command line tools. The first one that comes to mind is that Beeline uses jdbc url while HiveCLI uses the thrift uri to the HMS. Or would it be enough to simply change [this line|https://github.com/apache/oozie/blob/master/sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java#L310] to call beeline? > Create Hive compatibility action executor to run hive actions using beeline > --- > > Key: OOZIE-3328 > URL: https://issues.apache.org/jira/browse/OOZIE-3328 > Project: Oozie > Issue Type: Task > Components: action, core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Labels: features, usability > > If I am correct then Hive will not support HiveCli for long and Oozie may > have to handle this. > A new executor shall be created which can understand the original hive action > format while this executor shall run the action using beeline. > What are your thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[ https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579814#comment-16579814 ] Denes Bodo commented on OOZIE-3328: --- [~andras.piros] Actually no. I am looking for an action, which understands the (not ) tag in the workflow xml and uses beeline instead of HiveCli. > Create Hive compatibility action executor to run hive actions using beeline > --- > > Key: OOZIE-3328 > URL: https://issues.apache.org/jira/browse/OOZIE-3328 > Project: Oozie > Issue Type: Task > Components: action, core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Labels: features, usability > > If I am correct then Hive will not support HiveCli for long and Oozie may > have to handle this. > A new executor shall be created which can understand the original hive action > format while this executor shall run the action using beeline. > What are your thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3328) Create Hive compatibility action executor to run hive actions using beeline
[
https://issues.apache.org/jira/browse/OOZIE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579808#comment-16579808
]
Andras Piros commented on OOZIE-3328:
-
[~dionusos] actually we do have [*a Hive2
action*|https://oozie.apache.org/docs/5.0.0/DG_Hive2ActionExtension.html] that
[*uses
{{Beeline}}*|https://github.com/apache/oozie/blob/master/sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java#L264-L273].
Is this you're looking for?
> Create Hive compatibility action executor to run hive actions using beeline
> ---
>
> Key: OOZIE-3328
> URL: https://issues.apache.org/jira/browse/OOZIE-3328
> Project: Oozie
> Issue Type: Task
> Components: action, core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Labels: features, usability
>
> If I am correct then Hive will not support HiveCli for long and Oozie may
> have to handle this.
> A new executor shall be created which can understand the original hive action
> format while this executor shall run the action using beeline.
> What are your thoughts?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
