[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012763#comment-17012763
]
Hadoop QA commented on OOZIE-3575:
----------------------------------
Testing JIRA OOZIE-3575
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [6] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:red}-1{color} There are [5] new bugs found below threshold in
[core] that must be fixed.
. You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important SpotBugs errors are:
. At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
. At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
. At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
. This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
. At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/oozie] that must be fixed.
. You can find the SpotBugs diff here (look for the red and orange ones):
sharelib/oozie/findbugs-new.html
. The most important SpotBugs errors are:
. At ShellMain.java:[line 93]: This usage of
java/lang/ProcessBuilder.<init>(Ljava/util/List;)V can be vulnerable to
Command Injection
. At ShellMain.java:[line 91]: At ShellMain.java:[line 90]
. At ShellMain.java:[line 92]
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3206
. {color:orange}Tests failed at first run:{color}
124654-[WARNING]
TestCredentials#testThrowErrorWhenMissingFileSystemPathProperty
TestCredentials#testThrowErrorWhenMissingFileSystemPathProperty
TestBlockingInputStream#testLimitedWritingBlockingInputStream
. For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1286/
> Add credential support for cloud file systems
> ---------------------------------------------
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
> Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
> Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
> <credential name="aws_auth" type="filesystem">
> <property>
> <name>filesystem</name>
> <value>s3a://qe-s3-bucket-mst</value>
> </property>
> </credential>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
