[ https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014436#comment-17014436 ]
Hadoop QA commented on OOZIE-3575: ---------------------------------- Testing JIRA OOZIE-3575 Cleaning local git workspace ---------------------------- {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} . {color:green}+1{color} the patch does not introduce any @author tags . {color:green}+1{color} the patch does not introduce any tabs . {color:green}+1{color} the patch does not introduce any trailing spaces . {color:green}+1{color} the patch does not introduce any star imports . {color:green}+1{color} the patch does not introduce any line longer than 132 . {color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} . {color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} . {color:green}+1{color} Javadoc generation succeeded with the patch . {color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) {color:green}+1 COMPILE{color} . {color:green}+1{color} HEAD compiles . {color:green}+1{color} patch compiles . {color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [5] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:red}-1{color} There are [5] new bugs found below threshold in [core] that must be fixed. . You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html . The most important SpotBugs errors are: . At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection . At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] . At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] . This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206] . At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127] . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/git]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} . {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . {color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} . Tests run: 3206 . {color:orange}Tests failed at first run:{color} TestCoordActionsKillXCommand#testActionKillCommandDate TestCoordChangeXCommand#testCoord_throwException TestBlockingInputStream#testFastWritingBlockingInputStream . For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} . {color:green}+1{color} distro tarball builds with the patch {color:green}+1 MODERNIZER{color} ---------------------------- {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1288/ > Add credential support for cloud file systems > --------------------------------------------- > > Key: OOZIE-3575 > URL: https://issues.apache.org/jira/browse/OOZIE-3575 > Project: Oozie > Issue Type: Sub-task > Components: core > Affects Versions: 5.2.0 > Reporter: Mate Juhasz > Assignee: Mate Juhasz > Priority: Major > Fix For: trunk > > Attachments: OOZIE-3575-v2.patch, OOZIE-3575-v3.patch, > OOZIE-3575.patch > > > Oozie by default gathers delegation tokens for the nodes defined in > _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_ > in case of distcp actions) and for the workflow path. > Though this implementation is good for hdfs, we dont support occasions where > the job related resources, which we want to access in runtime are present on > different file systems/buckets etc... > The HDFSCredentials class should be revised to handle getting tokens for > different cloud storages. > *The following scenarios should be addressed:* > Oozie should obtain delegation token in case > * the defaultFs is cloud > * the workload.xml is in cloud > * input/output/auxiliary files referred from workflow are in cloud > * (newly introduced feature) user could define filesystem credentials for the > workflow (as its done with hive/hcat etc..) -> this would allow the user to > handle the situation where Oozie could not decide which tokens are needed at > launch time by default and could also get tokens for different cloud storages > and buckets as well > Example for credentials addition: > {noformat} > <credential name="aws_auth" type="filesystem"> > <property> > <name>filesystem</name> > <value>s3a://qe-s3-bucket-mst</value> > </property> > </credential> > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)