[
https://issues.apache.org/jira/browse/OOZIE-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601808#comment-17601808
]
Hadoop QA commented on OOZIE-3254:
----------------------------------
Testing JIRA OOZIE-3254
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:green}+1{color} the patch adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [6] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
. You can find the SpotBugs diff here: server/findbugs-new.html
. {color:red}-1{color} There are [6] new bugs found below threshold in
[core] that must be fixed, listing only the first [5] ones.
. You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The top [5] most important SpotBugs errors are:
. At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
. At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
. At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
. This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
. At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3224
. {color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommandNonUTC>TestCoordActionInputCheckXCommand#testNone
TestBlockingInputStream#testFastWritingBlockingInputStream
TestBlockingInputStream#testLimitedWritingBlockingInputStream
. For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://ci-hadoop.apache.org/job/PreCommit-OOZIE-Build/96/
> [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError
> when there are too many coordinator actions to materialize
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-3254
> URL: https://issues.apache.org/jira/browse/OOZIE-3254
> Project: Oozie
> Issue Type: Bug
> Components: coordinator
> Affects Versions: 5.0.0
> Reporter: Andras Piros
> Assignee: Janos Makai
> Priority: Major
> Attachments: OOZIE-3254-002.patch, OOZIE-3254-01-wip.patch
>
>
> If there is a coordinator job defined with a {{frequency}} by the minute
> (e.g. {{frequency="* * * * *"}}), and {{start-time}} lies well in the past,
> and the coordinator job's {{execution-mode}} is {{LAST_ONLY}} or {{NONE}}, it
> can happen that too many {{CoordinatorActionBean}} instances are kept on JVM
> heap within {{CoordMaterializeTransitionXCommand#insertList}} as those
> execution modes [*omit the check for the {{throttle}}
> value*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java#L439-L443].
> As a consequence, we can see as many as multiple hundred thousands of log
> entries [*trying to increase
> {{CoordMaterializeTransitionXCommand#insertList}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java#L560-L566]:
> {noformat}
> [user@host ~]$ grep 'In storeToDB() coord action id'
> /var/log/oozie/oozie-HOSTNAME.log.out | wc -l
> 478408
> {noformat}
> A much worse consequence is that those {{CoordinatorActionBean}} instances
> are attached to GC root (the {{insertList}} itself), and thus, JVM is unable
> to free them until a consequent call to {{insertList.clear()}}. This will
> result in {{OutOfMemoryError}} occurrence in worst case.
> {{CoordMaterializeTransitionXCommand#insertList}} should be watched for a
> configurable limit parameter (default value something like 1000), and
> persisted / cleared when that limit is reached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
