Thanks for the reply Kyle! I will skip streaming tests for the Spark runner for now.
After *disabling* the *GameStatsIT* test, these are the results. At first sight it looks like all three runners successfully execute some or all tests, but upon closer inspection I realized those were just false positives. Flink LeaderBoardIT: OutOfMemoryError: GC overhead limit exceeded StatefulTeamScoreIT: OutOfMemoryError: GC overhead limit exceeded Would increasing the memory of the jenkins runner help? HourlyTeamScoreIT and UserScoreIT successful execution is a false positive. Logs get truncated but I see a message about setting the fasterCopy option for better performance. Will try that and let you know my findings. Direct All tests failing in Jenkins but for some reason they get reported as successful. Looks like something fails but the log gets truncated. Locally all run successfully. Dataflow StatefulTeamScoreIT, LeaderBoardIT, are failing but the log file gets truncated. HourlyTeamScoreIT and UserScoreIT are successful. On Mon, Mar 21, 2022 at 9:50 AM Kyle Weaver <[email protected]> wrote: > These examples are all streaming pipelines, so I'm not surprised the Spark > runner has issues there. I would describe the Spark runner's streaming mode > as sketchy at best. > > Not sure what is happening with other runners. Your latest email summary > doesn't seem to match the test results on > https://github.com/apache/beam/pull/17015. > > On Fri, Mar 18, 2022 at 8:17 PM Ahmet Altay <[email protected]> wrote: > >> Adding @Kyle Weaver <[email protected]> and @Kenneth Knowles >> <[email protected]> - they might have some idea. >> >> On Fri, Mar 18, 2022 at 2:48 PM Fer Morales Martinez < >> [email protected]> wrote: >> >>> Hello everyone >>> >>> These are my latest findings. If anyone has a clue as to what's >>> happening, any help would be greatly appreciated. >>> >>> Spark >>> >>> *StatefulTeamScoreIT* >>> >>> According to the compatibility matrix [1], stateful processing is not >>> yet supported hence the error. Will sickbay this test and wait for that to >>> be implemented. >>> >>> *LeaderBoardIT* >>> >>> Looks like PubSubClient is being shutdown before reading can complete. >>> >>> *GameStatsIT* >>> >>> No TransformEvaluator registered for UNBOUNDED transform >>> View.CreatePCollectionView. >>> >>> Flink >>> >>> Runner dies after two hours, possibly due to GameStats example. Will >>> disable that test in order to complete the other ones. >>> >>> Direct >>> >>> All tests failing in Jenkins. Locally, only GameStats fails. >>> >>> Dataflow >>> >>> Contrary to the previous email, StatefulTeamScoreIT, LeaderBoardIT, >>> GameStatsIT are failing but the log file gets truncated so no further >>> details here at the moment. >>> >>> HourlyTeamScoreIT and UserScoreIT are successful. >>> >>> >>> Again, PR is here [2] in case you want to take a look at how the >>> integration tests scaffolding is being made. >>> >>> >>> Thanks! >>> >>> >>> [1] >>> https://beam.apache.org/documentation/runners/capability-matrix/what-is-being-computed/ >>> >>> [2] https://github.com/apache/beam/pull/17015/files >>> >>> >>> On Wed, Mar 16, 2022 at 4:03 PM Fer Morales Martinez < >>> [email protected]> wrote: >>> >>>> Hi team! >>>> >>>> I've been implementing integration tests [1] for the examples under the >>>> beam/examples/complete/game folder. >>>> Most of the integration tests (the one for *GameStats* being the only >>>> outlier so far) run successfully when executed *locally* with the >>>> *Direct* runner. >>>> However, I've found that when executed against the Spark, Flink and >>>> Direct runners in jenkins, they fail with a different error depending on >>>> the runner. >>>> Direct runner on jenkins plays mostly well [2] with all the ITs; again, >>>> *GameStats* being the only one that for some reason gets stuck when >>>> trying to get a fixedWindow [3]. >>>> With Spark [4] three tests fail, two of those being executed >>>> successfully by the Direct runner. >>>> Flink [5] gives up after two hours executing, possibly due to the same >>>> *GameStats* issue. >>>> Fortunately, DataFlow executes all the ITs successfully [6], even the >>>> *GameStats* one. >>>> >>>> I guess my question would be, what could be improved integration >>>> testing-wise? How would you pinpoint the root cause in this kind of >>>> scenario? >>>> >>>> Thanks! >>>> >>>> - Fer >>>> >>>> >>>> [1] https://github.com/apache/beam/pull/17015/files >>>> [2] >>>> https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/9/testReport/ >>>> [3] >>>> https://github.com/apache/beam/blob/3e6e8a7e2afa08552849518439577eee07700f77/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java#L292 >>>> [4] >>>> https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Spark/7/testReport/ >>>> [5] >>>> https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Flink/7/ >>>> [6] >>>> https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_PR/59/testReport/ >>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *This email and its contents (including any attachments) are being sent >>> toyou on the condition of confidentiality and may be protected by >>> legalprivilege. Access to this email by anyone other than the intended >>> recipientis unauthorized. If you are not the intended recipient, please >>> immediatelynotify the sender by replying to this message and delete the >>> materialimmediately from your system. Any further use, dissemination, >>> distributionor reproduction of this email is strictly prohibited. Further, >>> norepresentation is made with respect to any content contained in this >>> email.* >> >> -- *This email and its contents (including any attachments) are being sent to you on the condition of confidentiality and may be protected by legal privilege. Access to this email by anyone other than the intended recipient is unauthorized. If you are not the intended recipient, please immediately notify the sender by replying to this message and delete the material immediately from your system. Any further use, dissemination, distribution or reproduction of this email is strictly prohibited. Further, no representation is made with respect to any content contained in this email.*
