Re: Question about integration tests for the examples classes

2022-03-23 Thread Fer Morales Martinez
Looks like the only runner left to make work is the Flink one. Let's take the LeaderBoardIT [1] test as an example; when calling the runLeaderBoard method from LeaderBoard all the reading, processing and writing is done successfully. I can see the BigQuery dataset and table get created and populate

Re: Question about integration tests for the examples classes

2022-03-22 Thread Fer Morales Martinez
Great! I will change the log retention policy just to double check they are passing. Thanks again, Kyle! On Tue, Mar 22, 2022 at 12:51 PM Kyle Weaver wrote: > Those log messages are only printed by the Dataflow runner, so we > shouldn't expect to see them printed by other runners. > https://git

Re: Question about integration tests for the examples classes

2022-03-22 Thread Fer Morales Martinez
For some tests [1], [2] I see something along the lines of: INFO: Job 2022-03-17_09_56_46-14482025605245680862 finished with status DONE. Mar 17, 2022 5:02:44 PM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2022-03-17_09_56_46-144

Re: Question about integration tests for the examples classes

2022-03-22 Thread Fer Morales Martinez
Totally! Flink https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Flink/8/testReport/org.apache.beam.examples.complete.game/ Direct https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/10/testReport/org.apache.beam.examples.complete.game/ Dataflow https://ci-beam.apac

Re: Question about integration tests for the examples classes

2022-03-22 Thread Fer Morales Martinez
Thanks for the reply Kyle! I will skip streaming tests for the Spark runner for now. After *disabling* the *GameStatsIT* test, these are the results. At first sight it looks like all three runners successfully execute some or all tests, but upon closer inspection I realized those were just false

Re: Question about integration tests for the examples classes

2022-03-18 Thread Fer Morales Martinez
Hello everyone These are my latest findings. If anyone has a clue as to what's happening, any help would be greatly appreciated. Spark *StatefulTeamScoreIT* According to the compatibility matrix [1], stateful processing is not yet supported hence the error. Will sickbay this test and wait for t

Question about integration tests for the examples classes

2022-03-16 Thread Fer Morales Martinez
Hi team! I've been implementing integration tests [1] for the examples under the beam/examples/complete/game folder. Most of the integration tests (the one for *GameStats* being the only outlier so far) run successfully when executed *locally* with the *Direct* runner. However, I've found that whe