+1 http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201610.mbox/%3CCAFFRZHX4yq%3D%3DxuvkPjwDFezVhWH82oj%2BgpS-OhUMc%3D3QUVaS1g%40mail.gmail.com%3E
On Fri, Jun 23, 2017 at 9:23 AM, Davor Bonaci <da...@apache.org> wrote: > This would be a great contribution if anyone wants to give it a try. > > On Thu, Jun 22, 2017 at 9:23 PM, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > > > Hi Eugene > > > > I like the idea ! > > > > Regards > > JB > > > > > > On 06/23/2017 12:27 AM, Eugene Kirpichov wrote: > > > >> Hi folks and especially runner developers, > >> > >> https://issues.apache.org/jira/browse/BEAM-2506 - quoting from there: > >> > >> Currently ValidatesRunner test suites run 1 pipeline per unit test. > That's > >> a lot of small pipelines, and consumes a lot of resources especially in > >> case of a pretty heavyweight runner like Dataflow, so tests take a long > >> time and can't be run in parallel due to quota issues, etc. > >> > >> Jason Kuster says he and Davor Bonaci discussed that we could execute > >> multiple unit tests in a single TestPipeline. > >> > >> To further develop it: in case of Java, we could create a custom JUnit > >> Runner http://junit.org/junit4/javadoc/4.12/org/junit/runner/ > Runner.html > >> that would apply all the transforms and PAsserts in unit tests to a > single > >> instance of TestPipeline (per class, rather than per method), and run > the > >> whole thing at the end. PAssert captures the source location of its > >> application, so we could still report which particular test failed. > >> > >> This obviously has fewer isolation between unit test methods, cause they > >> effectively run in parallel instead of in sequence, so things like > >> per-method setup and teardown will no longer be applicable. There'll > >> probably be other issues. > >> > >> Anyway, this seems doable and high-impact. > >> > >> Just bringing this to the attention of the community - it seems worth > >> discussing and perhaps someone will be interested in developing this > idea > >> further or implementing it. > >> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > >
