Re: Testing and the Capability Matrix
@Thomas Completely agree, this is also how it is currently handled in the Flink runner. I was talking about the presentation of the compatibility matrix on the web site, whether we should have separate columns for Flink Stream/Batch and Spark Stream/Batch. (And possibly other runners in the future) On Tue, 14 Jun 2016 at 18:57 Thomas Groh wrote: > It is also worth noting that this document is a snapshot rather than the > long-term plan. As the SDK evolves, the annotations will almost certainly > change with it (and will certainly expand). > > +Aljoscha > > For streaming/batch execution separation, this is better served by > configuration in the runner's build (e.g. specifying two separate > executions in the pom.xml, one for streaming and one for batch). Given that > the tests live in a separate module from the runner, this is likened to how > RunnableOnService tests are currently executed by all of the runners. > > For sink, I think given the current implementations of sink there isn't a > huge need; however, most sinks should be annotated with some form of > superclass (although the implementation of sink requires side inputs, so > this is also worth considering). > > +jb > > These would live on the tests proper, yes. > > On Sun, Jun 12, 2016 at 11:05 PM, Jean-Baptiste Onofré > wrote: > > > Hi Thomas, > > > > it looks good to me. > > > > Just curious: the proposed annotations will be directly in the Java SDK > > Test jar right ? > > > > Thanks, > > Regards > > JB > > > > > > On 06/11/2016 01:34 AM, Thomas Groh wrote: > > > >> Hey Beamers! > >> > >> We have a lovely Capability Matrix ( > >> http://beam.incubator.apache.org/capability-matrix/) which describes > what > >> runners can do, and what's in the model. However, right now we only have > >> one way to specify that a test is useful to be executed in a runner, the > >> RunnableOnService category. > >> > >> I've worked on a document to expand the number of annotations to be more > >> in > >> line with the capability matrix, which should help runner writers test > >> more > >> precisely with regards to the Beam model. The document is located at > >> > >> > https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing > >> , > >> and I've added edit access for all of our committers. > >> > >> Feel free to take a look and leave any comments you may have, > >> > >> Thanks, > >> > >> Thomas > >> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > >
Re: Testing and the Capability Matrix
It is also worth noting that this document is a snapshot rather than the long-term plan. As the SDK evolves, the annotations will almost certainly change with it (and will certainly expand). +Aljoscha For streaming/batch execution separation, this is better served by configuration in the runner's build (e.g. specifying two separate executions in the pom.xml, one for streaming and one for batch). Given that the tests live in a separate module from the runner, this is likened to how RunnableOnService tests are currently executed by all of the runners. For sink, I think given the current implementations of sink there isn't a huge need; however, most sinks should be annotated with some form of superclass (although the implementation of sink requires side inputs, so this is also worth considering). +jb These would live on the tests proper, yes. On Sun, Jun 12, 2016 at 11:05 PM, Jean-Baptiste Onofré wrote: > Hi Thomas, > > it looks good to me. > > Just curious: the proposed annotations will be directly in the Java SDK > Test jar right ? > > Thanks, > Regards > JB > > > On 06/11/2016 01:34 AM, Thomas Groh wrote: > >> Hey Beamers! >> >> We have a lovely Capability Matrix ( >> http://beam.incubator.apache.org/capability-matrix/) which describes what >> runners can do, and what's in the model. However, right now we only have >> one way to specify that a test is useful to be executed in a runner, the >> RunnableOnService category. >> >> I've worked on a document to expand the number of annotations to be more >> in >> line with the capability matrix, which should help runner writers test >> more >> precisely with regards to the Beam model. The document is located at >> >> https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing >> , >> and I've added edit access for all of our committers. >> >> Feel free to take a look and leave any comments you may have, >> >> Thanks, >> >> Thomas >> >> > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com >
Re: Testing and the Capability Matrix
Hi Thomas, it looks good to me. Just curious: the proposed annotations will be directly in the Java SDK Test jar right ? Thanks, Regards JB On 06/11/2016 01:34 AM, Thomas Groh wrote: Hey Beamers! We have a lovely Capability Matrix ( http://beam.incubator.apache.org/capability-matrix/) which describes what runners can do, and what's in the model. However, right now we only have one way to specify that a test is useful to be executed in a runner, the RunnableOnService category. I've worked on a document to expand the number of annotations to be more in line with the capability matrix, which should help runner writers test more precisely with regards to the Beam model. The document is located at https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing, and I've added edit access for all of our committers. Feel free to take a look and leave any comments you may have, Thanks, Thomas -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
Re: Testing and the Capability Matrix
Ah I forgot this: we should also think about specifying the capabilities for Streaming/Batch execution separately. AFAIK, the Flink and Spark runners have completely separate code paths for translation depending on that mode switch and also different capabilities in the two modes. On Sat, 11 Jun 2016 at 07:48 Aljoscha Krettek wrote: > Hi, > this looks very good! One thing I noticed is that there is no WriteToSink. > Right now, there are no tests in the RunnableOnService category that test > the Sink API but we might be able to add some. (Sinks are implemented as a > couple of DoFns, so it's not strictly necessary but it might still be good > to verify the output of these sinks.) > > Best, > Aljoscha > > On Sat, 11 Jun 2016 at 01:34 Thomas Groh wrote: > >> Hey Beamers! >> >> We have a lovely Capability Matrix ( >> http://beam.incubator.apache.org/capability-matrix/) which describes what >> runners can do, and what's in the model. However, right now we only have >> one way to specify that a test is useful to be executed in a runner, the >> RunnableOnService category. >> >> I've worked on a document to expand the number of annotations to be more >> in >> line with the capability matrix, which should help runner writers test >> more >> precisely with regards to the Beam model. The document is located at >> >> https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing >> , >> and I've added edit access for all of our committers. >> >> Feel free to take a look and leave any comments you may have, >> >> Thanks, >> >> Thomas >> >
Re: Testing and the Capability Matrix
Hi, this looks very good! One thing I noticed is that there is no WriteToSink. Right now, there are no tests in the RunnableOnService category that test the Sink API but we might be able to add some. (Sinks are implemented as a couple of DoFns, so it's not strictly necessary but it might still be good to verify the output of these sinks.) Best, Aljoscha On Sat, 11 Jun 2016 at 01:34 Thomas Groh wrote: > Hey Beamers! > > We have a lovely Capability Matrix ( > http://beam.incubator.apache.org/capability-matrix/) which describes what > runners can do, and what's in the model. However, right now we only have > one way to specify that a test is useful to be executed in a runner, the > RunnableOnService category. > > I've worked on a document to expand the number of annotations to be more in > line with the capability matrix, which should help runner writers test more > precisely with regards to the Beam model. The document is located at > > https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing > , > and I've added edit access for all of our committers. > > Feel free to take a look and leave any comments you may have, > > Thanks, > > Thomas >
Testing and the Capability Matrix
Hey Beamers! We have a lovely Capability Matrix ( http://beam.incubator.apache.org/capability-matrix/) which describes what runners can do, and what's in the model. However, right now we only have one way to specify that a test is useful to be executed in a runner, the RunnableOnService category. I've worked on a document to expand the number of annotations to be more in line with the capability matrix, which should help runner writers test more precisely with regards to the Beam model. The document is located at https://docs.google.com/document/d/1fICxq32t9yWn9qXhmT07xpclHeHX2VlUyVtpi2WzzGM/edit?usp=sharing, and I've added edit access for all of our committers. Feel free to take a look and leave any comments you may have, Thanks, Thomas
