Re: [DISCUSS] Capability Matrix revamp

Closing the loop on this thread, I've summarized the suggestions into a mega-ticket at https://issues.apache.org/jira/browse/BEAM-2888 Eventually, we'll need a redesign, but there is a lot that we can do incrementally. If you want to help, make a subtask for the piece you are handling, or I can

Re: [DISCUSS] Capability Matrix revamp

Agree, it sounds like a good idea to me. Regards JB On 08/31/2017 10:35 AM, Etienne Chauchot wrote: Hi, I think Nexmark (https://github.com/apache/beam/tree/master/sdks/java/nexmark) could help in getting quantitative benchmark metrics for all the runners like Tyler suggested. Another

Re: [DISCUSS] Capability Matrix revamp

+1 for having plain English feature descriptions. Nitpick: the capability matrix uses the "~" symbol, the meaning of which is not entirely clear from the context. I think a legend would be helpful given things have gone beyond ✘ and ✓. -Stas On Mon, Aug 28, 2017 at 7:23 PM Lukasz Cwik

Re: [DISCUSS] Capability Matrix revamp

I like where this is going! Regarding benchmarking, I think we could do this if we had common benchmarking infrastructure and pipelines that regularly run on different Runners so that we have up-to-date data. I think we can also have a more technical section where we show stats on the level

Re: [DISCUSS] Capability Matrix revamp

I would like to have an API compatibility testing. AFAIK there's still gap to achieve our goal (one job for any runner), that means developers should notice the limitation when writing the job. For example PCollectionView is not well supported in FlinkRunner(not quite sure the current status as my

Re: [DISCUSS] Capability Matrix revamp

Oh, I missed 11. Quantitative properties. This seems like an interesting and important project all on its own. Since Beam is so generic, we need pretty diverse measurements for a user to have a hope of extrapolating to their use case. Kenn On Tue, Aug 22, 2017 at 7:22 PM, Kenneth Knowles

Re: [DISCUSS] Capability Matrix revamp

OK, so adding these good ideas to the list: 8. Plain-English summary that comes before the nitty-gritty. 9. Comment on production readiness from maintainers. Maybe testimonials are helpful if they can be obtained? 10. Versioning of all of the above Any more thoughts? I'll summarize in a JIRA in

Re: [DISCUSS] Capability Matrix revamp

Hi, I'd also like to ask if versioning as proposed in BEAM-166 < https://issues.apache.org/jira/browse/BEAM-166> is still relevant? If it is, would this be something we want to add to this proposal? G On 21 August 2017 at 08:31, Tyler Akidau wrote: > Is there any

Re: [DISCUSS] Capability Matrix revamp

Is there any way we could add quantitative runner metrics to this as well? Like by having some benchmarks that process X amount of data, and then detailing in the matrix latency, throughput, and (where possible) cost, etc, numbers for each of the given runners? Semantic support is one thing, but

Re: [DISCUSS] Capability Matrix revamp

It'd be awesome to see these updated. I'd add two more: 1. A plain English summary of the runner's support in Beam. People who are new to Beam won't understand the in-depth coverage and need a general idea of how it is supported. 2. The production readiness of the runner. Does the

[DISCUSS] Capability Matrix revamp

Hi all, I want to revamp https://beam.apache.org/documentation/runners/capability-matrix/ When Beam first started, we didn't work on feature branches for the core runners, and they had a lot more gaps compared to what goes on `master` today, so this tracked our progress in a way that was easy