Just added the two I mentioned in my previous message.Thanks Davor. On Wed, Mar 1, 2017 at 6:27 PM, Aljoscha Krettek <aljos...@apache.org> wrote:
> On it! > > On Wed, 1 Mar 2017 at 18:17 Davor Bonaci <da...@apache.org> wrote: > > > We've now moved the discussion into the content of the first stable > > release. > > > > I've created a version in JIRA called "First stable release". I'd like to > > invite everyone to triage JIRA issues you care about, and assign "Fix > > Versions" field to "First stable release" to mark the issue blocking for > > the first stable release. This creates a project-wide burndown list and > we > > can track our progress towards the goal. > > > > I'll try make a pass over as many JIRA issues as possible over the next > day > > or two, but it would be great if everyone, particularly component leads > in > > JIRA, take a pass too! > > > > On Wed, Mar 1, 2017 at 2:51 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > > > > > Yes, fully agree. > > > > > > As far as I understood/know, BEAM-59 is targeted for Beam 1.0 (it's > what > > > we discussed with Pei and Davor). > > > > > > Regards > > > JB > > > > > > > > > On 03/01/2017 11:39 AM, Ismaël Mejía wrote: > > > > > >> Also joining a bit late, I agree with Amit, HDFS improvements are a > > really > > >> good thing to have before the stable release. I will also add the > > >> IOChannelFactory refactorings to support things like > > Read.from(“hdfs://”) > > >> aka BEAM-59. > > >> > > >> In the worse case particular IOs can still be marked as experimental > to > > >> show users that they can still evolve, even after the first ‘stable’ > > >> release, the part that we have to pay more attention is not to break > the > > >> core SDK. And the question about Data Locality (BEAM-673) is where I > am > > >> afraid that we can have some breaking changes because there is not a > way > > >> from the IOs (Source/Sink) to send ‘a hint’ to the runner about Data > > >> Locality (please correct me if I am wrong). And this even if not > > supported > > >> in the first stable release by any runner, would be a really great > thing > > >> to > > >> have and I think this is a good moment to do it, to avoid breaking any > > >> IO/runner signature because of new methods. > > >> > > >> What do the others think ? > > >> Ismaël > > >> > > >> > > >> > > >> On Tue, Feb 28, 2017 at 6:29 PM, Amit Sela <amitsel...@gmail.com> > > wrote: > > >> > > >> Joining in just a bit late, I'll be quick and say that IMHO the SDK is > > >>> mature enough and so my only point to add is *HDFS support*. > > >>> I think that in terms of adoption we have to support HDFS as a > > >>> "first-class > > >>> citizen" via the FileSystem API, and provide data locality (batch) on > > top > > >>> of it - it serves not only HDFS, but other eco-system IOs such as > > HBase. > > >>> From my experience with talking to people and companies, most are > > running > > >>> batch in production with some streaming POC or even production use, > but > > >>> batch still takes most of production work. If we give them the same > > >>> production results, with the Beam API, we can on-board them faster > and > > >>> make > > >>> it easier for them to adopt streaming as well. > > >>> > > >>> Thanks, > > >>> Amit > > >>> > > >>> On Tue, Feb 28, 2017 at 7:12 PM Davor Bonaci <da...@apache.org> > wrote: > > >>> > > >>> Alright -- sounds like we have a consensus to proceed with the first > > >>>> > > >>> stable > > >>> > > >>>> release after 0.6.0, targeting end of March / early April. I'll kick > > off > > >>>> separate threads for specific decisions we need to make. > > >>>> > > >>>> On Thu, Feb 23, 2017 at 6:07 AM, Aljoscha Krettek < > > aljos...@apache.org> > > >>>> wrote: > > >>>> > > >>>> I think we're ready for this! The public APIs are in very good > shape, > > >>>>> especially now that we have the new DoFn, user facing state and > > timers > > >>>>> > > >>>> and > > >>>> > > >>>>> splittable DoFn. Not all Runners support the more advanced features > > but > > >>>>> > > >>>> we > > >>>> > > >>>>> can work on this after a stable release and there are enough > runners > > >>>>> > > >>>> that > > >>> > > >>>> support a large part of the features. > > >>>>> > > >>>>> Best, > > >>>>> Aljoscha > > >>>>> > > >>>>> On Thu, 23 Feb 2017 at 06:15 Kenneth Knowles > <k...@google.com.invalid > > > > > >>>>> wrote: > > >>>>> > > >>>>> On Wed, Feb 22, 2017 at 5:35 PM, Chamikara Jayalath < > > >>>>>> > > >>>>> chamik...@apache.org> > > >>>>> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> > > >>>>>>> I think, this point applies to Python SDK as well (though as you > > >>>>>>> > > >>>>>> mentioned, > > >>>>>> > > >>>>>>> API hiding in Python is a mere convention (prefix with > underscore) > > >>>>>>> > > >>>>>> not > > >>>> > > >>>>> enforced. We already have mechanism for marking APIs as deprecated > > >>>>>>> > > >>>>>> which > > >>>>> > > >>>>>> might be useful here: > > >>>>>>> https://github.com/apache/beam/blob/master/sdks/python/ > > >>>>>>> apache_beam/utils/annotations.py > > >>>>>>> > > >>>>>>> - Cham > > >>>>>>> > > >>>>>>> > > >>>>>> Perhaps an explicit @public annotation would fit. I could imagine > > >>>>>> > > >>>>> easily > > >>>> > > >>>>> generating a spec to check against from such annotations, though > > >>>>>> > > >>>>> tooling > > >>>> > > >>>>> is > > >>>>> > > >>>>>> secondary to documentation. > > >>>>>> > > >>>>>> Kenn > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > -- > > > Jean-Baptiste Onofré > > > jbono...@apache.org > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > >
