Currently, Pasan is working on a design for adding a couple implementations to the Filesystem interface in Python, and it's not necessary to consider SDF here. IMHO.
On the other hand, Python's fileio[1] could probably use SDF-based improvements to split when many files are being matched. Best -P. On Mon, Apr 8, 2019 at 10:00 AM Alex Amato <ajam...@google.com> wrote: > +Lukasz Cwik <lc...@google.com>, +Boyuan Zhang <boyu...@google.com>, +Lara > Schmidt <laraschm...@google.com> > > Should splittable DoFn be considered in this design? In order to split and > scale the source step properly? > > On Mon, Apr 8, 2019 at 9:11 AM Ahmet Altay <al...@google.com> wrote: > >> +dev <d...@beam.apache.org> +Pablo Estrada <pabl...@google.com> +Chamikara >> Jayalath <chamik...@google.com> +Udi Meiri <eh...@google.com> >> >> Thank you Pasan. I quickly looked at the proposal and it looks good. >> Added a few folks who could offer additional feedback. >> >> On Mon, Apr 8, 2019 at 12:13 AM Pasan Kamburugamuwa < >> pasankamburugamu...@gmail.com> wrote: >> >>> Hi, >>> >>> I have updated the project proposal according to the given feedback. So >>> can you guys check my proposal again and give me your feedback about >>> corrections I have done. >>> >>> Here is the link to the updated project proposal >>> >>> https://docs.google.com/document/d/1i_PoIrbmhNgwKCS1TYWC28A9RsyZQFsQCJic3aCXO-8/edit?usp=sharing >>> >>> Thank you >>> Pasan Kamburugamuwa >>> >>
