On Fri, May 29, 2020 at 8:53 PM Kenneth Knowles <k...@apache.org> wrote:
> -0, sorry I'm late. But I guess you are not sorry since everyone else > agrees :-) > > I just don't know what this accomplishes. Most components correspond to a > software component/codebase. The only components that do not correspond to > a codebase are things that are sort of independent, like "test-failures" > and "gcp-quota". > > This seems like a label does all the same things, since you can tag with > multiple components. AFAIK the only difference between a component and a > label is that Jira requires >= component (actually I bet we control this). > Does it make analytics dashboards more useful? > > A hypothetical test is: when will I tag with this component? That is > actually not so clear. If I am a Python user in the future, and xlang is > done very well, then I may not even know that I am using a Java-based > component. So what I want to find is either "io-python-kafka" (for the > wrapper) or "io-kafka" (if the wrapper is so good I don't even know). I > think the future of portable Beam is probably to go with "io-kafka". > > The reason I am -0 is that it is harmless to have extra components and use > them when a label would suffice. > I get your point and in the distant future where cross-language transforms framework works seamlessly for all major SDK/runner combinations we might come to a state where we do not need this Jira component. But we are not there yet :). Given that various folks are working on and/or interested in picking up tasks related to cross-language transforms and given that many users may potentially start using cross-language transforms in the near future I think the Jira component will make things easier to manage. Also please note that we at least have one component per language, expansion-service, as a cross-language specific component. Labels might achieve the same thing but seems like it's harder to maintain a consistent label across folks working on various runners/SDKs. Thanks, Cham > > Kenn > > On Fri, May 29, 2020 at 10:38 AM Chamikara Jayalath <chamik...@google.com> > wrote: > >> Thanks. I'll go ahead and add related Jiras to it. >> >> - Cham >> >> On Fri, May 29, 2020 at 9:49 AM Luke Cwik <lc...@google.com> wrote: >> >>> I have added the new component: >>> >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20cross-language >>> >>> I didn't set a component owner for it and made the default assignee be >>> the project default (unassigned). If you would like any of these to be >>> changed, please let me know. >>> >>> On Fri, May 29, 2020 at 9:18 AM Chamikara Jayalath <chamik...@google.com> >>> wrote: >>> >>>> Good point. "cross-language" sgtm. >>>> >>>> On Fri, May 29, 2020, 8:57 AM Kyle Weaver <kcwea...@google.com> wrote: >>>> >>>>> Nit: can we name it `cross-language` instead of `xlang`? >>>>> Component names auto-complete, so there's no reason to abbreviate. >>>>> >>>>> On Fri, May 29, 2020 at 11:54 AM Chamikara Jayalath < >>>>> chamik...@google.com> wrote: >>>>> >>>>>> Thanks for the comments. >>>>>> >>>>>> Can someone please create a single Jira component named "xlang" ? >>>>>> Looks like I don't have access to create components. >>>>>> >>>>>> Thanks, >>>>>> Cham >>>>>> >>>>>> On Fri, May 29, 2020 at 7:24 AM Alexey Romanenko < >>>>>> aromanenko....@gmail.com> wrote: >>>>>> >>>>>>> +1 to new non split component. >>>>>>> >>>>>>> On 29 May 2020, at 07:19, Heejong Lee <heej...@google.com> wrote: >>>>>>> >>>>>>> If we use one meta component tag for all xlang related issues, I >>>>>>> would prefer just "xlang". Then we could attach the "xlang" tag to not >>>>>>> only >>>>>>> language specific sdk tags but also other runner tags e.g. ['xlang', >>>>>>> 'io-java-kafka'], ['xlang'', 'runner-dataflow']. >>>>>>> >>>>>>> On Thu, May 28, 2020 at 7:49 PM Robert Burke <rob...@frantil.com> >>>>>>> wrote: >>>>>>> >>>>>>>> +1 to new component not split. The language concerns can be >>>>>>>> represented and filtered with the existing sdk tags. I know I'm >>>>>>>> interested >>>>>>>> in all sdk-go issues, and would prefer not to have to union tags when >>>>>>>> searching for Go related issues. >>>>>>>> >>>>>>>> On Thu, 28 May 2020 at 15:48, Ismaël Mejía <ieme...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> +1 to new component not splitted >>>>>>>>> >>>>>>>>> Other use case is using libraries not available in your language >>>>>>>>> e.g. using some python transform that relies in a python only API in >>>>>>>>> the >>>>>>>>> middle of a Java pipeline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, May 28, 2020 at 11:12 PM Chamikara Jayalath < >>>>>>>>> chamik...@google.com> wrote: >>>>>>>>> >>>>>>>>>> I proposed three components since the audience might be >>>>>>>>>> different. Also we can use the same component to track issues >>>>>>>>>> related to >>>>>>>>>> all cross-language wrappers available in a given SDK. If this is too >>>>>>>>>> much a >>>>>>>>>> single component is fine as well. >>>>>>>>>> >>>>>>>>>> Ashwin, as others pointed out, the cross-language transforms >>>>>>>>>> framework is primarily for giving SDKs access to transforms that are >>>>>>>>>> not >>>>>>>>>> available natively. But there are other potential use-cases as well >>>>>>>>>> (for >>>>>>>>>> example, using two different Python environments within the same >>>>>>>>>> pipeline). >>>>>>>>>> Exact performance will depend on the runner implementation as >>>>>>>>>> well as the additional cost involved due to >>>>>>>>>> serializing/deserializing data >>>>>>>>>> across environment boundaries. But we haven't done enough >>>>>>>>>> analysis/benchmarking to provide more details on this. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Cham >>>>>>>>>> >>>>>>>>>> On Thu, May 28, 2020 at 1:55 PM Kyle Weaver <kcwea...@google.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> > What are some of the benefits / drawbacks of using >>>>>>>>>>> cross-language transforms? Would a native Python transform perform >>>>>>>>>>> better >>>>>>>>>>> than a cross-language transform written in Java that is then used >>>>>>>>>>> in a >>>>>>>>>>> Python pipeline? >>>>>>>>>>> >>>>>>>>>>> As Rui says, the main advantage is code reuse. See >>>>>>>>>>> https://beam.apache.org/roadmap/connectors-multi-sdk/ for more >>>>>>>>>>> information. >>>>>>>>>>> >>>>>>>>>>> On Thu, May 28, 2020 at 4:53 PM Rui Wang <ruw...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> +1 on dedicated components for cross-language transform. It >>>>>>>>>>>> might be easy to manage to have one component (one tag for all >>>>>>>>>>>> SDK) rather >>>>>>>>>>>> than multiple ones. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Re Ashwin, >>>>>>>>>>>> >>>>>>>>>>>> Cham knows more than me. AFAIK, cross-language transforms will >>>>>>>>>>>> maximize code reuse for newly developed SDK (e.g. IO transforms >>>>>>>>>>>> for Go >>>>>>>>>>>> SDK). Of course, a SDK can develop its own IOs, but it's lots of >>>>>>>>>>>> work. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -Rui >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 28, 2020 at 1:50 PM Ashwin Ramaswami < >>>>>>>>>>>> aramaswa...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> What are some of the benefits / drawbacks of using >>>>>>>>>>>>> cross-language transforms? Would a native Python transform >>>>>>>>>>>>> perform better >>>>>>>>>>>>> than a cross-language transform written in Java that is then used >>>>>>>>>>>>> in a >>>>>>>>>>>>> Python pipeline? >>>>>>>>>>>>> >>>>>>>>>>>>> Ashwin Ramaswami >>>>>>>>>>>>> Student >>>>>>>>>>>>> *Find me on my:* LinkedIn >>>>>>>>>>>>> <https://www.linkedin.com/in/ashwin-r> | Website >>>>>>>>>>>>> <https://epicfaace.github.io/> | GitHub >>>>>>>>>>>>> <https://github.com/epicfaace> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, May 28, 2020 at 4:44 PM Kyle Weaver < >>>>>>>>>>>>> kcwea...@google.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> SGTM. Though I'm not sure it's necessary to split by >>>>>>>>>>>>>> language. It might be easier to use a single cross-language tag, >>>>>>>>>>>>>> rather >>>>>>>>>>>>>> than having to tag lots of issues as both sdks-python-xlang and >>>>>>>>>>>>>> sdks-java-xlang. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, May 28, 2020 at 4:29 PM Chamikara Jayalath < >>>>>>>>>>>>>> chamik...@google.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think it's good if we can have new Jira components to >>>>>>>>>>>>>>> easily track various issues related to cross-language >>>>>>>>>>>>>>> transforms. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What do you think about adding the following Jira components >>>>>>>>>>>>>>> ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> sdks-python-xlang >>>>>>>>>>>>>>> sdks-java-xlang >>>>>>>>>>>>>>> sdks-go-xlang >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Jira component sdks-foo-xlang is for tracking issues related >>>>>>>>>>>>>>> to cross-language transforms for SDK Foo. For example, >>>>>>>>>>>>>>> * Issues related cross-language transforms wrappers written >>>>>>>>>>>>>>> in SDK Foo >>>>>>>>>>>>>>> * Issues related to transforms implemented in SDK Foo that >>>>>>>>>>>>>>> are offered as cross-language transforms to other SDKs >>>>>>>>>>>>>>> * Issues related to cross-language transform expansion >>>>>>>>>>>>>>> service implemented for SDK Foo >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Cham >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>
