Thanks. I'll go ahead and add related Jiras to it. - Cham
On Fri, May 29, 2020 at 9:49 AM Luke Cwik <lc...@google.com> wrote: > I have added the new component: > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20cross-language > > I didn't set a component owner for it and made the default assignee be the > project default (unassigned). If you would like any of these to be > changed, please let me know. > > On Fri, May 29, 2020 at 9:18 AM Chamikara Jayalath <chamik...@google.com> > wrote: > >> Good point. "cross-language" sgtm. >> >> On Fri, May 29, 2020, 8:57 AM Kyle Weaver <kcwea...@google.com> wrote: >> >>> Nit: can we name it `cross-language` instead of `xlang`? Component names >>> auto-complete, so there's no reason to abbreviate. >>> >>> On Fri, May 29, 2020 at 11:54 AM Chamikara Jayalath < >>> chamik...@google.com> wrote: >>> >>>> Thanks for the comments. >>>> >>>> Can someone please create a single Jira component named "xlang" ? Looks >>>> like I don't have access to create components. >>>> >>>> Thanks, >>>> Cham >>>> >>>> On Fri, May 29, 2020 at 7:24 AM Alexey Romanenko < >>>> aromanenko....@gmail.com> wrote: >>>> >>>>> +1 to new non split component. >>>>> >>>>> On 29 May 2020, at 07:19, Heejong Lee <heej...@google.com> wrote: >>>>> >>>>> If we use one meta component tag for all xlang related issues, I would >>>>> prefer just "xlang". Then we could attach the "xlang" tag to not only >>>>> language specific sdk tags but also other runner tags e.g. ['xlang', >>>>> 'io-java-kafka'], ['xlang'', 'runner-dataflow']. >>>>> >>>>> On Thu, May 28, 2020 at 7:49 PM Robert Burke <rob...@frantil.com> >>>>> wrote: >>>>> >>>>>> +1 to new component not split. The language concerns can be >>>>>> represented and filtered with the existing sdk tags. I know I'm >>>>>> interested >>>>>> in all sdk-go issues, and would prefer not to have to union tags when >>>>>> searching for Go related issues. >>>>>> >>>>>> On Thu, 28 May 2020 at 15:48, Ismaël Mejía <ieme...@gmail.com> wrote: >>>>>> >>>>>>> +1 to new component not splitted >>>>>>> >>>>>>> Other use case is using libraries not available in your language >>>>>>> e.g. using some python transform that relies in a python only API in the >>>>>>> middle of a Java pipeline. >>>>>>> >>>>>>> >>>>>>> On Thu, May 28, 2020 at 11:12 PM Chamikara Jayalath < >>>>>>> chamik...@google.com> wrote: >>>>>>> >>>>>>>> I proposed three components since the audience might be different. >>>>>>>> Also we can use the same component to track issues related to all >>>>>>>> cross-language wrappers available in a given SDK. If this is too much a >>>>>>>> single component is fine as well. >>>>>>>> >>>>>>>> Ashwin, as others pointed out, the cross-language transforms >>>>>>>> framework is primarily for giving SDKs access to transforms that are >>>>>>>> not >>>>>>>> available natively. But there are other potential use-cases as well >>>>>>>> (for >>>>>>>> example, using two different Python environments within the same >>>>>>>> pipeline). >>>>>>>> Exact performance will depend on the runner implementation as well >>>>>>>> as the additional cost involved due to serializing/deserializing data >>>>>>>> across environment boundaries. But we haven't done enough >>>>>>>> analysis/benchmarking to provide more details on this. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Cham >>>>>>>> >>>>>>>> On Thu, May 28, 2020 at 1:55 PM Kyle Weaver <kcwea...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> > What are some of the benefits / drawbacks of using >>>>>>>>> cross-language transforms? Would a native Python transform perform >>>>>>>>> better >>>>>>>>> than a cross-language transform written in Java that is then used in a >>>>>>>>> Python pipeline? >>>>>>>>> >>>>>>>>> As Rui says, the main advantage is code reuse. See >>>>>>>>> https://beam.apache.org/roadmap/connectors-multi-sdk/ for more >>>>>>>>> information. >>>>>>>>> >>>>>>>>> On Thu, May 28, 2020 at 4:53 PM Rui Wang <ruw...@google.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> +1 on dedicated components for cross-language transform. It might >>>>>>>>>> be easy to manage to have one component (one tag for all SDK) rather >>>>>>>>>> than >>>>>>>>>> multiple ones. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Re Ashwin, >>>>>>>>>> >>>>>>>>>> Cham knows more than me. AFAIK, cross-language transforms will >>>>>>>>>> maximize code reuse for newly developed SDK (e.g. IO transforms for >>>>>>>>>> Go >>>>>>>>>> SDK). Of course, a SDK can develop its own IOs, but it's lots of >>>>>>>>>> work. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Rui >>>>>>>>>> >>>>>>>>>> On Thu, May 28, 2020 at 1:50 PM Ashwin Ramaswami < >>>>>>>>>> aramaswa...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> What are some of the benefits / drawbacks of using >>>>>>>>>>> cross-language transforms? Would a native Python transform perform >>>>>>>>>>> better >>>>>>>>>>> than a cross-language transform written in Java that is then used >>>>>>>>>>> in a >>>>>>>>>>> Python pipeline? >>>>>>>>>>> >>>>>>>>>>> Ashwin Ramaswami >>>>>>>>>>> Student >>>>>>>>>>> *Find me on my:* LinkedIn <https://www.linkedin.com/in/ashwin-r> | >>>>>>>>>>> Website <https://epicfaace.github.io/> | GitHub >>>>>>>>>>> <https://github.com/epicfaace> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, May 28, 2020 at 4:44 PM Kyle Weaver <kcwea...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> SGTM. Though I'm not sure it's necessary to split by language. >>>>>>>>>>>> It might be easier to use a single cross-language tag, rather than >>>>>>>>>>>> having >>>>>>>>>>>> to tag lots of issues as both sdks-python-xlang and >>>>>>>>>>>> sdks-java-xlang. >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 28, 2020 at 4:29 PM Chamikara Jayalath < >>>>>>>>>>>> chamik...@google.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> I think it's good if we can have new Jira components to easily >>>>>>>>>>>>> track various issues related to cross-language transforms. >>>>>>>>>>>>> >>>>>>>>>>>>> What do you think about adding the following Jira components ? >>>>>>>>>>>>> >>>>>>>>>>>>> sdks-python-xlang >>>>>>>>>>>>> sdks-java-xlang >>>>>>>>>>>>> sdks-go-xlang >>>>>>>>>>>>> >>>>>>>>>>>>> Jira component sdks-foo-xlang is for tracking issues related >>>>>>>>>>>>> to cross-language transforms for SDK Foo. For example, >>>>>>>>>>>>> * Issues related cross-language transforms wrappers written in >>>>>>>>>>>>> SDK Foo >>>>>>>>>>>>> * Issues related to transforms implemented in SDK Foo that are >>>>>>>>>>>>> offered as cross-language transforms to other SDKs >>>>>>>>>>>>> * Issues related to cross-language transform expansion service >>>>>>>>>>>>> implemented for SDK Foo >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Cham >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>
