I proposed three components since the audience might be different. Also we can use the same component to track issues related to all cross-language wrappers available in a given SDK. If this is too much a single component is fine as well.
Ashwin, as others pointed out, the cross-language transforms framework is primarily for giving SDKs access to transforms that are not available natively. But there are other potential use-cases as well (for example, using two different Python environments within the same pipeline). Exact performance will depend on the runner implementation as well as the additional cost involved due to serializing/deserializing data across environment boundaries. But we haven't done enough analysis/benchmarking to provide more details on this. Thanks, Cham On Thu, May 28, 2020 at 1:55 PM Kyle Weaver <kcwea...@google.com> wrote: > > What are some of the benefits / drawbacks of using cross-language > transforms? Would a native Python transform perform better than a > cross-language transform written in Java that is then used in a Python > pipeline? > > As Rui says, the main advantage is code reuse. See > https://beam.apache.org/roadmap/connectors-multi-sdk/ for more > information. > > On Thu, May 28, 2020 at 4:53 PM Rui Wang <ruw...@google.com> wrote: > >> +1 on dedicated components for cross-language transform. It might be easy >> to manage to have one component (one tag for all SDK) rather than >> multiple ones. >> >> >> Re Ashwin, >> >> Cham knows more than me. AFAIK, cross-language transforms will maximize >> code reuse for newly developed SDK (e.g. IO transforms for Go SDK). Of >> course, a SDK can develop its own IOs, but it's lots of work. >> >> >> -Rui >> >> On Thu, May 28, 2020 at 1:50 PM Ashwin Ramaswami <aramaswa...@gmail.com> >> wrote: >> >>> What are some of the benefits / drawbacks of using cross-language >>> transforms? Would a native Python transform perform better than a >>> cross-language transform written in Java that is then used in a Python >>> pipeline? >>> >>> Ashwin Ramaswami >>> Student >>> *Find me on my:* LinkedIn <https://www.linkedin.com/in/ashwin-r> | >>> Website <https://epicfaace.github.io/> | GitHub >>> <https://github.com/epicfaace> >>> >>> >>> On Thu, May 28, 2020 at 4:44 PM Kyle Weaver <kcwea...@google.com> wrote: >>> >>>> SGTM. Though I'm not sure it's necessary to split by language. It might >>>> be easier to use a single cross-language tag, rather than having to tag >>>> lots of issues as both sdks-python-xlang and sdks-java-xlang. >>>> >>>> On Thu, May 28, 2020 at 4:29 PM Chamikara Jayalath < >>>> chamik...@google.com> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I think it's good if we can have new Jira components to easily track >>>>> various issues related to cross-language transforms. >>>>> >>>>> What do you think about adding the following Jira components ? >>>>> >>>>> sdks-python-xlang >>>>> sdks-java-xlang >>>>> sdks-go-xlang >>>>> >>>>> Jira component sdks-foo-xlang is for tracking issues related to >>>>> cross-language transforms for SDK Foo. For example, >>>>> * Issues related cross-language transforms wrappers written in SDK Foo >>>>> * Issues related to transforms implemented in SDK Foo that are >>>>> offered as cross-language transforms to other SDKs >>>>> * Issues related to cross-language transform expansion service >>>>> implemented for SDK Foo >>>>> >>>>> Thanks, >>>>> Cham >>>>> >>>>
