Hello, I ran a job today and it seems like on the latest version of beam this is working as jobs are actually running now! If I run into any issues i'll update this thread, but thank you seems like on 2.39 the earlier issues have been resolved.
Thanks, Sahith On Tue, Jun 14, 2022 at 4:14 PM Sahith Nallapareddy <[email protected]> wrote: > Hello, > > I will run another one on the latest beam today and let you know what > happens. The last version I tried this on was I think 2.35. I believe there > were no errors on the dataflow page, but some issues with getting the > workers started. I will try on the latest beam and update to see what > happens! > > Thanks, > > Sahith > > On Tue, Jun 14, 2022 at 12:26 PM Chamikara Jayalath <[email protected]> > wrote: > >> >> >> On Tue, Jun 14, 2022 at 8:47 AM Sahith Nallapareddy <[email protected]> >> wrote: >> >>> Hello, >>> >>> I was wondering if anyone has run a java job with java external >>> transforms in dataflow? We have had python beam jobs run great with java >>> external transforms. However, we tried to run a java job with java external >>> transforms but this seemed to stall on dataflow (this was done a while ago, >>> have to verify this is still happening). We are working on a system and >>> having transforms in an expansion service is a component of it. Is this >>> something that would be supported or is it only going to work cross >>> language? >>> >> >> Do you have any specific errors that your job ran into ? In theory this >> should work (with the latest Beam version) but I don't believe we >> currently have tests for this. >> One thing I can think of that could result in jobs getting stuck is >> forgetting to include "--experiments=use_runner_v2". >> >> +Heejong Lee <[email protected]> >> >> Thanks, >> Cham >> >> >>> >>> Thanks, >>> >>> Sahith >>> >>
