Sorry I was not specific enough. I ment using the SqlTransform registerUdf and registerUdaf. I use a lot of SQL in my pipeline and I would prefer using SQL UDFs in many cases over writing beam transforms. I already have UDFs but I did not find a way to make them work over nested structures. Thank you, Gyorgy
On Thu, Sep 28, 2023 at 5:40 PM Robert Bradshaw via user < user@beam.apache.org> wrote: > Yes, for sure. This is one of the areas Beam excels vs. more simple tools > like SQL. You can write arbitrary code to iterate over arbitrary structures > in the typical Java/Python/Go/Typescript/Scala/[pick your language] way. In > the Beam nomenclature. UDFs correspond to DoFns and UDAFs correspond to > CombineFns. > > On Thu, Sep 28, 2023 at 4:23 AM Balogh, György <bog...@ultinous.com> > wrote: > >> Hi, >> I've complex nested structure in my input data. Is it possible to have >> UDF/UDAF taking nested structure as input? I'm using java. >> Outputting nested structure is also a question. >> Thank you, >> Gyorgy >> -- >> >> György Balogh >> CTO >> E gyorgy.bal...@ultinous.com <zsolt.sala...@ultinous.com> >> M +36 30 270 8342 <+36%2030%20270%208342> >> A HU, 1117 Budapest, Budafoki út 209. >> W www.ultinous.com >> > -- György Balogh CTO E gyorgy.bal...@ultinous.com <zsolt.sala...@ultinous.com> M +36 30 270 8342 <+36%2030%20270%208342> A HU, 1117 Budapest, Budafoki út 209. W www.ultinous.com
