Re: Fun with WebAssembly transforms

+1 and this is exactly what I suggested as well. Python Dataframe, RunInference, Python Map are available via x-lang for Java already [1] and all three need/use simple UDFs to customize operation. There is some logic that needs to be added to use Python transforms from Go SDK. As you suggested ther

Re: Fun with WebAssembly transforms

I agree with Luke. Targeting little helper UDFs that go along with IOs are actually a major feature gap for xlang - like timestamp extractors that have to parse particular data formats. This could be a very useful place to try out the design options. I think we can simplify the problem by insisting

Re: Fun with WebAssembly transforms

I think an easier target would be to support things like DynamicDestinations for Java IO connectors that are exposed as XLang for Go/Python. This is because Go/Python have good transpiling support to WebAssembly and we already exposed several Java IO XLang connectors already so its about plumbing

Re: Fun with WebAssembly transforms

On Wed, Jul 13, 2022 at 9:31 AM Luke Cwik wrote: > First we'll want to choose whether we want to target Wasm, WASI or Wagi. > These terms are defined here

[PROPOSAL] Preparing for 2.41.0 Release

Hi all, The next (2.41.0) release branch cut is scheduled for July 27th, according to the release calendar [1]. I would like to volunteer myself to do this release. My plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help me make sure the r

Re: Fun with WebAssembly transforms

First we'll want to choose whether we want to target Wasm, WASI or Wagi. WASI adds a lot of simple things like access to a clock, random number generator, ... that would expand the scope of what transpiled code can do. It is debatable whether we'll want the power to run the transpiled code as a mic

[CDAP IO] CDAP connector code reviews

Hi community, Our team is working on the new CdapIO connector implementation and we prepared PR with HasOffset interface [1] for the SparkReceiverIO package. We would appreciate it very much if you could review this PR and leave comments. Thank you f

Beam High Priority Issue Report

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P0 Issues: https://github.com/apache/beam/issues/22204 [Bug]: Python build