Will Apache Beam adopt a Pandas-like syntax to program in Python?

2021-09-29 Thread David Ciudad Gomez
Hi, Apache Spark is adopting a new Pandas-like syntax ( https://github.com/databricks/koalas) for programming in Python. Will Apache Beam adopt a similar syntax in the future? Thanks and best regards. David Ciudad

Re: Will Apache Beam adopt a Pandas-like syntax to program in Python?

2021-09-29 Thread Brian Hulette
Hi David, Yes! Apache Beam now has a DataFrame API [1], which provides similar functionality. It exited experimental in Beam 2.32.0 [2]. You can see some example pipelines that use it here [3]. Brian [1] https://beam.apache.org/documentation/dsls/dataframes/overview/ [2] https://beam.apache.org/