Dear Flink community,
in order to better understand the needs of our users and to plan for the
future, I wanted to reach out to you and ask how much you use Flink's
Python API, namely flink-python and flink-streaming-python.
In order to gather feedback, I would like to ask all Python users to
res
After communicating with some of the internal users at Alibaba, my impression
is that:
* Most of them need C extentions support, they want to integrated their
algorithms with stream processingļ¼but Jython is unacceptable for them.
* For some users, who are only familiar with SQL/Python, developin
Xianda Ke
9:47 AM (11 minutes ago)
to dev, user
After communicating with some of the internal users at Alibaba, my
impression is that:
- Most of them need C extensions support, they want to integrated their
algorithms with stream processingļ¼but Jython is unacceptable for them.
- For some
Hi Xianda,
thanks for sharing this detailed feedback. Do I understand you correctly
that flink-python and flink-streaming-python are not usable for the use
cases at Alibaba atm?
Could you share a bit more details about the Python UDFs for SQL? How do
you execute the Python code? Will it work with
Hi Till,
1. So far as I know, most of the users at Alibaba are using SQL. Some of
users at Alibaba want integrated python libraries with Flink for streaming
processing, and Jython is unusable.
2. Python UDFs for SQL:
* declaring python UDF based on Alibaba's internal DDL syntax.
* start a Python
Did you take a look at Apache Beam? It already provides a comprehensive
Python SDK and can be used with Flink:
https://beam.apache.org/roadmap/portability/#python-on-flink
We are using it at Lyft for Python streaming pipelines.
Thomas
On Tue, Dec 11, 2018 at 5:54 AM Xianda Ke wrote:
> Hi Till,
I like that we are having a general discussion about how to use Python and
Flink together in the future.
The current python support has some shortcomings that were mentioned
before, so we clearly need something better.
Parts of the community have worked together with the Apache Beam project,
which
Hi Folks,
To avoid polluting the survey thread with discussions, we started separate
thread and maybe we can continue the discussion over there.
Regards,
Xianda
On Wed, Dec 12, 2018 at 3:34 AM Stephan Ewen wrote:
> I like that we are having a general discussion about how to use Python and
> Fli
You are right. Let's refocus this on the python user survey and spin out
another thread.
On Thu, Dec 13, 2018 at 9:56 AM Xianda Ke wrote:
> Hi Folks,
> To avoid polluting the survey thread with discussions, we started separate
> thread and maybe we can continue the discussion over there.
>
> Reg
Thanks a lot for the feedback for this survey. I will close it now since 6
days have passed without new activity.
To me it seems that we currently don't have many users who use flink-python
or flink-streaming-python because of their limitations (mentioned in the
survey by Xianda). This information
10 matches
Mail list logo
