[
https://issues.apache.org/jira/browse/FLINK-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345245#comment-14345245
]
ASF GitHub Bot commented on FLINK-377:
--------------------------------------
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/202#issuecomment-76973194
Yes, this sounds good? Another thing: it has probably already come up but I
just want to make sure, you implement CoGroup and Reduce the way you do because
of performance, correct? That is, you don't do any work in the user code of a
ReduceOperator but you do it in a chained MapPartition because there you get
all the elements which makes communication with the python process more
efficient. Same with CoGroup, where you implement your own grouping logic in
python from the raw input streams.
Overall I like the architecture, the communication between the host and the
guest language is well abstracted and I can see this being reused for other
languages.
Could you rename the CoGroupPython* classes to something more generic?
Because they really are a part of the generic language binding stuff and not
specific to python, correct?
> Create a general purpose framework for language bindings
> --------------------------------------------------------
>
> Key: FLINK-377
> URL: https://issues.apache.org/jira/browse/FLINK-377
> Project: Flink
> Issue Type: Improvement
> Reporter: GitHub Import
> Assignee: Chesnay Schepler
> Labels: github-import
> Fix For: pre-apache
>
>
> A general purpose API to run operators with arbitrary binaries.
> This will allow to run Stratosphere programs written in Python, JavaScript,
> Ruby, Go or whatever you like.
> We suggest using Google Protocol Buffers for data serialization. This is the
> list of languages that currently support ProtoBuf:
> https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns
> Very early prototype with python:
> https://github.com/rmetzger/scratch/tree/learn-protobuf (basically testing
> protobuf)
> For Ruby: https://github.com/infochimps-labs/wukong
> Two new students working at Stratosphere (@skunert and @filiphaase) are
> working on this.
> The reference binding language will be for Python, but other bindings are
> very welcome.
> The best name for this so far is "stratosphere-lang-bindings".
> I created this issue to track the progress (and give everybody a chance to
> comment on this)
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/377
> Created by: [rmetzger|https://github.com/rmetzger]
> Labels: enhancement,
> Assignee: [filiphaase|https://github.com/filiphaase]
> Created at: Tue Jan 07 19:47:20 CET 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
