[
https://issues.apache.org/jira/browse/FLINK-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024176#comment-15024176
]
ASF GitHub Bot commented on FLINK-2837:
---------------------------------------
GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/1398
[FLINK-2837][storm] various improvements for Storm compatibility
This pull request contains various fixes. Most prominently, the parsing
logic for Storm topologies has been changed to support multiple inputs. The API
has been redone slightly. Two new examples have been added.
- refactor to use Storm's topology builder
- remove FlinkTopologyBuilder
- instantiate context-based StreamExecutionEnvironment (local or remote)
- remove Flink and Storm behavior replicating classes
- modify FlinkTopology to parse Storm topology directly
- replace StormTestBase with StreamingTestBase
- let the FiniteFileSpout finish in corner cases
- expose taskId, fix off by one task id
- add print example
- FlinkTopologyBuilder changes (check if all inputs are available before
processing)
- correct package typo
- support getter methods on TupleWrapper
- two input support
- add join example
- update docs
- use Flink file system access
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink storm-dev-dev
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1398.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1398
----
commit 613936cc7dce75e8dd811626cd2152b1f1383fe0
Author: Maximilian Michels <m...@apache.org>
Date: 2015-11-12T13:39:45Z
[FLINK-2837][storm] various improvements for Storm compatibility
- refactor to use Storm's topology builder
- remove FlinkTopologyBuilder
- instantiate context-based StreamExecutionEnvironment (local or remote)
- remove Flink and Storm behavior replicating classes
- modify FlinkTopology to parse Storm topology directly
- replace StormTestBase with StreamingTestBase
- let the FiniteFileSpout finish in corner cases
- expose taskId, fix off by one task id
- add print example
- FlinkTopologyBuilder changes (check if all inputs are available before
processing)
- correct package typo
- two input support
- add join example
- update docs
- use Flink file system access
----
> FlinkTopologyBuilder cannot handle multiple input streams
> ---------------------------------------------------------
>
> Key: FLINK-2837
> URL: https://issues.apache.org/jira/browse/FLINK-2837
> Project: Flink
> Issue Type: Bug
> Components: Storm Compatibility
> Reporter: Matthias J. Sax
> Assignee: Maximilian Michels
>
> FlinkTopologyBuilder cannot handle multiple input streams correctly. Instead
> of union the incoming streams, it replicates the consuming bolt and each
> (logical) instance processes one of the input streams.
> For example:
> {noformat}
> final FlinkTopologyBuilder builder = new FlinkTopologyBuilder();
> builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
> builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
> builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));
> builder.setBolt(boltId, new MergerBolt())
> .shuffleGrouping(spoutId1)
> .shuffleGrouping(spoutId2)
> .shuffleGrouping(spoutId3);
> builder.setBolt("sink", new BoltPrintSink(new SimpleOutputFormatter()))
> .shuffleGrouping(boltId);
> {noformat}
> will only print the data from a single source instead of all sources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
