[
https://issues.apache.org/jira/browse/FLINK-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15836700#comment-15836700
]
ASF GitHub Bot commented on FLINK-3551:
---------------------------------------
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2761#discussion_r97206373
--- Diff:
flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples/iteration/IterateExample.scala
---
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.scala.examples.iteration
+
+import java.util.Random
+
+import org.apache.flink.api.java.utils.ParameterTool
+import org.apache.flink.api.scala._
+import org.apache.flink.streaming.api.functions.source.SourceFunction
+import
org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext
+import org.apache.flink.streaming.api.scala.{DataStream,
StreamExecutionEnvironment}
+
+/**
+ * Example illustrating iterations in Flink streaming.
+ * <p> The program sums up random numbers and counts additions
+ * it performs to reach a specific threshold in an iterative streaming
fashion. </p>
+ *
+ * <p>
+ * This example shows how to use:
+ * <ul>
+ * <li>streaming iterations,
+ * <li>buffer timeout to enhance latency,
+ * <li>directed outputs.
+ * </ul>
+ * </p>
+ */
+object IterateExample {
+
+ private val Bound = 100
+
+ def main(args: Array[String]): Unit = {
+ // Checking input parameters
+ val params = ParameterTool.fromArgs(args)
+
+ // obtain execution environment and set setBufferTimeout to 1 to enable
+ // continuous flushing of the output buffers (lowest latency)
+ val env =
StreamExecutionEnvironment.getExecutionEnvironment.setBufferTimeout(1)
+
+ // make parameters available in the web interface
+ env.getConfig.setGlobalJobParameters(params)
+
+ // create input stream of integer pairs
+ val inputStream =
+ if (params.has("input")) {
+ env.readTextFile(params.get("input")).map { value: String =>
+ val record = value.substring(1, value.length - 1)
+ val splitted = record.split(",")
+ (Integer.parseInt(splitted(0)), Integer.parseInt(splitted(1)))
+ }
+ } else {
+ println("Executing Iterate example with default input data set.")
+ println("Use --input to specify file input.")
+ env.addSource(new RandomFibonacciSource)
+ }
+
+ def withinBound(value: (Int, Int)) = value._1 < Bound && value._2 <
Bound
+
+ // create an iterative data stream from the input with 5 second timeout
+ val numbers = inputStream
+ // Map the inputs so that the next Fibonacci numbers can be
calculated
+ // while preserving the original input tuple
+ // A counter is attached to the tuple and incremented in every
iteration step
+ .map(value => (value._1, value._2, value._1, value._2, 0))
+ .iterate(
+ (iteration: DataStream[(Int, Int, Int, Int, Int)]) => {
+ // calculates the next Fibonacci number and increment the counter
+ val step = iteration.map(value =>
+ (value._1, value._2, value._4, value._3 + value._4, value._5 +
1))
+ // testing which tuple needs to be iterated again
+ val feedback = step.filter(value => withinBound(value._3,
value._4))
+ // get the input pairs that have the greatest iteration counter
--- End diff --
Please check this comment. I do not see a sliding window or how the
greatest iteration counter is identified.
> Sync Scala and Java Streaming Examples
> --------------------------------------
>
> Key: FLINK-3551
> URL: https://issues.apache.org/jira/browse/FLINK-3551
> Project: Flink
> Issue Type: Sub-task
> Components: Examples
> Affects Versions: 1.0.0
> Reporter: Stephan Ewen
> Assignee: Lim Chee Hau
>
> The Scala Examples lack behind the Java Examples
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
