alpinegizmo commented on a change in pull request #11834: URL: https://github.com/apache/flink/pull/11834#discussion_r412151143
########## File path: docs/tutorials/datastream_api.md ########## @@ -0,0 +1,252 @@ +--- +title: Intro to the DataStream API +nav-id: datastream-api +nav-pos: 2 +nav-title: Intro to the DataStream API +nav-parent_id: tutorials +permalink: /tutorials/datastream_api.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +The focus of this tutorial is to broadly cover the DataStream API well enough that you will be +able to get started writing streaming applications. + +* This will be replaced by the TOC +{:toc} + +## What can be Streamed? + +Flink's DataStream APIs for Java and Scala will let you stream anything they can serialize. Flink's +own serializer is used for + +- basic types, i.e., String, Long, Integer, Boolean, Array +- composite types: Tuples, POJOs, and Scala case classes + +and Flink falls back to Kryo for other types. It's also possible to use other serializers with +Flink. Avro, in particular, is well supported. + +### Java tuples and POJOs + +Flink's native serializer can operate efficiently on tuples and POJOs. + +#### Tuples + +For Java, Flink defines its own Tuple1 thru Tuple25 types. + +{% highlight java %} +Tuple2<String, Integer> person = new Tuple2<>("Fred", 35); + +// zero based index! +String name = person.f0; +Integer age = person.f1; +{% endhighlight %} + +#### POJOs + +A POJO (plain old Java object) is any Java class that + +- has an empty default constructor +- all fields are either + - public, or + - have a default getter and setter + +Example: + +{% highlight java %} +public class Person { + public String name; + public Integer age; + public Person() {}; + public Person(String name, Integer age) { + . . . + }; +} + +Person person = new Person("Fred Flintstone", 35); +{% endhighlight %} + +Flink's serializer [supports schema evolution for POJO types]({{ site.baseurl }}{% link dev/stream/state/schema_evolution.md %}#pojo-types). + +### Scala tuples and case classes + +These work just as you'd expect. + +{% top %} + +## A Complete Example + +This example takes a stream of records about people as input, and filters it to only include the adults. + +{% highlight java %} +import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; +import org.apache.flink.streaming.api.datastream.DataStream; +import org.apache.flink.api.common.functions.FilterFunction; + +public class Example { + + public static void main(String[] args) throws Exception { + final StreamExecutionEnvironment env = + StreamExecutionEnvironment.getExecutionEnvironment(); + + DataStream<Person> flintstones = env.fromElements( + new Person("Fred", 35), + new Person("Wilma", 35), + new Person("Pebbles", 2)); + + DataStream<Person> adults = flintstones.filter(new FilterFunction<Person>() { + @Override + public boolean filter(Person person) throws Exception { + return person.age >= 18; + } + }); Review comment: This is the only explanation of a FilterFunction that they're going to get before the exercise (which uses one). For that reason, I wrote this out in full. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
