This is an automated email from the ASF dual-hosted git repository.
arvid pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4a39d70 Rebuild website
4a39d70 is described below
commit 4a39d70eeb6a0bfc4d07d5c5d969155d7d386bee
Author: Arvid Heise
AuthorDate: Fri Oct 22 20:53:40 2021 +0200
Rebuild website
---
content/blog/feed.xml | 4 ++--
content/news/2021/09/29/release-1.14.0.html | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/content/blog/feed.xml b/content/blog/feed.xml
index b699e73..18bf32d 100644
--- a/content/blog/feed.xml
+++ b/content/blog/feed.xml
@@ -213,7 +213,7 @@ seamlessly switching over from one source to the
other.
The motivating use case for the Hybrid Source was to read streams
from tiered storage setups as if there was one
stream that spans all tiers. For example, new data may land in Kafa and is
eventually
migrated to S3 (typically in compressed columnar format, for cost efficiency
and performance).
-The Hybrid Source can read this as one contiguous logical stream, starting
with the the historic data on S3
+The Hybrid Source can read this as one contiguous logical stream, starting
with the historic data on S3
and transitioning over to the more recent data in Kafka.
@@ -268,7 +268,7 @@ reduce the amount of in-flight data stored in unaliged
checkpoints under backpre
-
Buffer Deloating acts as a complementary feature, or even
alternative, to unaligned checkpoints.
+
Buffer Debloating acts as a complementary feature, or even
alternative, to unaligned checkpoints.
Checkout the documentation;
to see how to activate this feature.
diff --git a/content/news/2021/09/29/release-1.14.0.html
b/content/news/2021/09/29/release-1.14.0.html
index 7e506dc..ba241c0 100644
--- a/content/news/2021/09/29/release-1.14.0.html
+++ b/content/news/2021/09/29/release-1.14.0.html
@@ -344,7 +344,7 @@ seamlessly switching over from one source to the other.
The motivating use case for the Hybrid Source was to read streams from
tiered storage setups as if there was one
stream that spans all tiers. For example, new data may land in Kafa and is
eventually
migrated to S3 (typically in compressed columnar format, for cost efficiency
and performance).
-The Hybrid Source can read this as one contiguous logical stream, starting
with the the historic data on S3
+The Hybrid Source can read this as one contiguous logical stream, starting
with the historic data on S3
and transitioning over to the more recent data in Kafka.
@@ -399,7 +399,7 @@ reduce the amount of in-flight data stored in unaliged
checkpoints under backpre
-Buffer Deloating acts as a complementary feature, or even alternative, to
unaligned checkpoints.
+Buffer Debloating acts as a complementary feature, or even alternative, to
unaligned checkpoints.
Checkout the https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/memory/network_mem_tuning/#the-buffer-debloating-mechanism";>documentation
to see how to activate this feature.
This is an automated email from the ASF dual-hosted git repository.
arvid pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c143c1e fix some typos
c143c1e is described below
commit c143c1ef7c5038a1f84302b26be35bb2cad26cc5
Author: Jing Ge
AuthorDate: Wed Oct 13 16:39:13 2021 +0200
fix some typos
---
_posts/2021-09-29-release-1.14.0.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/_posts/2021-09-29-release-1.14.0.md
b/_posts/2021-09-29-release-1.14.0.md
index d104be9..7e89577 100644
--- a/_posts/2021-09-29-release-1.14.0.md
+++ b/_posts/2021-09-29-release-1.14.0.md
@@ -125,7 +125,7 @@ seamlessly switching over from one source to the other.
The motivating use case for the Hybrid Source was to read streams from tiered
storage setups as if there was one
stream that spans all tiers. For example, new data may land in Kafa and is
eventually
migrated to S3 (typically in compressed columnar format, for cost efficiency
and performance).
-The Hybrid Source can read this as one contiguous logical stream, starting
with the the historic data on S3
+The Hybrid Source can read this as one contiguous logical stream, starting
with the historic data on S3
and transitioning over to the more recent data in Kafka.
@@ -181,7 +181,7 @@ reduce the amount of in-flight data stored in unaliged
checkpoints under backpre
-Buffer Deloating acts as a complementary feature, or even alternative, to
unaligned checkpoints.
+Buffer Debloating acts as a complementary feature, or even alternative, to
unaligned checkpoints.
Checkout the
[documentation](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/memory/network_mem_tuning/#the-buffer-debloating-mechanism)
to see how to activate this feature.
This is an automated email from the ASF dual-hosted git repository.
arvid pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 499b55b50b1e79919cec672548211e244e8243e2
Author: Etienne Chauchot
AuthorDate: Thu Oct 14 12:30:37 2021 +0200
[FLINK-21407][doc][formats] Add parquet formats to DataSet connectors doc
---
.../docs/connectors/dataset/formats/parquet.md | 66 ++
docs/content/docs/connectors/dataset/overview.md | 1 +
2 files changed, 67 insertions(+)
diff --git a/docs/content/docs/connectors/dataset/formats/parquet.md
b/docs/content/docs/connectors/dataset/formats/parquet.md
new file mode 100644
index 000..30c254d
--- /dev/null
+++ b/docs/content/docs/connectors/dataset/formats/parquet.md
@@ -0,0 +1,66 @@
+---
+title: "Parquet"
+weight: 4
+type: docs
+aliases:
+ - /dev/batch/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink has extensive built-in support for [Apache
Parquet](http://parquet.apache.org/). This allows to easily read from Parquet
files with Flink.
+Be sure to include the Flink Parquet dependency to the pom.xml of your project.
+
+```xml
+
+ org.apache.flink
+ flink-parquet_{{< scala_version >}}
+ {{< version >}}
+
+```
+
+In order to read data from a Parquet file, you have to specify one of the
implementation of `ParquetInputFormat`. There are several depending on your
needs:
+- `ParquetPojoInputFormat` to read POJOs from parquet files
+- `ParquetRowInputFormat` to read Flink `Rows` (column oriented records) from
parquet files
+- `ParquetMapInputFormat` to read Map records (Map of nested Flink type
objects) from parquet files
+- `ParquetAvroInputFormat` to read Avro Generic Records from parquet files
+
+
+**Example for ParquetRowInputFormat**:
+
+```java
+MessageType parquetSchema = // use parquet libs to provide the parquet schema
file and parse it or extract it from the parquet files
+ParquetRowInputFormat parquetInputFormat = new ParquetRowInputFormat(new
Path(filePath), parquetSchema);
+// project only needed fields if suited to reduce the amount of data. Use:
parquetSchema#selectFields(projectedFieldNames);
+DataSet input = env.createInput(parquetInputFormat);
+```
+
+**Example for ParquetAvroInputFormat**:
+
+```java
+MessageType parquetSchema = // use parquet libs to provide the parquet schema
file and parse it or extract it from the parquet files
+ParquetAvroInputFormat parquetInputFormat = new ParquetAvroInputFormat(new
Path(filePath), parquetSchema);
+// project only needed fields if suited to reduce the amount of data. Use:
parquetSchema#selectFields(projectedFieldNames);
+DataSet input = env.createInput(parquetInputFormat);
+```
+
+
diff --git a/docs/content/docs/connectors/dataset/overview.md
b/docs/content/docs/connectors/dataset/overview.md
index 7bb2344..87f80884 100644
--- a/docs/content/docs/connectors/dataset/overview.md
+++ b/docs/content/docs/connectors/dataset/overview.md
@@ -40,5 +40,6 @@ Flink supports the following formats:
Microsoft Azure
tale storage format
MongoDb
format
+These formats are gradually replaced with new Flink Source API starting with
Flink 1.14.0.
{{< top >}}
This is an automated email from the ASF dual-hosted git repository.
arvid pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git
commit e3627738314f935ef8ee68988247150328bc525f
Author: Etienne Chauchot
AuthorDate: Tue Oct 19 16:09:37 2021 +0200
[FLINK-21407][doc][formats] Add formats to DataStream connectors doc
---
.../docs/connectors/datastream/formats/_index.md | 23
.../docs/connectors/datastream/formats/avro.md | 61 ++
.../datastream/formats/azure_table_storage.md | 130 +
.../docs/connectors/datastream/formats/hadoop.md | 38 ++
.../docs/connectors/datastream/formats/mongodb.md | 33 ++
.../docs/connectors/datastream/formats/parquet.md | 67 +++
.../content/docs/connectors/datastream/overview.md | 7 +-
7 files changed, 357 insertions(+), 2 deletions(-)
diff --git a/docs/content/docs/connectors/datastream/formats/_index.md
b/docs/content/docs/connectors/datastream/formats/_index.md
new file mode 100644
index 000..282fc69
--- /dev/null
+++ b/docs/content/docs/connectors/datastream/formats/_index.md
@@ -0,0 +1,23 @@
+---
+title: Formats
+bookCollapseSection: true
+weight: 2
+---
+
diff --git a/docs/content/docs/connectors/datastream/formats/avro.md
b/docs/content/docs/connectors/datastream/formats/avro.md
new file mode 100644
index 000..65c0bf9
--- /dev/null
+++ b/docs/content/docs/connectors/datastream/formats/avro.md
@@ -0,0 +1,61 @@
+---
+title: "Avro"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/avro.html
+- /apis/streaming/connectors/formats/avro.html
+---
+
+
+
+# Avro formats
+
+Flink has extensive built-in support for [Apache
Avro](http://avro.apache.org/). This allows to easily read from Avro files with
Flink.
+Also, the serialization framework of Flink is able to handle classes generated
from Avro schemas. Be sure to include the Flink Avro dependency to the pom.xml
of your project.
+
+```xml
+
+ org.apache.flink
+ flink-avro
+ {{< version >}}
+
+```
+
+In order to read data from an Avro file, you have to specify an
`AvroInputFormat`.
+
+**Example**:
+
+```java
+AvroInputFormat users = new AvroInputFormat(in, User.class);
+DataStream usersDS = env.createInput(users);
+```
+
+Note that `User` is a POJO generated by Avro. Flink also allows to perform
string-based key selection of these POJOs. For example:
+
+```java
+usersDS.keyBy("name")
+```
+
+
+Note that using the `GenericData.Record` type is possible with Flink, but not
recommended. Since the record contains the full schema, its very data intensive
and thus probably slow to use.
+
+Flink's POJO field selection also works with POJOs generated from Avro.
However, the usage is only possible if the field types are written correctly to
the generated class. If a field is of type `Object` you can not use the field
as a join or grouping key.
+Specifying a field in Avro like this `{"name": "type_double_test", "type":
"double"},` works fine, however specifying it as a UNION-type with only one
field (`{"name": "type_double_test", "type": ["double"]},`) will generate a
field of type `Object`. Note that specifying nullable types (`{"name":
"type_double_test", "type": ["null", "double"]},`) is possible!
diff --git
a/docs/content/docs/connectors/datastream/formats/azure_table_storage.md
b/docs/content/docs/connectors/datastream/formats/azure_table_storage.md
new file mode 100644
index 000..f5c02c4
--- /dev/null
+++ b/docs/content/docs/connectors/datastream/formats/azure_table_storage.md
@@ -0,0 +1,130 @@
+---
+title: "Microsoft Azure table"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/azure_table_storage.html
+- /apis/streaming/connectors/formats/azure_table_storage.html
+---
+
+
+# Microsoft Azure Table Storage format
+
+_Note: This example works starting from Flink 0.6-incubating_
+
+This example is using the `HadoopInputFormat` wrapper to use an existing
Hadoop input format implementation for accessing [Azure's Table
Storage](https://azure.microsoft.com/en-us/documentation/articles/storage-introduction/).
+
+1. Download and compile the `azure-tables-hadoop` project. The input format
developed by the project is not yet available in Maven Central, therefore, we
have to build the project ourselves.
+ Execute the following commands:
+
+```bash
+git clone https://github.com/mooso/azure-tables-hadoop.git
+cd azure-tables-hadoop
+mvn clean install
+```
+
+2. Setup a new Flink project using the quickstarts:
+
+```bash
+curl https://flink.apache.org/q/quickstart.sh | bash
+```
+
+3. Add the following dependencies (in the `` section) to your
`pom.xml` file:
+
+```xml
+
+ org.apache.flink
+ flink-hadoop-compatibility{{< scala_version >}}
+ {{< version >}}
+
+
+ com.microsoft.hadoop
+ microsoft-hadoop-azure
+ 0.0.4
+
+```
+
+`flink-hadoop-compatibility` is a Flink package that provides the Hadoop input
format wrappers.
+`microsoft-hadoop-azure` is adding the pro
This is an automated email from the ASF dual-hosted git repository.
arvid pushed a change to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git.
from ea53e4a [FLINK-24318][table-planner-blink] Casting a number to
boolean has different results between 'select' fields and 'where' condition
new 85a7f18 [FLINK-21407][doc][formats] Split DataSet connectors page
into different formats and create a formats sub-folder like in table api doc.
new 499b55b [FLINK-21407][doc][formats] Add parquet formats to DataSet
connectors doc
new e362773 [FLINK-21407][doc][formats] Add formats to DataStream
connectors doc
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/content/docs/connectors/dataset/_index.md | 23 ++
.../{table => dataset}/formats/_index.md | 0
.../docs/connectors/dataset/formats/avro.md| 60
.../formats/azure_table_storage.md}| 69 ++
.../docs/connectors/dataset/formats/hadoop.md | 39 +++
.../docs/connectors/dataset/formats/mongodb.md | 31 +
.../docs/connectors/dataset/formats/parquet.md | 66 ++
docs/content/docs/connectors/dataset/overview.md | 45
.../docs/connectors/datastream/formats/_index.md | 23 ++
.../docs/connectors/datastream/formats/avro.md | 61
.../formats/azure_table_storage.md}| 81 --
.../docs/connectors/datastream/formats/hadoop.md | 38 ++
.../docs/connectors/datastream/formats/mongodb.md | 33 +
.../docs/connectors/datastream/formats/parquet.md | 67 ++
.../content/docs/connectors/datastream/overview.md | 7 +-
15 files changed, 508 insertions(+), 135 deletions(-)
create mode 100644 docs/content/docs/connectors/dataset/_index.md
copy docs/content/docs/connectors/{table => dataset}/formats/_index.md (100%)
create mode 100644 docs/content/docs/connectors/dataset/formats/avro.md
copy docs/content/docs/connectors/{dataset.md =>
dataset/formats/azure_table_storage.md} (63%)
create mode 100644 docs/content/docs/connectors/dataset/formats/hadoop.md
create mode 100644 docs/content/docs/connectors/dataset/formats/mongodb.md
create mode 100644 docs/content/docs/connectors/dataset/formats/parquet.md
create mode 100644 docs/content/docs/connectors/dataset/overview.md
create mode 100644 docs/content/docs/connectors/datastream/formats/_index.md
create mode 100644 docs/content/docs/connectors/datastream/formats/avro.md
rename docs/content/docs/connectors/{dataset.md =>
datastream/formats/azure_table_storage.md} (57%)
create mode 100644 docs/content/docs/connectors/datastream/formats/hadoop.md
create mode 100644 docs/content/docs/connectors/datastream/formats/mongodb.md
create mode 100644 docs/content/docs/connectors/datastream/formats/parquet.md
This is an automated email from the ASF dual-hosted git repository.
arvid pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 85a7f18fa5d299b18776addc82c56fe691cc1e66
Author: Etienne Chauchot
AuthorDate: Thu Oct 14 11:28:12 2021 +0200
[FLINK-21407][doc][formats] Split DataSet connectors page into different
formats and create a formats sub-folder like in table api doc.
---
docs/content/docs/connectors/dataset/_index.md | 23
.../docs/connectors/dataset/formats/_index.md | 23
.../docs/connectors/dataset/formats/avro.md| 60 +++
.../formats/azure_table_storage.md}| 69 ++
.../docs/connectors/dataset/formats/hadoop.md | 39
.../docs/connectors/dataset/formats/mongodb.md | 31 ++
docs/content/docs/connectors/dataset/overview.md | 44 ++
7 files changed, 225 insertions(+), 64 deletions(-)
diff --git a/docs/content/docs/connectors/dataset/_index.md
b/docs/content/docs/connectors/dataset/_index.md
new file mode 100644
index 000..3b0d8f6
--- /dev/null
+++ b/docs/content/docs/connectors/dataset/_index.md
@@ -0,0 +1,23 @@
+---
+title: DataSet Connectors
+bookCollapseSection: true
+weight: 1
+---
+
diff --git a/docs/content/docs/connectors/dataset/formats/_index.md
b/docs/content/docs/connectors/dataset/formats/_index.md
new file mode 100644
index 000..8faac16
--- /dev/null
+++ b/docs/content/docs/connectors/dataset/formats/_index.md
@@ -0,0 +1,23 @@
+---
+title: Formats
+bookCollapseSection: true
+weight: 2
+---
+
\ No newline at end of file
diff --git a/docs/content/docs/connectors/dataset/formats/avro.md
b/docs/content/docs/connectors/dataset/formats/avro.md
new file mode 100644
index 000..b0c5572
--- /dev/null
+++ b/docs/content/docs/connectors/dataset/formats/avro.md
@@ -0,0 +1,60 @@
+---
+title: "Avro"
+weight: 4
+type: docs
+aliases:
+ - /dev/batch/connectors/formats/avro.html
+---
+
+
+
+# Avro formats
+
+Flink has extensive built-in support for [Apache
Avro](http://avro.apache.org/). This allows to easily read from Avro files with
Flink.
+Also, the serialization framework of Flink is able to handle classes generated
from Avro schemas. Be sure to include the Flink Avro dependency to the pom.xml
of your project.
+
+```xml
+
+ org.apache.flink
+ flink-avro
+ {{< version >}}
+
+```
+
+In order to read data from an Avro file, you have to specify an
`AvroInputFormat`.
+
+**Example**:
+
+```java
+AvroInputFormat users = new AvroInputFormat(in, User.class);
+DataSet usersDS = env.createInput(users);
+```
+
+Note that `User` is a POJO generated by Avro. Flink also allows to perform
string-based key selection of these POJOs. For example:
+
+```java
+usersDS.groupBy("name")
+```
+
+
+Note that using the `GenericData.Record` type is possible with Flink, but not
recommended. Since the record contains the full schema, its very data intensive
and thus probably slow to use.
+
+Flink's POJO field selection also works with POJOs generated from Avro.
However, the usage is only possible if the field types are written correctly to
the generated class. If a field is of type `Object` you can not use the field
as a join or grouping key.
+Specifying a field in Avro like this `{"name": "type_double_test", "type":
"double"},` works fine, however specifying it as a UNION-type with only one
field (`{"name": "type_double_test", "type": ["double"]},`) will generate a
field of type `Object`. Note that specifying nullable types (`{"name":
"type_double_test", "type": ["null", "double"]},`) is possible!
diff --git a/docs/content/docs/connectors/dataset.md
b/docs/content/docs/connectors/dataset/formats/azure_table_storage.md
similarity index 63%
rename from docs/content/docs/connectors/dataset.md
rename to docs/content/docs/connectors/dataset/formats/azure_table_storage.md
index a4ddca3..406949b 100644
--- a/docs/content/docs/connectors/dataset.md
+++ b/docs/content/docs/connectors/dataset/formats/azure_table_storage.md
@@ -1,9 +1,9 @@
---
-title: "DataSet Connectors"
-weight: 3
+title: "Microsoft Azure table"
+weight: 4
type: docs
aliases:
- - /dev/batch/connectors.html
+- /dev/batch/connectors/formats/azure_table_storage.html
---
-# DataSet Connectors
-
-## Reading from and writing to file systems
-
-The Apache Flink project supports multiple [file systems]({{< ref
"docs/deployment/filesystems/overview" >}}) that can be used as backing stores
-for input and output connectors.
-
-## Connecting to other systems using Input/OutputFormat wrappers for Hadoop
-
-Apache Flink allows users to access many different systems as data sources or
sinks.
-The system is designed for very easy extensibility. Similar to Apache Hadoop,
Flink has the concept
-of so called `InputFormat`s and `OutputFormat`s.
-
-One implementation of these `InputFormat`s is the `HadoopInputFormat`. This is
a wrapper that allows
-users t
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from d14ca21 [FLINK-24560][build][yarn] Copy example jars in
pre-integration-test phase
add 2cb4773 [FLINK-24561][build] Build archetype jars in compile phase
No new revisions were added by this update.
Summary of changes:
flink-quickstart/flink-quickstart-java/pom.xml | 17 +
flink-quickstart/flink-quickstart-scala/pom.xml | 17 +
2 files changed, 34 insertions(+)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from f070f0d [FLINK-24331][tests] Forbid to finish the TestEventSource if
it has not sent at least one value
add d14ca21 [FLINK-24560][build][yarn] Copy example jars in
pre-integration-test phase
No new revisions were added by this update.
Summary of changes:
flink-yarn-tests/pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a change to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 9ed6ad8 [FLINK-24468][runtime] Replaced submit by execute for
submitting buffer size calculation to mailbox
new ccd002f [refactor][tests] Expanded toString for TestEventSource
new 471d682 [FLINK-24331][tests] Forbid to finish the TestEventSource if
it has not sent at least one value
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../operators/lifecycle/event/OperatorFinishedEvent.java | 15 +++
.../operators/lifecycle/graph/TestEventSource.java| 3 ++-
2 files changed, 17 insertions(+), 1 deletion(-)
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a commit to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 471d682eedabb06fe795adf2dc583ce0e6081ec0
Author: Anton Kalashnikov
AuthorDate: Thu Oct 21 16:59:18 2021 +0200
[FLINK-24331][tests] Forbid to finish the TestEventSource if it has not
sent at least one value
---
.../flink/runtime/operators/lifecycle/graph/TestEventSource.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
b/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
index 0d32de9..1badb6a 100644
---
a/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
+++
b/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
@@ -73,7 +73,8 @@ class TestEventSource extends
RichSourceFunction
public void run(SourceContext ctx) {
long lastSent = 0;
while (isRunning) {
-TestCommand cmd = scheduledCommands.poll();
+// Don't finish the source if it has not sent at least one value.
+TestCommand cmd = lastSent == 0 ? null : scheduledCommands.poll();
if (cmd == FINISH_SOURCES) {
ack(cmd);
isRunning = false;
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit f070f0d9ae19275b1e32c2587209d7d8ae87b57e
Author: Anton Kalashnikov
AuthorDate: Thu Oct 21 16:59:18 2021 +0200
[FLINK-24331][tests] Forbid to finish the TestEventSource if it has not
sent at least one value
---
.../flink/runtime/operators/lifecycle/graph/TestEventSource.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
b/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
index 0d32de9..1badb6a 100644
---
a/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
+++
b/flink-tests/src/test/java/org/apache/flink/runtime/operators/lifecycle/graph/TestEventSource.java
@@ -73,7 +73,8 @@ class TestEventSource extends
RichSourceFunction
public void run(SourceContext ctx) {
long lastSent = 0;
while (isRunning) {
-TestCommand cmd = scheduledCommands.poll();
+// Don't finish the source if it has not sent at least one value.
+TestCommand cmd = lastSent == 0 ? null : scheduledCommands.poll();
if (cmd == FINISH_SOURCES) {
ack(cmd);
isRunning = false;
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 86c9694 [FLINK-24019][build][dist] Package Scala APIs separately
new 63bd6a3 [refactor][tests] Expanded toString for TestEventSource
new f070f0d [FLINK-24331][tests] Forbid to finish the TestEventSource if
it has not sent at least one value
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../operators/lifecycle/event/OperatorFinishedEvent.java | 15 +++
.../operators/lifecycle/graph/TestEventSource.java| 3 ++-
2 files changed, 17 insertions(+), 1 deletion(-)
