[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312780843 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -1,5 +1,5 @@ --- -title: "Apache Kafka Connector" +title: "Apache Kafka 连接器" Review comment: Yes. We have a consensus on connector. https://cwiki.apache.org/confluence/display/FLINK/Flink+Translation+Specifications This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9342: [FLINK-13438][hive] Fix DataTypes.DATE/TIME/TIMESTAMP support for hive connectors
flinkbot edited a comment on issue #9342: [FLINK-13438][hive] Fix DataTypes.DATE/TIME/TIMESTAMP support for hive connectors URL: https://github.com/apache/flink/pull/9342#issuecomment-517770642 ## CI report: * 76704f271662b57cbe36679d3d249bcdd7fdf66a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121784366) * 7b4a9226cfffc1ea505c8d20b5b5f9ce8c5d2113 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122239651) * ec81369c4e332d9290a2b42e386f9be724d8e2ad : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122307651) * b2d4875b20874041f90db3473010cf454a2cba66 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122365586) * 83860f4cb617d777093dce251e4145c5e8f79e7f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122423996) * 8e2e5fee6859c7cfa1e3cfcc6f5d3bfe0dd8edbc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122540373) * 01b029b137545f281e4b3209cc6e663d0692d71a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122743888) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
TisonKun commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312777886 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -1,5 +1,5 @@ --- -title: "Apache Kafka Connector" +title: "Apache Kafka 连接器" Review comment: @wuchong shall we translate proper nouns such as "connector"? Do we have a general translation with consensus or at least listed out in Glossary? IMHO there is no need to translate "connector". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13685) YARN deployment hangs when requested container is above the maximum vcore size
[ https://issues.apache.org/jira/browse/FLINK-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated FLINK-13685: -- Description: When deploying on YARN flink explicitly checks whether the requested container vcores is smaller than yarn.nodemanager.resource.cpu-vcores. Unfortunately this doesn't guarantee that the request can be fulfilled if yarn.scheduler.maximum-allocation-vcores is set to a lower number. In these cases flink will just keep trying to request a container but will never succeed. was: When deploying on YARN flink explicitly checks whether the requested container memory is smaller than yarn.nodemanager.resource.cpu-vcores. Unfortunately this doesn't guarantee that the request can be fulfilled if yarn.scheduler.maximum-allocation-vcores is set to a lower number. In these cases flink will just keep trying to request a container but will never succeed. > YARN deployment hangs when requested container is above the maximum vcore size > -- > > Key: FLINK-13685 > URL: https://issues.apache.org/jira/browse/FLINK-13685 > Project: Flink > Issue Type: New Feature > Components: Deployment / YARN >Affects Versions: 1.8.1 >Reporter: Gyula Fora >Priority: Major > > When deploying on YARN flink explicitly checks whether the requested > container vcores is smaller than yarn.nodemanager.resource.cpu-vcores. > Unfortunately this doesn't guarantee that the request can be fulfilled if > yarn.scheduler.maximum-allocation-vcores is set to a lower number. > In these cases flink will just keep trying to request a container but will > never succeed. > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13685) YARN deployment hangs when requested container is above the maximum vcore size
[ https://issues.apache.org/jira/browse/FLINK-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904830#comment-16904830 ] Yang Wang commented on FLINK-13685: --- Hi [~gyfora], Thanks for creating this ticket. You are right. Currently we only check whether the requested vcores is smaller than the max vcores of each NodeManager in the Yarn cluster. It is not enough when the yarn.scheduler.maximum-allocation-vcores is smaller. The requested vcores should be also checked in AbstractYarnClusterDescriptor#validateClusterResources() like memory. Will you attach a PR to fix this? > YARN deployment hangs when requested container is above the maximum vcore size > -- > > Key: FLINK-13685 > URL: https://issues.apache.org/jira/browse/FLINK-13685 > Project: Flink > Issue Type: New Feature > Components: Deployment / YARN >Affects Versions: 1.8.1 >Reporter: Gyula Fora >Priority: Major > > When deploying on YARN flink explicitly checks whether the requested > container memory is smaller than yarn.nodemanager.resource.cpu-vcores. > Unfortunately this doesn't guarantee that the request can be fulfilled if > yarn.scheduler.maximum-allocation-vcores is set to a lower number. > In these cases flink will just keep trying to request a container but will > never succeed. > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13683) Translate "Code Style - Component Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904829#comment-16904829 ] Jark Wu commented on FLINK-13683: - Sure. I assigned to you. > Translate "Code Style - Component Guide" page into Chinese > -- > > Key: FLINK-13683 > URL: https://issues.apache.org/jira/browse/FLINK-13683 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: ChaojianZhang >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-components.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-components.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (FLINK-13683) Translate "Code Style - Component Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-13683: --- Assignee: ChaojianZhang > Translate "Code Style - Component Guide" page into Chinese > -- > > Key: FLINK-13683 > URL: https://issues.apache.org/jira/browse/FLINK-13683 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: ChaojianZhang >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-components.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-components.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] theopenlab-ci[bot] commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition
theopenlab-ci[bot] commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416#issuecomment-520291310 Build succeeded. - [flink-build-and-test-arm64-core ](https://logs.openlabtesting.org/logs/16/9416/dbf749f0d0c3167265287b182dd1ea7b4aed84c1/check/flink-build-and-test-arm64-core/87cfc59/) : FAILURE in 3m 34s (non-voting) - [flink-build-and-test-arm64-tests ](https://logs.openlabtesting.org/logs/16/9416/dbf749f0d0c3167265287b182dd1ea7b4aed84c1/check/flink-build-and-test-arm64-tests/3e0c238/) : FAILURE in 3m 54s (non-voting) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wangxiyuan commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition
wangxiyuan commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416#issuecomment-520290844 recheck This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13688) HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed with 1.9.0-rc2
[
https://issues.apache.org/jira/browse/FLINK-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Young updated FLINK-13688:
---
Environment:
Linux server
kernal version: 3.10.0
java version: "1.8.0_102"
processor count: 96
was:
Linux server, kernal 3.10.0
java version "1.8.0_102"
> HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed with 1.9.0-rc2
> ---
>
> Key: FLINK-13688
> URL: https://issues.apache.org/jira/browse/FLINK-13688
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive, Tests
>Affects Versions: 1.9.0
> Environment: Linux server
> kernal version: 3.10.0
> java version: "1.8.0_102"
> processor count: 96
>Reporter: Kurt Young
>Assignee: Jingsong Lee
>Priority: Blocker
> Fix For: 1.9.0
>
>
> I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
> server, HiveCatalogUseBlinkITCase.testBlinkUdf will be constantly fail.
>
> Fail trace:
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 313.228 s <<< FAILURE! - in
> org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase
> [ERROR]
> testBlinkUdf(org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase)
> Time elapsed: 305.155 s <<< ERROR!
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at
> org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase.testBlinkUdf(HiveCatalogUseBlinkITCase.java:180)
> Caused by:
> org.apache.flink.runtime.resourcemanager.exceptions.UnfulfillableSlotRequestException:
> Could not fulfill slot request 35cf6fdc1b525de9b6eed13894e2e31d. Requested
> resource profile (ResourceProfile{cpuCores=0.0, heapMemoryInMB=0,
> directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0,
> managedMemoryInMB=128}) is unfulfillable.
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (FLINK-13687) elasticsearch5.ElasticsearchSinkITCase constantly fail with 1.9.0-rc2
[
https://issues.apache.org/jira/browse/FLINK-13687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Young updated FLINK-13687:
---
Environment:
Linux server
kernal version: 3.10.0
java version: "1.8.0_102"
processor count: 96
was:
Linux server, kernal 3.10.0
java version "1.8.0_102"
> elasticsearch5.ElasticsearchSinkITCase constantly fail with 1.9.0-rc2
> -
>
> Key: FLINK-13687
> URL: https://issues.apache.org/jira/browse/FLINK-13687
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ElasticSearch, Tests
>Affects Versions: 1.9.0
> Environment: Linux server
> kernal version: 3.10.0
> java version: "1.8.0_102"
> processor count: 96
>Reporter: Kurt Young
>Priority: Blocker
> Fix For: 1.9.0
>
>
> I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
> server, elasticsearch5.ElasticsearchSinkITCase will be constantly fail.
>
> failed trace:
> {code:java}
> 1528 [main] INFO
> org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase
> - -
> 1528 [main] INFO
> org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase
> - Starting embedded Elasticsearch node
> 1528 [main] INFO
> org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase
> - -
> ERROR StatusLogger Unable to create class
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
> specified in
> file:/dump/1/kete.yangkt/flink-1.9.0-rc2/flink-1.9.0/flink-connectors/flink-connector-elasticsearch5/target/classes/META-INF/log4j-provider.properties
> java.lang.ClassNotFoundException:
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:448)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
> at
> org.apache.logging.log4j.spi.Provider.loadLoggerContextFactory(Provider.java:96)
> at org.apache.logging.log4j.LogManager.(LogManager.java:91)
> at
> org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.java:49)
> at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:105)
> at org.elasticsearch.node.Node.(Node.java:237)
> at
> org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl$PluginNode.(EmbeddedElasticsearchNodeEnvironmentImpl.java:78)
> at
> org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl.start(EmbeddedElasticsearchNodeEnvironmentImpl.java:54)
> at
> org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase.prepare(ElasticsearchSinkTestBase.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
[jira] [Commented] (FLINK-13683) Translate "Code Style - Component Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904820#comment-16904820 ] ChaojianZhang commented on FLINK-13683: --- Hi [~jark], can you assign this to me? I want to translate this page. > Translate "Code Style - Component Guide" page into Chinese > -- > > Key: FLINK-13683 > URL: https://issues.apache.org/jira/browse/FLINK-13683 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-components.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-components.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13688) HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed with 1.9.0-rc2
Kurt Young created FLINK-13688:
--
Summary: HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed
with 1.9.0-rc2
Key: FLINK-13688
URL: https://issues.apache.org/jira/browse/FLINK-13688
Project: Flink
Issue Type: Bug
Components: Connectors / Hive, Tests
Affects Versions: 1.9.0
Environment: Linux server, kernal 3.10.0
java version "1.8.0_102"
Reporter: Kurt Young
Assignee: Jingsong Lee
Fix For: 1.9.0
I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
server, HiveCatalogUseBlinkITCase.testBlinkUdf will be constantly fail.
Fail trace:
{code:java}
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 313.228
s <<< FAILURE! - in
org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase
[ERROR]
testBlinkUdf(org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase)
Time elapsed: 305.155 s <<< ERROR!
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at
org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase.testBlinkUdf(HiveCatalogUseBlinkITCase.java:180)
Caused by:
org.apache.flink.runtime.resourcemanager.exceptions.UnfulfillableSlotRequestException:
Could not fulfill slot request 35cf6fdc1b525de9b6eed13894e2e31d. Requested
resource profile (ResourceProfile{cpuCores=0.0, heapMemoryInMB=0,
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0,
managedMemoryInMB=128}) is unfulfillable.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13687) elasticsearch5.ElasticsearchSinkITCase constantly fail with 1.9.0-rc2
Kurt Young created FLINK-13687:
--
Summary: elasticsearch5.ElasticsearchSinkITCase constantly fail
with 1.9.0-rc2
Key: FLINK-13687
URL: https://issues.apache.org/jira/browse/FLINK-13687
Project: Flink
Issue Type: Bug
Components: Connectors / ElasticSearch, Tests
Affects Versions: 1.9.0
Environment: Linux server, kernal 3.10.0
java version "1.8.0_102"
Reporter: Kurt Young
Fix For: 1.9.0
I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
server, elasticsearch5.ElasticsearchSinkITCase will be constantly fail.
failed trace:
{code:java}
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
Starting embedded Elasticsearch node
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
ERROR StatusLogger Unable to create class
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
specified in
file:/dump/1/kete.yangkt/flink-1.9.0-rc2/flink-1.9.0/flink-connectors/flink-connector-elasticsearch5/target/classes/META-INF/log4j-provider.properties
java.lang.ClassNotFoundException:
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:448)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
at
org.apache.logging.log4j.spi.Provider.loadLoggerContextFactory(Provider.java:96)
at org.apache.logging.log4j.LogManager.(LogManager.java:91)
at
org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.java:49)
at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:105)
at org.elasticsearch.node.Node.(Node.java:237)
at
org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl$PluginNode.(EmbeddedElasticsearchNodeEnvironmentImpl.java:78)
at
org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl.start(EmbeddedElasticsearchNodeEnvironmentImpl.java:54)
at
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase.prepare(ElasticsearchSinkTestBase.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
ERROR StatusLogger No log4j2 configuration file found. Using default
configuration: logging only errors to the console.
2546 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
2546 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
Shutting down embedded Elasticsearch node
2546 [main] INFO
[GitHub] [flink] bowenli86 commented on issue #9399: [FLINK-13526][sql-client] Switching to a non existing catalog or data…
bowenli86 commented on issue #9399: [FLINK-13526][sql-client] Switching to a non existing catalog or data… URL: https://github.com/apache/flink/pull/9399#issuecomment-520288021 is there a contract that throwing SqlExecutionException guarantees executor won't crash? if so, we probably don't need extra test anymore. BTW, the example is named as `testUseNonExistingDatabaseDoesntCrush()` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9349: [FLINK-13564] [table-planner-blink] throw exception if constant with YEAR TO MONTH resolution was used for group windows
wuchong commented on a change in pull request #9349: [FLINK-13564] [table-planner-blink] throw exception if constant with YEAR TO MONTH resolution was used for group windows URL: https://github.com/apache/flink/pull/9349#discussion_r312771076 ## File path: flink-table/flink-table-planner-blink/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/WindowAggregateTest.xml ## @@ -131,6 +131,27 @@ Calc(select=[EXPR$0, wAvg, w$start AS EXPR$2, w$end AS EXPR$3]) +- Exchange(distribution=[single]) +- Calc(select=[proctime, c, a]) +- DataStreamScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c, proctime, rowtime]) +]]> + + + + + + + + + + +
[GitHub] [flink] asfgit closed pull request #9347: [FLINK-13563] [table-planner-blink] TumblingGroupWindow should implement toString method to explain more info
asfgit closed pull request #9347: [FLINK-13563] [table-planner-blink] TumblingGroupWindow should implement toString method to explain more info URL: https://github.com/apache/flink/pull/9347 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13686) Unstable download Kafka lead to Kafka e2e test failed
[
https://issues.apache.org/jira/browse/FLINK-13686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904810#comment-16904810
]
Kurt Young commented on FLINK-13686:
I also encountered this issue when running e2e tests for 1.9.0-rc2. I'm
wondering if we can support pre download all needed packages, save them in a
local directory, and directly use them instead of downloading them every time.
> Unstable download Kafka lead to Kafka e2e test failed
> -
>
> Key: FLINK-13686
> URL: https://issues.apache.org/jira/browse/FLINK-13686
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Reporter: Jark Wu
>Priority: Major
>
> Kafka e2e test will download Kafka binary first. However, I the download is
> not stable and usually consume much test time.
> {code}
> ==
> Running 'Kafka 0.10 end-to-end test'
> ==
> TEST_DATA_DIR:
> /home/travis/build/flink-ci/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-07823387384
> Flink dist directory:
> /home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT
> Downloading Kafka from
> https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 0 00 00 0 0 0 --:--:-- 0:02:10 --:--:--
> 0curl: (7) Failed to connect to archive.apache.org port 443: Connection timed
> out
> [FAIL] Test script contains errors.
> Checking for errors...
> No errors in log files.
> Checking for exceptions...
> No exceptions in log files.
> Checking for non-empty .out files...
> grep:
> /home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT/log/*.out:
> No such file or directory
> No non-empty .out files.
> [FAIL] 'Kafka 0.10 end-to-end test' failed after 2 minutes and 11 seconds!
> Test exited with exit code 1
> {code}
> Here is an instance: https://api.travis-ci.com/v3/job/223824142/log.txt
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (FLINK-13563) TumblingGroupWindow should implement toString method
[
https://issues.apache.org/jira/browse/FLINK-13563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu resolved FLINK-13563.
-
Resolution: Fixed
Assignee: godfrey he
Fix Version/s: 1.9.0
master: 0cda582487372d3f2717cb4fc7b0f8f818e18d03
1.9.0: d74f35216a95d502a942e7492098e0663931e3a3
> TumblingGroupWindow should implement toString method
>
>
> Key: FLINK-13563
> URL: https://issues.apache.org/jira/browse/FLINK-13563
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.9.0, 1.10.0
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:scala}
> @Test
> def testAllEventTimeTumblingGroupWindowOverTime(): Unit = {
> val util = streamTestUtil()
> val table = util.addDataStream[(Long, Int, String)](
> "T1", 'long, 'int, 'string, 'rowtime.rowtime)
> val windowedTable = table
> .window(Tumble over 5.millis on 'rowtime as 'w)
> .groupBy('w)
> .select('int.count)
> util.verifyPlan(windowedTable)
> }
> {code}
> currently, it's physical plan is
> {code:java}
> HashWindowAggregate(window=[TumblingGroupWindow],
> select=[Final_COUNT(count$0) AS EXPR$0])
> +- Exchange(distribution=[single])
>+- LocalHashWindowAggregate(window=[TumblingGroupWindow],
> select=[Partial_COUNT(int) AS count$0])
> +- TableSourceScan(table=[[default_catalog, default_database, Table1,
> source: [TestTableSource(long, int, string)]]], fields=[long, int, string])
> {code}
> we know nothing about the TumblingGroupWindow except its name. the expected
> plan is
> {code:java}
> HashWindowAggregate(window=[TumblingGroupWindow('w, long, 5)],
> select=[Final_COUNT(count$0) AS EXPR$0])
> +- Exchange(distribution=[single])
>+- LocalHashWindowAggregate(window=[TumblingGroupWindow('w, long, 5)],
> select=[Partial_COUNT(int) AS count$0])
> +- TableSourceScan(table=[[default_catalog, default_database, Table1,
> source: [TestTableSource(long, int, string)]]], fields=[long, int, string])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9147: [FLINK-11631][test] Harden TaskExecutorITCase
flinkbot edited a comment on issue #9147: [FLINK-11631][test] Harden TaskExecutorITCase URL: https://github.com/apache/flink/pull/9147#issuecomment-512249534 ## CI report: * 8b7412c2b24f4025939b13e86780897465c452b2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119472379) * e84b908966241b25969d93c3967e430a34230ec4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119498986) * 76c31d134c95282ebdef09c6668e3886154200e6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120365351) * 7bdc6e47f473d37cdf776e9e2feaa954c57cd8d6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/121059725) * 15768441c26b7dde3d9b736b749982dca58c3551 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122052374) * b8f805723eb02d31f86b0a9cbae00e4b4587f385 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122738636) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13686) Unstable download Kafka lead to Kafka e2e test failed
[
https://issues.apache.org/jira/browse/FLINK-13686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-13686:
Description:
Kafka e2e test will download Kafka binary first. However, I the download is not
stable and usually consume much test time.
{code}
==
Running 'Kafka 0.10 end-to-end test'
==
TEST_DATA_DIR:
/home/travis/build/flink-ci/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-07823387384
Flink dist directory:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT
Downloading Kafka from
https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- 0:02:10 --:--:--
0curl: (7) Failed to connect to archive.apache.org port 443: Connection timed
out
[FAIL] Test script contains errors.
Checking for errors...
No errors in log files.
Checking for exceptions...
No exceptions in log files.
Checking for non-empty .out files...
grep:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT/log/*.out:
No such file or directory
No non-empty .out files.
[FAIL] 'Kafka 0.10 end-to-end test' failed after 2 minutes and 11 seconds! Test
exited with exit code 1
{code}
Here is an instance: https://api.travis-ci.com/v3/job/223824142/log.txt
was:
Kafka e2e test will download Kafka binary first. However, I the download is not
stable and usually consume much test time.
{code}
==
Running 'Kafka 0.10 end-to-end test'
==
TEST_DATA_DIR:
/home/travis/build/flink-ci/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-07823387384
Flink dist directory:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT
Downloading Kafka from
https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- 0:02:10 --:--:--
0curl: (7) Failed to connect to archive.apache.org port 443: Connection timed
out
[FAIL] Test script contains errors.
Checking for errors...
No errors in log files.
Checking for exceptions...
No exceptions in log files.
Checking for non-empty .out files...
grep:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT/log/*.out:
No such file or directory
No non-empty .out files.
[FAIL] 'Kafka 0.10 end-to-end test' failed after 2 minutes and 11 seconds! Test
exited with exit code 1
{code}
> Unstable download Kafka lead to Kafka e2e test failed
> -
>
> Key: FLINK-13686
> URL: https://issues.apache.org/jira/browse/FLINK-13686
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Reporter: Jark Wu
>Priority: Major
>
> Kafka e2e test will download Kafka binary first. However, I the download is
> not stable and usually consume much test time.
> {code}
> ==
> Running 'Kafka 0.10 end-to-end test'
> ==
> TEST_DATA_DIR:
> /home/travis/build/flink-ci/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-07823387384
> Flink dist directory:
> /home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT
> Downloading Kafka from
> https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 0 00 00 0 0 0 --:--:-- 0:02:10 --:--:--
> 0curl: (7) Failed to connect to archive.apache.org port 443: Connection timed
> out
> [FAIL] Test script contains errors.
> Checking for errors...
> No errors in log files.
> Checking for exceptions...
> No exceptions in log files.
> Checking for non-empty .out files...
> grep:
> /home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT/log/*.out:
> No such file or directory
> No non-empty .out files.
> [FAIL] 'Kafka 0.10 end-to-end test' failed after 2 minutes and 11 seconds!
> Test
[jira] [Created] (FLINK-13686) Unstable download Kafka lead to Kafka e2e test failed
Jark Wu created FLINK-13686:
---
Summary: Unstable download Kafka lead to Kafka e2e test failed
Key: FLINK-13686
URL: https://issues.apache.org/jira/browse/FLINK-13686
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Reporter: Jark Wu
Kafka e2e test will download Kafka binary first. However, I the download is not
stable and usually consume much test time.
{code}
==
Running 'Kafka 0.10 end-to-end test'
==
TEST_DATA_DIR:
/home/travis/build/flink-ci/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-07823387384
Flink dist directory:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT
Downloading Kafka from
https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- 0:02:10 --:--:--
0curl: (7) Failed to connect to archive.apache.org port 443: Connection timed
out
[FAIL] Test script contains errors.
Checking for errors...
No errors in log files.
Checking for exceptions...
No exceptions in log files.
Checking for non-empty .out files...
grep:
/home/travis/build/flink-ci/flink/flink-dist/target/flink-1.10-SNAPSHOT-bin/flink-1.10-SNAPSHOT/log/*.out:
No such file or directory
No non-empty .out files.
[FAIL] 'Kafka 0.10 end-to-end test' failed after 2 minutes and 11 seconds! Test
exited with exit code 1
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition
flinkbot commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416#issuecomment-520285701 ## CI report: * dbf749f0d0c3167265287b182dd1ea7b4aed84c1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122738631) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9147: [FLINK-11631][test] Harden TaskExecutorITCase
flinkbot edited a comment on issue #9147: [FLINK-11631][test] Harden TaskExecutorITCase URL: https://github.com/apache/flink/pull/9147#issuecomment-512249534 ## CI report: * 8b7412c2b24f4025939b13e86780897465c452b2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119472379) * e84b908966241b25969d93c3967e430a34230ec4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119498986) * 76c31d134c95282ebdef09c6668e3886154200e6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120365351) * 7bdc6e47f473d37cdf776e9e2feaa954c57cd8d6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/121059725) * 15768441c26b7dde3d9b736b749982dca58c3551 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122052374) * b8f805723eb02d31f86b0a9cbae00e4b4587f385 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122738636) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] theopenlab-ci[bot] commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition
theopenlab-ci[bot] commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416#issuecomment-520285403 Build succeeded. - [flink-build-and-test-arm64-core ](https://logs.openlabtesting.org/logs/16/9416/dbf749f0d0c3167265287b182dd1ea7b4aed84c1/check/flink-build-and-test-arm64-core/d199b69/) : FAILURE in 4m 09s (non-voting) - [flink-build-and-test-arm64-tests ](https://logs.openlabtesting.org/logs/16/9416/dbf749f0d0c3167265287b182dd1ea7b4aed84c1/check/flink-build-and-test-arm64-tests/ac0a1a5/) : FAILURE in 4m 08s (non-voting) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13646) Add ARM CI job definition scripts
[ https://issues.apache.org/jira/browse/FLINK-13646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904802#comment-16904802 ] wangxiyuan commented on FLINK-13646: [~Zentol] Done And could anybody assign this ticket to me? Thanks very much. > Add ARM CI job definition scripts > - > > Key: FLINK-13646 > URL: https://issues.apache.org/jira/browse/FLINK-13646 > Project: Flink > Issue Type: Sub-task > Components: Build System >Affects Versions: 2.0.0 >Reporter: wangxiyuan >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > OpenLab CI github-app now is added to Flink repo. It's a good time to add the > related ARM job definition scripts now. > OpenLab uses zuul[1] as the CI infrastructure which uses ansible[2] for job > execution and definition. > The ansible scripts for job definition is always contained in a file named > *.zuul.yaml*. Here is an example[3]. > So for Flink ARM support work, I suggest to devide it into some steps(we will > add *flink-core* and *flink-test* related modules as the beginner. Other > modules can be added later if people want them): > # Add the basic *build* script to ensure the CI system and build job works > as expect. The job should be marked as non-voting first, it means the CI test > failure won't block Flink PR to be merged. > # Add the *test* script to run unit/intergration test. At this step the > *--fn* parameter will be added to *mvn test*. It will run the full test cases > in Flink, so that we can find what test is failed on ARM. > # Fix the test failure one by one. > # Once all the tests are passed, remove the *--fn* parameter and keep watch > the CI's status for some days. If some bugs raise then, fix them as what we > usually do for travis-ci. > # Once the CI is stable enought, remove the non-voting tag, so that the ARM > CI will be the same as travis-ci, to be one of the gate for Flink PR. > # Finally, Flink community can announce and release Flink ARM version. > OpenLab will keep helping and maintaining the ARM work. If you have any > question or requirement, welcome to job IRC channel: #askopenlab > Any thought? > Thanks. > [1]: [https://zuul-ci.org/docs/zuul/] > [2]: [https://www.ansible.com/] > [3]: [https://github.com/theopenlab/flink/pull/1/files] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition
flinkbot commented on issue #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416#issuecomment-520285134 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit dbf749f0d0c3167265287b182dd1ea7b4aed84c1 (Mon Aug 12 02:36:01 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13646).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) * b2cd904545fba2ff99d19ab81db9783294fc762a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717406) * 61f710f8c259610b0bb1b48131063d3c08b4980d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122717629) * 7e1b565d83be8c3009e666aaefdbbfed778caeac : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122735611) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13646) Add ARM CI job definition scripts
[ https://issues.apache.org/jira/browse/FLINK-13646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13646: --- Labels: pull-request-available (was: ) > Add ARM CI job definition scripts > - > > Key: FLINK-13646 > URL: https://issues.apache.org/jira/browse/FLINK-13646 > Project: Flink > Issue Type: Sub-task > Components: Build System >Affects Versions: 2.0.0 >Reporter: wangxiyuan >Priority: Major > Labels: pull-request-available > > OpenLab CI github-app now is added to Flink repo. It's a good time to add the > related ARM job definition scripts now. > OpenLab uses zuul[1] as the CI infrastructure which uses ansible[2] for job > execution and definition. > The ansible scripts for job definition is always contained in a file named > *.zuul.yaml*. Here is an example[3]. > So for Flink ARM support work, I suggest to devide it into some steps(we will > add *flink-core* and *flink-test* related modules as the beginner. Other > modules can be added later if people want them): > # Add the basic *build* script to ensure the CI system and build job works > as expect. The job should be marked as non-voting first, it means the CI test > failure won't block Flink PR to be merged. > # Add the *test* script to run unit/intergration test. At this step the > *--fn* parameter will be added to *mvn test*. It will run the full test cases > in Flink, so that we can find what test is failed on ARM. > # Fix the test failure one by one. > # Once all the tests are passed, remove the *--fn* parameter and keep watch > the CI's status for some days. If some bugs raise then, fix them as what we > usually do for travis-ci. > # Once the CI is stable enought, remove the non-voting tag, so that the ARM > CI will be the same as travis-ci, to be one of the gate for Flink PR. > # Finally, Flink community can announce and release Flink ARM version. > OpenLab will keep helping and maintaining the ARM work. If you have any > question or requirement, welcome to job IRC channel: #askopenlab > Any thought? > Thanks. > [1]: [https://zuul-ci.org/docs/zuul/] > [2]: [https://www.ansible.com/] > [3]: [https://github.com/theopenlab/flink/pull/1/files] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] wangxiyuan opened a new pull request #9416: [FLINK-13646][build system] Add basic ARM CI job definition
wangxiyuan opened a new pull request #9416: [FLINK-13646][build system] Add basic ARM CI job definition URL: https://github.com/apache/flink/pull/9416 ## What is the purpose of the change We plan some steps for FLINK ARM support, taking *flink-core* and *flink-test* related modules as the first targets: 1. Add the basic build script to ensure the CI system and build job works as expect. The job should be marked as non-voting first, it means the CI test failure won't block Flink PR to be merged. 2. Add the test script to run unit/intergration test. At this step the --fn parameter will be added to mvn test. It will run the full test cases in Flink, so that we can find what test is failed on ARM. 3. Fix the test failure one by one. 4. Once all the tests are passed, remove the --fn parameter and keep watch the CI's status for some days. If some bugs raise then, fix them as what we usually do for travis-ci. 5. Once the CI is stable enought, remove the non-voting tag, so that the ARM CI will be the same as travis-ci, to be one of the gate for Flink PR. 6. Finally, Flink community can announce and release Flink ARM version. This PR is the first step that adding the basic ARM CI job definition script for *flink-core* and *flink-test* build. ## Brief change log - Add job definition file (.zuul.yaml). - Add job job execution file (files in openlab folder). ## Verifying this change This change added new CI and can be verified through the CI report. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-13562) throws exception when FlinkRelMdColumnInterval meets two stage stream group aggregate
[
https://issues.apache.org/jira/browse/FLINK-13562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu resolved FLINK-13562.
-
Resolution: Fixed
Fix Version/s: 1.9.0
master: 0b65aeaede36f0bc706dfdb82f039352b15069e9
1.9.0: bca6330fce03bc177ca48f142d786ce2e79c6c45
> throws exception when FlinkRelMdColumnInterval meets two stage stream group
> aggregate
> -
>
> Key: FLINK-13562
> URL: https://issues.apache.org/jira/browse/FLINK-13562
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> test case:
> {code:scala}
> @Test
> def testTwoDistinctAggregateWithNonDistinctAgg(): Unit = {
> util.addTableSource[(Int, Long, String)]("MyTable", 'a, 'b, 'c)
> util.verifyPlan("SELECT c, SUM(DISTINCT a), SUM(a), COUNT(DISTINCT b)
> FROM MyTable GROUP BY c")
> }
> {code}
> org.apache.flink.table.api.TableException: Sum aggregate function does not
> support type: ''VARCHAR''.
> Please re-check the data type.
> at
> org.apache.flink.table.planner.plan.utils.AggFunctionFactory.createSumAggFunction(AggFunctionFactory.scala:191)
> at
> org.apache.flink.table.planner.plan.utils.AggFunctionFactory.createAggFunction(AggFunctionFactory.scala:74)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$9.apply(AggregateUtil.scala:285)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$9.apply(AggregateUtil.scala:279)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$.transformToAggregateInfoList(AggregateUtil.scala:279)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$.getOutputIndexToAggCallIndexMap(AggregateUtil.scala:154)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdColumnInterval.getAggCallIndexInLocalAgg$1(FlinkRelMdColumnInterval.scala:504)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdColumnInterval.estimateColumnIntervalOfAggregate(FlinkRelMdColumnInterval.scala:526)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdColumnInterval.getColumnInterval(FlinkRelMdColumnInterval.scala:417)
> at GeneratedMetadataHandler_ColumnInterval.getColumnInterval_$(Unknown
> Source)
> at GeneratedMetadataHandler_ColumnInterval.getColumnInterval(Unknown
> Source)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMetadataQuery.getColumnInterval(FlinkRelMetadataQuery.java:122)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] asfgit closed pull request #9346: [FLINK-13562] [table-planner-blink] fix incorrect input type for local stream group aggregate in FlinkRelMdColumnInterval
asfgit closed pull request #9346: [FLINK-13562] [table-planner-blink] fix incorrect input type for local stream group aggregate in FlinkRelMdColumnInterval URL: https://github.com/apache/flink/pull/9346 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13405) Translate "Basic API Concepts" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904792#comment-16904792 ] Jark Wu commented on FLINK-13405: - Yes. Please use English. Thanks. > Translate "Basic API Concepts" page into Chinese > > > Key: FLINK-13405 > URL: https://issues.apache.org/jira/browse/FLINK-13405 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation >Affects Versions: 1.10.0 >Reporter: WangHengWei >Assignee: WangHengWei >Priority: Major > Labels: documentation, pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The page url is > [https://github.com/apache/flink/blob/master/docs/dev/api_concepts.zh.md] > The markdown file is located in flink/docs/dev/api_concepts.zh.md -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] wuchong commented on issue #9346: [FLINK-13562] [table-planner-blink] fix incorrect input type for local stream group aggregate in FlinkRelMdColumnInterval
wuchong commented on issue #9346: [FLINK-13562] [table-planner-blink] fix incorrect input type for local stream group aggregate in FlinkRelMdColumnInterval URL: https://github.com/apache/flink/pull/9346#issuecomment-520281538 Merging This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13405) Translate "Basic API Concepts" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904786#comment-16904786 ] WangHengWei commented on FLINK-13405: - Should I send a message with a headline beginning with [Discuss] to d...@flink.apache.com? > Translate "Basic API Concepts" page into Chinese > > > Key: FLINK-13405 > URL: https://issues.apache.org/jira/browse/FLINK-13405 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation >Affects Versions: 1.10.0 >Reporter: WangHengWei >Assignee: WangHengWei >Priority: Major > Labels: documentation, pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The page url is > [https://github.com/apache/flink/blob/master/docs/dev/api_concepts.zh.md] > The markdown file is located in flink/docs/dev/api_concepts.zh.md -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (FLINK-13679) Translate "Code Style - Pull Requests & Changes" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-13679: --- Assignee: LakeShen > Translate "Code Style - Pull Requests & Changes" page into Chinese > -- > > Key: FLINK-13679 > URL: https://issues.apache.org/jira/browse/FLINK-13679 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: LakeShen >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-pull-requests.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-pull-requests.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13679) Translate "Code Style - Pull Requests & Changes" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904780#comment-16904780 ] Jark Wu commented on FLINK-13679: - Sure. I assigned to you. > Translate "Code Style - Pull Requests & Changes" page into Chinese > -- > > Key: FLINK-13679 > URL: https://issues.apache.org/jira/browse/FLINK-13679 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: LakeShen >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-pull-requests.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-pull-requests.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13679) Translate "Code Style - Pull Requests & Changes" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904778#comment-16904778 ] LakeShen commented on FLINK-13679: -- Hi Jark , could you assign this issue to me , I want to translate this page . I will finish it on time , thanks. > Translate "Code Style - Pull Requests & Changes" page into Chinese > -- > > Key: FLINK-13679 > URL: https://issues.apache.org/jira/browse/FLINK-13679 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-pull-requests.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-pull-requests.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) * b2cd904545fba2ff99d19ab81db9783294fc762a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717406) * 61f710f8c259610b0bb1b48131063d3c08b4980d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122717629) * 7e1b565d83be8c3009e666aaefdbbfed778caeac : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122735611) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13002) Expand Concept -> Glossary Section
[ https://issues.apache.org/jira/browse/FLINK-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904775#comment-16904775 ] Jeff Yang commented on FLINK-13002: --- A few more : * watermark > Expand Concept -> Glossary Section > -- > > Key: FLINK-13002 > URL: https://issues.apache.org/jira/browse/FLINK-13002 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Konstantin Knauf >Priority: Major > > We use this ticket to collect terms, we would like to add to the Glossary in > the future: > * Snapshot > * Checkpoint > * Savepoint > * Parallelism > * Backpressure > * TaskSlot -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312763686
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -577,171 +474,101 @@ stream.addSink(myProducer);
val stream: DataStream[String] = ...
val myProducer = new FlinkKafkaProducer011[String](
-"localhost:9092", // broker list
-"my-topic", // target topic
-new SimpleStringSchema) // serialization schema
+"localhost:9092", // broker 列表
+"my-topic", // 目标 topic
+new SimpleStringSchema) // 序列化 schema
-// versions 0.10+ allow attaching the records' event timestamp when writing
them to Kafka;
-// this method is not available for earlier Kafka versions
+// 0.10+ 版本的 Kafka 允许在将记录写入 Kafka 时附加记录的事件时间戳；
+// 此方法不适用于早期版本的 Kafka
myProducer.setWriteTimestampToKafka(true)
stream.addSink(myProducer)
{% endhighlight %}
-The above examples demonstrate the basic usage of creating a Flink Kafka
Producer
-to write streams to a single Kafka target topic. For more advanced usages,
there
-are other constructor variants that allow providing the following:
-
- * *Providing custom properties*:
- The producer allows providing a custom properties configuration for the
internal `KafkaProducer`.
- Please refer to the [Apache Kafka
documentation](https://kafka.apache.org/documentation.html) for
- details on how to configure Kafka Producers.
- * *Custom partitioner*: To assign records to specific
- partitions, you can provide an implementation of a `FlinkKafkaPartitioner` to
the
- constructor. This partitioner will be called for each record in the stream
- to determine which exact partition of the target topic the record should be
sent to.
- Please see [Kafka Producer Partitioning
Scheme](#kafka-producer-partitioning-scheme) for more details.
- * *Advanced serialization schema*: Similar to the consumer,
- the producer also allows using an advanced serialization schema called
`KeyedSerializationSchema`,
- which allows serializing the key and value separately. It also allows to
override the target topic,
- so that one producer instance can send data to multiple topics.
-
-### Kafka Producer Partitioning Scheme
-
-By default, if a custom partitioner is not specified for the Flink Kafka
Producer, the producer will use
-a `FlinkFixedPartitioner` that maps each Flink Kafka Producer parallel subtask
to a single Kafka partition
-(i.e., all records received by a sink subtask will end up in the same Kafka
partition).
-
-A custom partitioner can be implemented by extending the
`FlinkKafkaPartitioner` class. All
-Kafka versions' constructors allow providing a custom partitioner when
instantiating the producer.
-Note that the partitioner implementation must be serializable, as they will be
transferred across Flink nodes.
-Also, keep in mind that any state in the partitioner will be lost on job
failures since the partitioner
-is not part of the producer's checkpointed state.
-
-It is also possible to completely avoid using and kind of partitioner, and
simply let Kafka partition
-the written records by their attached key (as determined for each record using
the provided serialization schema).
-To do this, provide a `null` custom partitioner when instantiating the
producer. It is important
-to provide `null` as the custom partitioner; as explained above, if a custom
partitioner is not specified
-the `FlinkFixedPartitioner` is used instead.
-
-### Kafka Producers and Fault Tolerance
+上面的例子演示了创建 Flink Kafka Producer 来将流消息写入单个 Kafka 目标 topic 的基本用法。
+对于更高级的用法，这还有其他构造函数变体允许提供以下内容：
+
+ * *提供自定义属性*： producer 允许为内部 `KafkaProducer` 提供自定义属性配置。 有关如何配置 Kafka Producers
的详细信息，请参阅 [Apache Kafka 文档](https://kafka.apache.org/documentation.html)。
+ * *自定义分区器*：要将消息分配给特定的分区，可以向构造函数提供一个 `FlinkKafkaPartitioner`
的实现。这个分区器将被流中的每条记录调用，以确定消息应该发送到目标 topic 的哪个具体分区里。有关详细信息，请参阅 [Kafka Producer
分区方案](#kafka-producer-partitioning-scheme)。
+ * *高级的序列化 schema* ：与 consumer 类似， producer 还允许使用名为 `KeyedSerializationSchema`
的高级序列化 schema ，该 schema 允许单独序列化 key 和 value 。它还允许覆盖目标 topic ，以便 producer
实例可以将数据发送到多个 topics 。
+
+### Kafka Producer 分区方案
+
+默认情况下，如果没有为 Flink Kafka Producer 指定自定义分区程序，则 producer 将使用
`FlinkFixedPartitioner` 为每个 Flink Kafka Producer 并行子任务映射到单个 Kafka
分区（即，接收子任务接收到的所有消息都将位于同一个 Kafka 分区中）。
+
+可以通过扩展 `FlinkKafkaPartitioner` 类来实现自定义分区程序。 所有 Kafka 版本的构造函数都允许在实例化 producer
时提供自定义分区程序。
+注意：分区器实现必须是可序列化的，因为它们将在 Flink 节点之间传输。此外，请记住分区器中的任何状态都将在作业失败时丢失，因为分区器不是
producer 的检查点状态的一部分。
+
+也可以完全避免使用分区器，并简单地让 Kafka 通过其附加 key 写入的消息进行分区（使用提供的序列化 schema 为每条记录确定分区）。
+为此，在实例化 producer 时提供 `null` 自定义分区程序，提供 `null` 作为自定义分区器是很重要的;
如上所述，如果未指定自定义分区程序，则默认使用 `FlinkFixedPartitioner` 。
+
+### Kafka Producers 和容错
Kafka 0.8
-Before 0.9 Kafka did not provide any mechanisms to guarantee at-least-once or
exactly-once semantics.
+在 0.9 版本之前，
[jira] [Commented] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904697#comment-16904697 ] Rong Rong commented on FLINK-13548: --- Hi [~boswell] thanks for the contribution. I just come to realize that the priority scheduling feature was only there in [YARN 2.8.x and up|https://hadoop.apache.org/docs/r2.8.5/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setup_for_application_priority.]. Although we can build Flink with Hadoop 2.8.x + using [specific commands|https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html#hadoop-versions]. I was wondering if we should implement a feature that only works in one version. CC [~till.rohrmann] who might have better insight here. > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: boxiu >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is based on YARN official doc. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > priority value is non-negative, the bigger the number, the higher the > priority. By default, we take -1. When the priority is negative, we use > default yarn queue priority. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) * b2cd904545fba2ff99d19ab81db9783294fc762a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717406) * 61f710f8c259610b0bb1b48131063d3c08b4980d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122717629) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-13685) YARN deployment hangs when requested container is above the maximum vcore size
Gyula Fora created FLINK-13685: -- Summary: YARN deployment hangs when requested container is above the maximum vcore size Key: FLINK-13685 URL: https://issues.apache.org/jira/browse/FLINK-13685 Project: Flink Issue Type: New Feature Components: Deployment / YARN Affects Versions: 1.8.1 Reporter: Gyula Fora When deploying on YARN flink explicitly checks whether the requested container memory is smaller than yarn.nodemanager.resource.cpu-vcores. Unfortunately this doesn't guarantee that the request can be fulfilled if yarn.scheduler.maximum-allocation-vcores is set to a lower number. In these cases flink will just keep trying to request a container but will never succeed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] walterddr commented on a change in pull request #8631: [FLINK-12745][ml] add sparse and dense vector class, and dense matrix class with basic operations.
walterddr commented on a change in pull request #8631: [FLINK-12745][ml] add sparse and dense vector class, and dense matrix class with basic operations. URL: https://github.com/apache/flink/pull/8631#discussion_r312744346 ## File path: flink-ml-parent/flink-ml-lib/src/main/resources/META-INF/licenses/LICENSE.core ## @@ -0,0 +1,49 @@ +Copyright (c) 2013 Samuel Halliday Review comment: I am not familiar with the naming convention of the license notice. but seems to me `core` is not a very meaningful name of the package we depend on. all other `LICENSE.*` comes with the package short name like `LICENSE.jpython`, `LICENSE.protobuf` ... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8631: [FLINK-12745][ml] add sparse and dense vector class, and dense matrix class with basic operations.
walterddr commented on a change in pull request #8631: [FLINK-12745][ml] add sparse and dense vector class, and dense matrix class with basic operations. URL: https://github.com/apache/flink/pull/8631#discussion_r312744375 ## File path: flink-ml-parent/flink-ml-lib/src/main/resources/META-INF/licenses/LICENSE.core ## @@ -0,0 +1,49 @@ +Copyright (c) 2013 Samuel Halliday Review comment: also maybe someone who's familiar with this can take a second look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312744349
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -577,171 +474,101 @@ stream.addSink(myProducer);
val stream: DataStream[String] = ...
val myProducer = new FlinkKafkaProducer011[String](
-"localhost:9092", // broker list
-"my-topic", // target topic
-new SimpleStringSchema) // serialization schema
+"localhost:9092", // broker 列表
+"my-topic", // 目标 topic
+new SimpleStringSchema) // 序列化 schema
-// versions 0.10+ allow attaching the records' event timestamp when writing
them to Kafka;
-// this method is not available for earlier Kafka versions
+// 0.10+ 版本的 Kafka 允许在将记录写入 Kafka 时附加记录的事件时间戳；
+// 此方法不适用于早期版本的 Kafka
myProducer.setWriteTimestampToKafka(true)
stream.addSink(myProducer)
{% endhighlight %}
-The above examples demonstrate the basic usage of creating a Flink Kafka
Producer
-to write streams to a single Kafka target topic. For more advanced usages,
there
-are other constructor variants that allow providing the following:
-
- * *Providing custom properties*:
- The producer allows providing a custom properties configuration for the
internal `KafkaProducer`.
- Please refer to the [Apache Kafka
documentation](https://kafka.apache.org/documentation.html) for
- details on how to configure Kafka Producers.
- * *Custom partitioner*: To assign records to specific
- partitions, you can provide an implementation of a `FlinkKafkaPartitioner` to
the
- constructor. This partitioner will be called for each record in the stream
- to determine which exact partition of the target topic the record should be
sent to.
- Please see [Kafka Producer Partitioning
Scheme](#kafka-producer-partitioning-scheme) for more details.
- * *Advanced serialization schema*: Similar to the consumer,
- the producer also allows using an advanced serialization schema called
`KeyedSerializationSchema`,
- which allows serializing the key and value separately. It also allows to
override the target topic,
- so that one producer instance can send data to multiple topics.
-
-### Kafka Producer Partitioning Scheme
-
-By default, if a custom partitioner is not specified for the Flink Kafka
Producer, the producer will use
-a `FlinkFixedPartitioner` that maps each Flink Kafka Producer parallel subtask
to a single Kafka partition
-(i.e., all records received by a sink subtask will end up in the same Kafka
partition).
-
-A custom partitioner can be implemented by extending the
`FlinkKafkaPartitioner` class. All
-Kafka versions' constructors allow providing a custom partitioner when
instantiating the producer.
-Note that the partitioner implementation must be serializable, as they will be
transferred across Flink nodes.
-Also, keep in mind that any state in the partitioner will be lost on job
failures since the partitioner
-is not part of the producer's checkpointed state.
-
-It is also possible to completely avoid using and kind of partitioner, and
simply let Kafka partition
-the written records by their attached key (as determined for each record using
the provided serialization schema).
-To do this, provide a `null` custom partitioner when instantiating the
producer. It is important
-to provide `null` as the custom partitioner; as explained above, if a custom
partitioner is not specified
-the `FlinkFixedPartitioner` is used instead.
-
-### Kafka Producers and Fault Tolerance
+上面的例子演示了创建 Flink Kafka Producer 来将流消息写入单个 Kafka 目标 topic 的基本用法。
+对于更高级的用法，这还有其他构造函数变体允许提供以下内容：
+
+ * *提供自定义属性*： producer 允许为内部 `KafkaProducer` 提供自定义属性配置。 有关如何配置 Kafka Producers
的详细信息，请参阅 [Apache Kafka 文档](https://kafka.apache.org/documentation.html)。
+ * *自定义分区器*：要将消息分配给特定的分区，可以向构造函数提供一个 `FlinkKafkaPartitioner`
的实现。这个分区器将被流中的每条记录调用，以确定消息应该发送到目标 topic 的哪个具体分区里。有关详细信息，请参阅 [Kafka Producer
分区方案](#kafka-producer-partitioning-scheme)。
+ * *高级的序列化 schema* ：与 consumer 类似， producer 还允许使用名为 `KeyedSerializationSchema`
的高级序列化 schema ，该 schema 允许单独序列化 key 和 value 。它还允许覆盖目标 topic ，以便 producer
实例可以将数据发送到多个 topics 。
+
+### Kafka Producer 分区方案
+
+默认情况下，如果没有为 Flink Kafka Producer 指定自定义分区程序，则 producer 将使用
`FlinkFixedPartitioner` 为每个 Flink Kafka Producer 并行子任务映射到单个 Kafka
分区（即，接收子任务接收到的所有消息都将位于同一个 Kafka 分区中）。
+
+可以通过扩展 `FlinkKafkaPartitioner` 类来实现自定义分区程序。 所有 Kafka 版本的构造函数都允许在实例化 producer
时提供自定义分区程序。
+注意：分区器实现必须是可序列化的，因为它们将在 Flink 节点之间传输。此外，请记住分区器中的任何状态都将在作业失败时丢失，因为分区器不是
producer 的检查点状态的一部分。
+
+也可以完全避免使用分区器，并简单地让 Kafka 通过其附加 key 写入的消息进行分区（使用提供的序列化 schema 为每条记录确定分区）。
+为此，在实例化 producer 时提供 `null` 自定义分区程序，提供 `null` 作为自定义分区器是很重要的;
如上所述，如果未指定自定义分区程序，则默认使用 `FlinkFixedPartitioner` 。
+
+### Kafka Producers 和容错
Kafka 0.8
-Before 0.9 Kafka did not provide any mechanisms to guarantee at-least-once or
exactly-once semantics.
+在 0.9 版本之前， Kafka
[jira] [Commented] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904678#comment-16904678 ] Jark Wu commented on FLINK-13684: - The community decide to execute the new JIRA workflow last month. You do not need any contributor permissions. If you are interested in an issue, you can left you message there, and committers will assign issue to you. I assigned this ticket to you. Thanks for the contribution. [~hyl] > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website > Environment: Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. >Reporter: Jark Wu >Assignee: jeremy huang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-13684: Description: Translate page https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html into Chinese. The page is located in https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: jeremy huang >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-13684: --- Assignee: jeremy huang > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website > Environment: Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. >Reporter: Jark Wu >Assignee: jeremy huang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-13684: Environment: (was: Translate page https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html into Chinese. The page is located in https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. ) > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: jeremy huang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) * b2cd904545fba2ff99d19ab81db9783294fc762a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717406) * 61f710f8c259610b0bb1b48131063d3c08b4980d : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122717629) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) * b2cd904545fba2ff99d19ab81db9783294fc762a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122717406) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
flinkbot edited a comment on issue #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#issuecomment-52013 ## CI report: * 78530c52201e8bfb07001939a9bf9121f3d9ca79 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/122676569) * fe9be668cac64fe4988015e6bfe3e91cdfb04ddf : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122678008) * f48827994bdd5b220d581144023ae811df9538e8 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122717228) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312743386 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -532,41 +442,28 @@ stream = env -Internally, an instance of the assigner is executed per Kafka partition. -When such an assigner is specified, for each record read from Kafka, the -`extractTimestamp(T element, long previousElementTimestamp)` is called to assign a timestamp to the record and -the `Watermark getCurrentWatermark()` (for periodic) or the -`Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` (for punctuated) is called to determine -if a new watermark should be emitted and with which timestamp. - -**Note**: If a watermark assigner depends on records read from Kafka to advance its watermarks -(which is commonly the case), all topics and partitions need to have a continuous stream of records. -Otherwise, the watermarks of the whole application cannot advance and all time-based operations, -such as time windows or functions with timers, cannot make progress. A single idle Kafka partition causes this behavior. -A Flink improvement is planned to prevent this from happening -(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions]( -https://issues.apache.org/jira/browse/FLINK-5479)). -In the meanwhile, a possible workaround is to send *heartbeat messages* to all consumed partitions that advance the watermarks of idle partitions. +在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （用于标点符号）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 + +**请注意**： 如果水位线 assigner 依赖于从 Kafka 读取的消息来上涨其水位(通常就是这种情况)，那么所有主题和分区都需要有连续的消息流。否则，整个应用程序的水位将无法上涨，所有基于时间的算子(例如时间窗口或带有计时器的函数)也无法运行。单个的 Kafka 分区也会导致这种反应。这是一个已在计划中的 Flink improvement ，目的是为了防止这种情况发生（请见[FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions](https://issues.apache.org/jira/browse/FLINK-5479)）。同时，可能的解决方法是将*心跳消息*发送到所有 consumer 的分区里，从而上涨空闲分区的水位。 Review comment: 我也觉得 “watermark” 应该是一个专用名词，不应该翻译其字面的意思。 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312743235
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -107,40 +100,36 @@ Then, import the connector in your maven project:
{% endhighlight %}
-Note that the streaming connectors are currently not part of the binary
distribution.
-See how to link with them for cluster execution [here]({{
site.baseurl}}/dev/projectsetup/dependencies.html).
+请注意：目前流连接器还不是二进制分发的一部分。
+[在此处]({{
site.baseurl}}/dev/projectsetup/dependencies.html)可以了解到如何链接它们以实现在集群中执行。
Review comment:
Okay thank you! I will pay attention to this point in my future
contributions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
highfei2011 commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312742931
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -577,171 +474,101 @@ stream.addSink(myProducer);
val stream: DataStream[String] = ...
val myProducer = new FlinkKafkaProducer011[String](
-"localhost:9092", // broker list
-"my-topic", // target topic
-new SimpleStringSchema) // serialization schema
+"localhost:9092", // broker 列表
+"my-topic", // 目标 topic
+new SimpleStringSchema) // 序列化 schema
-// versions 0.10+ allow attaching the records' event timestamp when writing
them to Kafka;
-// this method is not available for earlier Kafka versions
+// 0.10+ 版本的 Kafka 允许在将记录写入 Kafka 时附加记录的事件时间戳；
+// 此方法不适用于早期版本的 Kafka
myProducer.setWriteTimestampToKafka(true)
stream.addSink(myProducer)
{% endhighlight %}
-The above examples demonstrate the basic usage of creating a Flink Kafka
Producer
-to write streams to a single Kafka target topic. For more advanced usages,
there
-are other constructor variants that allow providing the following:
-
- * *Providing custom properties*:
- The producer allows providing a custom properties configuration for the
internal `KafkaProducer`.
- Please refer to the [Apache Kafka
documentation](https://kafka.apache.org/documentation.html) for
- details on how to configure Kafka Producers.
- * *Custom partitioner*: To assign records to specific
- partitions, you can provide an implementation of a `FlinkKafkaPartitioner` to
the
- constructor. This partitioner will be called for each record in the stream
- to determine which exact partition of the target topic the record should be
sent to.
- Please see [Kafka Producer Partitioning
Scheme](#kafka-producer-partitioning-scheme) for more details.
- * *Advanced serialization schema*: Similar to the consumer,
- the producer also allows using an advanced serialization schema called
`KeyedSerializationSchema`,
- which allows serializing the key and value separately. It also allows to
override the target topic,
- so that one producer instance can send data to multiple topics.
-
-### Kafka Producer Partitioning Scheme
-
-By default, if a custom partitioner is not specified for the Flink Kafka
Producer, the producer will use
-a `FlinkFixedPartitioner` that maps each Flink Kafka Producer parallel subtask
to a single Kafka partition
-(i.e., all records received by a sink subtask will end up in the same Kafka
partition).
-
-A custom partitioner can be implemented by extending the
`FlinkKafkaPartitioner` class. All
-Kafka versions' constructors allow providing a custom partitioner when
instantiating the producer.
-Note that the partitioner implementation must be serializable, as they will be
transferred across Flink nodes.
-Also, keep in mind that any state in the partitioner will be lost on job
failures since the partitioner
-is not part of the producer's checkpointed state.
-
-It is also possible to completely avoid using and kind of partitioner, and
simply let Kafka partition
-the written records by their attached key (as determined for each record using
the provided serialization schema).
-To do this, provide a `null` custom partitioner when instantiating the
producer. It is important
-to provide `null` as the custom partitioner; as explained above, if a custom
partitioner is not specified
-the `FlinkFixedPartitioner` is used instead.
-
-### Kafka Producers and Fault Tolerance
+上面的例子演示了创建 Flink Kafka Producer 来将流消息写入单个 Kafka 目标 topic 的基本用法。
+对于更高级的用法，这还有其他构造函数变体允许提供以下内容：
+
+ * *提供自定义属性*： producer 允许为内部 `KafkaProducer` 提供自定义属性配置。 有关如何配置 Kafka Producers
的详细信息，请参阅 [Apache Kafka 文档](https://kafka.apache.org/documentation.html)。
+ * *自定义分区器*：要将消息分配给特定的分区，可以向构造函数提供一个 `FlinkKafkaPartitioner`
的实现。这个分区器将被流中的每条记录调用，以确定消息应该发送到目标 topic 的哪个具体分区里。有关详细信息，请参阅 [Kafka Producer
分区方案](#kafka-producer-partitioning-scheme)。
+ * *高级的序列化 schema* ：与 consumer 类似， producer 还允许使用名为 `KeyedSerializationSchema`
的高级序列化 schema ，该 schema 允许单独序列化 key 和 value 。它还允许覆盖目标 topic ，以便 producer
实例可以将数据发送到多个 topics 。
+
+### Kafka Producer 分区方案
+
+默认情况下，如果没有为 Flink Kafka Producer 指定自定义分区程序，则 producer 将使用
`FlinkFixedPartitioner` 为每个 Flink Kafka Producer 并行子任务映射到单个 Kafka
分区（即，接收子任务接收到的所有消息都将位于同一个 Kafka 分区中）。
+
+可以通过扩展 `FlinkKafkaPartitioner` 类来实现自定义分区程序。 所有 Kafka 版本的构造函数都允许在实例化 producer
时提供自定义分区程序。
+注意：分区器实现必须是可序列化的，因为它们将在 Flink 节点之间传输。此外，请记住分区器中的任何状态都将在作业失败时丢失，因为分区器不是
producer 的检查点状态的一部分。
+
+也可以完全避免使用分区器，并简单地让 Kafka 通过其附加 key 写入的消息进行分区（使用提供的序列化 schema 为每条记录确定分区）。
+为此，在实例化 producer 时提供 `null` 自定义分区程序，提供 `null` 作为自定义分区器是很重要的;
如上所述，如果未指定自定义分区程序，则默认使用 `FlinkFixedPartitioner` 。
+
+### Kafka Producers 和容错
Kafka 0.8
-Before 0.9 Kafka did not provide any mechanisms to guarantee at-least-once or
exactly-once semantics.
+在 0.9 版本之前，
[jira] [Commented] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904674#comment-16904674 ] jeremy huang commented on FLINK-13684: -- I wonder why I can't directly assign this issue to me ? I can't find the 'assign to me' button but I got the contribute permission six month ago.@ [~jark] > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website > Environment: Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. >Reporter: Jark Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904667#comment-16904667 ] jeremy huang edited comment on FLINK-13684 at 8/11/19 2:25 PM: --- Hi, Jark. I'm interesting in this translation and could you assign it to me? thx. was (Author: hyl): Hi, Jark. I'm interesting in this translation and I will assign it to me, thx. > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website > Environment: Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. >Reporter: Jark Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13684) Translate "Code Style - Formatting Guide" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904667#comment-16904667 ] jeremy huang commented on FLINK-13684: -- Hi, Jark. I'm interesting in this translation and I will assign it to me, thx. > Translate "Code Style - Formatting Guide" page into Chinese > --- > > Key: FLINK-13684 > URL: https://issues.apache.org/jira/browse/FLINK-13684 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website > Environment: Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-formatting.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-formatting.zh.md. >Reporter: Jark Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13678) Translate "Code Style - Preamble" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904658#comment-16904658 ] Jark Wu commented on FLINK-13678: - Sure. I assigned it to you [~WangHW]. > Translate "Code Style - Preamble" page into Chinese > --- > > Key: FLINK-13678 > URL: https://issues.apache.org/jira/browse/FLINK-13678 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: WangHengWei >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-preamble.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-scala.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (FLINK-13678) Translate "Code Style - Preamble" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-13678: --- Assignee: WangHengWei > Translate "Code Style - Preamble" page into Chinese > --- > > Key: FLINK-13678 > URL: https://issues.apache.org/jira/browse/FLINK-13678 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: WangHengWei >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-preamble.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-scala.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738050
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
+2. 用于反序列化 Kafka 数据的 DeserializationSchema 或者 KafkaDeserializationSchema
+3. Kafka consume 的属性。需要以下属性：
+ - "bootstrap.servers" （以逗号分隔的 Kafka broker 列表）
+ - "zookeeper.connect" （以逗号分割的 Zookeeper servers 列表) (**仅限 Kafka 0.8 使用**)
Review comment:
```suggestion
- "zookeeper.connect" （以逗号分割的 Zookeeper servers 列表) (**仅 Kafka 0.8 需要**)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738024
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
+2. 用于反序列化 Kafka 数据的 DeserializationSchema 或者 KafkaDeserializationSchema
+3. Kafka consume 的属性。需要以下属性：
Review comment:
```suggestion
3. Kafka 消费者的属性。需要以下属性：
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312739419 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -532,41 +442,28 @@ stream = env -Internally, an instance of the assigner is executed per Kafka partition. -When such an assigner is specified, for each record read from Kafka, the -`extractTimestamp(T element, long previousElementTimestamp)` is called to assign a timestamp to the record and -the `Watermark getCurrentWatermark()` (for periodic) or the -`Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` (for punctuated) is called to determine -if a new watermark should be emitted and with which timestamp. - -**Note**: If a watermark assigner depends on records read from Kafka to advance its watermarks -(which is commonly the case), all topics and partitions need to have a continuous stream of records. -Otherwise, the watermarks of the whole application cannot advance and all time-based operations, -such as time windows or functions with timers, cannot make progress. A single idle Kafka partition causes this behavior. -A Flink improvement is planned to prevent this from happening -(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions]( -https://issues.apache.org/jira/browse/FLINK-5479)). -In the meanwhile, a possible workaround is to send *heartbeat messages* to all consumed partitions that advance the watermarks of idle partitions. +在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （用于标点符号）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 + +**请注意**： 如果水位线 assigner 依赖于从 Kafka 读取的消息来上涨其水位(通常就是这种情况)，那么所有主题和分区都需要有连续的消息流。否则，整个应用程序的水位将无法上涨，所有基于时间的算子(例如时间窗口或带有计时器的函数)也无法运行。单个的 Kafka 分区也会导致这种反应。这是一个已在计划中的 Flink improvement ，目的是为了防止这种情况发生（请见[FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions](https://issues.apache.org/jira/browse/FLINK-5479)）。同时，可能的解决方法是将*心跳消息*发送到所有 consumer 的分区里，从而上涨空闲分区的水位。 ## Kafka Producer -Flink’s Kafka Producer is called `FlinkKafkaProducer011` (or `010` for Kafka 0.10.0.x versions, etc. or just `FlinkKafkaProducer` for Kafka >= 1.0.0 versions). -It allows writing a stream of records to one or more Kafka topics. +Flink Kafka Producer 被称为 `FlinkKafkaProducer011` （或者 `010` 只适合 Kafka 0.10.0.x 的版本使用 ，或 `FlinkKafkaProducer` 只适用于 Kafka> = 1.0.0 的版本使用）。它允许将消息流写入一个或多个 Kafka topics 。 Review comment: ```suggestion Flink Kafka Producer 被称为 `FlinkKafkaProducer011` （或适用于 Kafka 0.10.0.x 版本的 `FlinkKafkaProducer010`，或适用于 Kafka >= 1.0.0 版本的 `FlinkKafkaProducer`）。它允许将消息流写入一个或多个 Kafka topics 。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737498
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -54,31 +49,32 @@ For most users, the `FlinkKafkaConsumer08` (part of
`flink-connector-kafka`) is
FlinkKafkaConsumer08
FlinkKafkaProducer08
0.8.x
-Uses the https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example;>SimpleConsumer
API of Kafka internally. Offsets are committed to ZK by Flink.
+这个连接器在内部使用 Kafka 的 https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example;>SimpleConsumer
API 。偏移量由 Flink 提交给 ZK 。
+
flink-connector-kafka-0.9{{ site.scala_version_suffix }}
1.0.0
FlinkKafkaConsumer09
FlinkKafkaProducer09
0.9.x
-Uses the new http://kafka.apache.org/documentation.html#newconsumerapi;>Consumer
API Kafka.
+这个连接器使用新的 Kafka http://kafka.apache.org/documentation.html#newconsumerapi;>Consumer
API
flink-connector-kafka-0.10{{ site.scala_version_suffix }}
1.2.0
FlinkKafkaConsumer010
FlinkKafkaProducer010
0.10.x
-This connector supports https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message;>Kafka
messages with timestamps both for producing and consuming.
+这个连接器支持 https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message;>带有时间戳的
Kafka 消息，用于生产和消费。
flink-connector-kafka-0.11{{ site.scala_version_suffix }}
1.4.0
FlinkKafkaConsumer011
FlinkKafkaProducer011
0.11.x
-Since 0.11.x Kafka does not support scala 2.10. This connector
supports https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging;>Kafka
transactional messaging to provide exactly once semantic for the
producer.
+由于 Kafka 的 0.11.x 版本不支持 scala 2.10 ，所以此连接器通过支持 https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging;>Kafka
事务性的消息传递来为生产者提供 Exactly once 语义。
Review comment:
```suggestion
Kafka 从 0.11.x 版本开始不支持 Scala 2.10。此连接器支持了 https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging;>Kafka
事务性的消息传递来为生产者提供 Exactly once 语义。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738065
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
+2. 用于反序列化 Kafka 数据的 DeserializationSchema 或者 KafkaDeserializationSchema
+3. Kafka consume 的属性。需要以下属性：
+ - "bootstrap.servers" （以逗号分隔的 Kafka broker 列表）
+ - "zookeeper.connect" （以逗号分割的 Zookeeper servers 列表) (**仅限 Kafka 0.8 使用**)
+ - "group.id" 消费组 ID
-Example:
+例：
Review comment:
```suggestion
示例：
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312738631 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -254,33 +229,24 @@ For convenience, Flink provides the following schemas: -When encountering a corrupted message that cannot be deserialized for any reason, there -are two options - either throwing an exception from the `deserialize(...)` method -which will cause the job to fail and be restarted, or returning `null` to allow -the Flink Kafka consumer to silently skip the corrupted message. Note that -due to the consumer's fault tolerance (see below sections for more details), -failing the job on the corrupted message will let the consumer attempt -to deserialize the message again. Therefore, if deserialization still fails, the -consumer will fall into a non-stop restart and fail loop on that corrupted -message. +当遇到因一些原因而无法反序列化的损坏消息时，这里有两个选项 - 从 `deserialize（...）` 方法抛出异常会导致作业失败并重新启动，或返回 `null` ，以允许 Flink Kafka 消费者悄悄地跳过损坏的消息。请注意，由于 Consumer 的容错能力（请参阅下面的部分以获取更多详细信息），在损坏的消息上失败作业将使 consumer 尝试再次反序列化消息。因此，如果反序列化仍然失败，则 consumer 将在该损坏的消息上进入不间断重启和失败的循环。 Review comment: How about to translate "corrupted message" as"脏数据"？ IMO, "损坏的消息" is not native. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738087
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
+2. 用于反序列化 Kafka 数据的 DeserializationSchema 或者 KafkaDeserializationSchema
+3. Kafka consume 的属性。需要以下属性：
+ - "bootstrap.servers" （以逗号分隔的 Kafka broker 列表）
+ - "zookeeper.connect" （以逗号分割的 Zookeeper servers 列表) (**仅限 Kafka 0.8 使用**)
+ - "group.id" 消费组 ID
-Example:
+例：
{% highlight java %}
Properties properties = new Properties();
properties.setProperty("bootstrap.servers", "localhost:9092");
-// only required for Kafka 0.8
+// 仅限 Kafka 0.8 使用
properties.setProperty("zookeeper.connect", "localhost:2181");
properties.setProperty("group.id", "test");
DataStream stream = env
- .addSource(new FlinkKafkaConsumer08<>("topic", new
SimpleStringSchema(), properties));
+ .addSource(new FlinkKafkaConsumer08<>("topic", new SimpleStringSchema(),
properties));
{% endhighlight %}
{% highlight scala %}
val properties = new Properties()
properties.setProperty("bootstrap.servers", "localhost:9092")
-// only required for Kafka 0.8
+// 仅限 Kafka 0.8 使用
Review comment:
```suggestion
// 仅 Kafka 0.8 需要
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312738435 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -197,41 +182,31 @@ stream = env -### The `DeserializationSchema` +### `DeserializationSchema` -The Flink Kafka Consumer needs to know how to turn the binary data in Kafka into Java/Scala objects. The -`DeserializationSchema` allows users to specify such a schema. The `T deserialize(byte[] message)` -method gets called for each Kafka message, passing the value from Kafka. +Flink Kafka Consumer 需要知道如何将 Kafka 中的二进制数据转换为 Java 或者 Scala 对象。 `DeserializationSchema` 允许用户指定这样的 schema ，为每条 Kafka 消息调用 `T deserialize(byte[] message)` 方法，从 Kafka 中传递值。 -It is usually helpful to start from the `AbstractDeserializationSchema`, which takes care of describing the -produced Java/Scala type to Flink's type system. Users that implement a vanilla `DeserializationSchema` need -to implement the `getProducedType(...)` method themselves. +从 `AbstractDeserializationSchema` 开始通常很有帮助，它负责将生成的 Java 或 Scala 类型描述为 Flink 的类型系统。 +实现带有 vanilla `DeserializationSchema` 的用户需要自己实现 `getProducedType（...）` 方法。 -For accessing the key, value and metadata of the Kafka message, the `KafkaDeserializationSchema` has -the following deserialize method `T deserialize(ConsumerRecord record)`. +为了访问 Kafka 消息的 key 、 value 和元数据， `KafkaDeserializationSchema` 具有以下反序列化方法 `T deserialize(ConsumerRecord record)` 。 -For convenience, Flink provides the following schemas: +为了方便使用， Flink 提供了以下几种 schemas ： -1. `TypeInformationSerializationSchema` (and `TypeInformationKeyValueSerializationSchema`) which creates -a schema based on a Flink's `TypeInformation`. This is useful if the data is both written and read by Flink. -This schema is a performant Flink-specific alternative to other generic serialization approaches. +1. `TypeInformationSerializationSchema` （和 `TypeInformationKeyValueSerializationSchema`) 基于 Flink 的 `TypeInformation` 创建 `schema` 。 +如果 Flink 既负责数据的读也负责写，那么这将是非常有用的。此 schema 是其他通用序列化方法的高性能 Flink 替代方案。 Review comment: ```suggestion 如果 Flink 既负责数据的读也负责写，那么这将是非常有用的。此 schema 是其他通用序列化方法的高性能 Flink 替代方案。 ``` ```suggestion 如果该数据的读和写都发生在 Flink 中，那么这将是非常有用的。此 schema 是其他通用序列化方法的高性能 Flink 替代方案。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312739159
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -448,61 +383,36 @@ val stream = env.addSource(myConsumer)
-In the above example, all topics with names that match the specified regular
expression
-(starting with `test-topic-` and ending with a single digit) will be
subscribed by the consumer
-when the job starts running.
-
-To allow the consumer to discover dynamically created topics after the job
started running,
-set a non-negative value for `flink.partition-discovery.interval-millis`. This
allows
-the consumer to discover partitions of new topics with names that also match
the specified
-pattern.
-
-### Kafka Consumers Offset Committing Behaviour Configuration
-
-The Flink Kafka Consumer allows configuring the behaviour of how offsets
-are committed back to Kafka brokers (or Zookeeper in 0.8). Note that the
-Flink Kafka Consumer does not rely on the committed offsets for fault
-tolerance guarantees. The committed offsets are only a means to expose
-the consumer's progress for monitoring purposes.
-
-The way to configure offset commit behaviour is different, depending on
-whether or not checkpointing is enabled for the job.
-
- - *Checkpointing disabled:* if checkpointing is disabled, the Flink Kafka
- Consumer relies on the automatic periodic offset committing capability
- of the internally used Kafka clients. Therefore, to disable or enable offset
- committing, simply set the `enable.auto.commit` (or `auto.commit.enable`
- for Kafka 0.8) / `auto.commit.interval.ms` keys to appropriate values
- in the provided `Properties` configuration.
-
- - *Checkpointing enabled:* if checkpointing is enabled, the Flink Kafka
- Consumer will commit the offsets stored in the checkpointed states when
- the checkpoints are completed. This ensures that the committed offsets
- in Kafka brokers is consistent with the offsets in the checkpointed states.
- Users can choose to disable or enable offset committing by calling the
- `setCommitOffsetsOnCheckpoints(boolean)` method on the consumer (by default,
- the behaviour is `true`).
- Note that in this scenario, the automatic periodic offset committing
- settings in `Properties` is completely ignored.
-
-### Kafka Consumers and Timestamp Extraction/Watermark Emission
-
-In many scenarios, the timestamp of a record is embedded (explicitly or
implicitly) in the record itself.
-In addition, the user may want to emit watermarks either periodically, or in
an irregular fashion, e.g. based on
-special records in the Kafka stream that contain the current event-time
watermark. For these cases, the Flink Kafka
-Consumer allows the specification of an `AssignerWithPeriodicWatermarks` or an
`AssignerWithPunctuatedWatermarks`.
-
-You can specify your custom timestamp extractor/watermark emitter as described
-[here]({{ site.baseurl }}/dev/event_timestamps_watermarks.html), or use one
from the
-[predefined ones]({{ site.baseurl }}/dev/event_timestamp_extractors.html).
After doing so, you
-can pass it to your consumer in the following way:
+在上面的例子中，当 Job 开始运行时， Consumer 将订阅名称与指定正则表达式匹配的所有主题（以 `test-topic` 开头并以单个数字结尾）。
+
+要允许 consumer 在作业开始运行后发现动态创建的主题，那么请为
`flink.partition-discovery.interval-millis` 设置非负值。 这允许 consumer
发现名称与指定模式匹配的新主题的分区。
+
+### Kafka Consumers 提交 Offset 的行为配置
+
+Flink Kafka Consumer 允许有配置如何将 offsets 提交回 Kafka broker （或 0.8 版本的 Zookeeper
）的行为。请注意： Flink Kafka Consumer 不依赖于提交的 offsets 来实现容错保证。提交的 offsets 只是一种方法，用于公开
consumer 的进度以便进行监控。
+
+配置 offset 提交行为的方法是否相同，取决于是否为 job 启用了 checkpointing 。
+
+ - *禁用 Checkpointing ：* 如果禁用了 checkpointing ，则 Flink Kafka Consumer 依赖于内部使用的
Kafka client 自动定期 offset 提交功能。
+ 因此，要禁用或启用 offset 的提交，只需将 `enable.auto.commit`（或 Kafka 0.8 的
`auto.commit.enable` ）或者 `auto.commit.interval.ms` 的Key 值设置为提供的 `Properties`
配置中的适当值。
+
+ - *启用 Checkpointing ：* 如果启用了 checkpointing ，那么当 checkpointing 完成时， Flink
Kafka Consumer 将提交的 offset 存储在 checkpoints 状态中。
+ 这确保 Kafka broker 中提交的 offset 与 checkpoints 状态中的 offset 一致。
+ 用户可以通过调用 consumer 上的 `setCommitOffsetsOnCheckpoints(boolean)` 方法来禁用或启用 offset
的提交(默认情况下，这个值是 true )。
+ 注意，在这个场景中，完全忽略 `Properties` 中的自动定期 offset 提交设置。
+
+### Kafka Consumers 和 Timestamp extractor 或者 Watermark Emission
+
+在许多场景中，记录的时间戳(显式或隐式)嵌入到记录本身中。此外，用户可能希望定期或以不规则的方式 Watermark Emission ，例如基于
Kafka 流中包含当前事件时间水位线的特殊记录。对于这些情况， Flink Kafka Consumer 允许指定
`AssignerWithPeriodicWatermarks` 或 `AssignerWithPunctuatedWatermarks`。
+
+你可以按照[此处]({{ site.baseurl }}/dev/event_timestamps_watermarks.html)的说明指定自定义
timestamp extractor 或者 Watermark Emission ，或者使用 [predefined ones]({{
site.baseurl }}/dev/event_timestamp_extractors.html)。你也可以通过以下方式将其传递给你的 consumer
：
{% highlight java %}
Properties properties = new Properties();
properties.setProperty("bootstrap.servers", "localhost:9092");
-// only required for
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738078
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
+2. 用于反序列化 Kafka 数据的 DeserializationSchema 或者 KafkaDeserializationSchema
+3. Kafka consume 的属性。需要以下属性：
+ - "bootstrap.servers" （以逗号分隔的 Kafka broker 列表）
+ - "zookeeper.connect" （以逗号分割的 Zookeeper servers 列表) (**仅限 Kafka 0.8 使用**)
+ - "group.id" 消费组 ID
-Example:
+例：
{% highlight java %}
Properties properties = new Properties();
properties.setProperty("bootstrap.servers", "localhost:9092");
-// only required for Kafka 0.8
+// 仅限 Kafka 0.8 使用
Review comment:
```suggestion
// 仅 Kafka 0.8 需要
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312738396 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -197,41 +182,31 @@ stream = env -### The `DeserializationSchema` +### `DeserializationSchema` -The Flink Kafka Consumer needs to know how to turn the binary data in Kafka into Java/Scala objects. The -`DeserializationSchema` allows users to specify such a schema. The `T deserialize(byte[] message)` -method gets called for each Kafka message, passing the value from Kafka. +Flink Kafka Consumer 需要知道如何将 Kafka 中的二进制数据转换为 Java 或者 Scala 对象。 `DeserializationSchema` 允许用户指定这样的 schema ，为每条 Kafka 消息调用 `T deserialize(byte[] message)` 方法，从 Kafka 中传递值。 -It is usually helpful to start from the `AbstractDeserializationSchema`, which takes care of describing the -produced Java/Scala type to Flink's type system. Users that implement a vanilla `DeserializationSchema` need -to implement the `getProducedType(...)` method themselves. +从 `AbstractDeserializationSchema` 开始通常很有帮助，它负责将生成的 Java 或 Scala 类型描述为 Flink 的类型系统。 +实现带有 vanilla `DeserializationSchema` 的用户需要自己实现 `getProducedType（...）` 方法。 -For accessing the key, value and metadata of the Kafka message, the `KafkaDeserializationSchema` has -the following deserialize method `T deserialize(ConsumerRecord record)`. +为了访问 Kafka 消息的 key 、 value 和元数据， `KafkaDeserializationSchema` 具有以下反序列化方法 `T deserialize(ConsumerRecord record)` 。 -For convenience, Flink provides the following schemas: +为了方便使用， Flink 提供了以下几种 schemas ： -1. `TypeInformationSerializationSchema` (and `TypeInformationKeyValueSerializationSchema`) which creates -a schema based on a Flink's `TypeInformation`. This is useful if the data is both written and read by Flink. -This schema is a performant Flink-specific alternative to other generic serialization approaches. +1. `TypeInformationSerializationSchema` （和 `TypeInformationKeyValueSerializationSchema`) 基于 Flink 的 `TypeInformation` 创建 `schema` 。 Review comment: ```suggestion 1. `TypeInformationSerializationSchema` （和 `TypeInformationKeyValueSerializationSchema`) 是一个基于 Flink 的`TypeInformation` 创建的 schema。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738999
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -448,61 +383,36 @@ val stream = env.addSource(myConsumer)
-In the above example, all topics with names that match the specified regular
expression
-(starting with `test-topic-` and ending with a single digit) will be
subscribed by the consumer
-when the job starts running.
-
-To allow the consumer to discover dynamically created topics after the job
started running,
-set a non-negative value for `flink.partition-discovery.interval-millis`. This
allows
-the consumer to discover partitions of new topics with names that also match
the specified
-pattern.
-
-### Kafka Consumers Offset Committing Behaviour Configuration
-
-The Flink Kafka Consumer allows configuring the behaviour of how offsets
-are committed back to Kafka brokers (or Zookeeper in 0.8). Note that the
-Flink Kafka Consumer does not rely on the committed offsets for fault
-tolerance guarantees. The committed offsets are only a means to expose
-the consumer's progress for monitoring purposes.
-
-The way to configure offset commit behaviour is different, depending on
-whether or not checkpointing is enabled for the job.
-
- - *Checkpointing disabled:* if checkpointing is disabled, the Flink Kafka
- Consumer relies on the automatic periodic offset committing capability
- of the internally used Kafka clients. Therefore, to disable or enable offset
- committing, simply set the `enable.auto.commit` (or `auto.commit.enable`
- for Kafka 0.8) / `auto.commit.interval.ms` keys to appropriate values
- in the provided `Properties` configuration.
-
- - *Checkpointing enabled:* if checkpointing is enabled, the Flink Kafka
- Consumer will commit the offsets stored in the checkpointed states when
- the checkpoints are completed. This ensures that the committed offsets
- in Kafka brokers is consistent with the offsets in the checkpointed states.
- Users can choose to disable or enable offset committing by calling the
- `setCommitOffsetsOnCheckpoints(boolean)` method on the consumer (by default,
- the behaviour is `true`).
- Note that in this scenario, the automatic periodic offset committing
- settings in `Properties` is completely ignored.
-
-### Kafka Consumers and Timestamp Extraction/Watermark Emission
-
-In many scenarios, the timestamp of a record is embedded (explicitly or
implicitly) in the record itself.
-In addition, the user may want to emit watermarks either periodically, or in
an irregular fashion, e.g. based on
-special records in the Kafka stream that contain the current event-time
watermark. For these cases, the Flink Kafka
-Consumer allows the specification of an `AssignerWithPeriodicWatermarks` or an
`AssignerWithPunctuatedWatermarks`.
-
-You can specify your custom timestamp extractor/watermark emitter as described
-[here]({{ site.baseurl }}/dev/event_timestamps_watermarks.html), or use one
from the
-[predefined ones]({{ site.baseurl }}/dev/event_timestamp_extractors.html).
After doing so, you
-can pass it to your consumer in the following way:
+在上面的例子中，当 Job 开始运行时， Consumer 将订阅名称与指定正则表达式匹配的所有主题（以 `test-topic` 开头并以单个数字结尾）。
+
+要允许 consumer 在作业开始运行后发现动态创建的主题，那么请为
`flink.partition-discovery.interval-millis` 设置非负值。 这允许 consumer
发现名称与指定模式匹配的新主题的分区。
+
+### Kafka Consumers 提交 Offset 的行为配置
+
+Flink Kafka Consumer 允许有配置如何将 offsets 提交回 Kafka broker （或 0.8 版本的 Zookeeper
）的行为。请注意： Flink Kafka Consumer 不依赖于提交的 offsets 来实现容错保证。提交的 offsets 只是一种方法，用于公开
consumer 的进度以便进行监控。
+
+配置 offset 提交行为的方法是否相同，取决于是否为 job 启用了 checkpointing 。
+
+ - *禁用 Checkpointing ：* 如果禁用了 checkpointing ，则 Flink Kafka Consumer 依赖于内部使用的
Kafka client 自动定期 offset 提交功能。
+ 因此，要禁用或启用 offset 的提交，只需将 `enable.auto.commit`（或 Kafka 0.8 的
`auto.commit.enable` ）或者 `auto.commit.interval.ms` 的Key 值设置为提供的 `Properties`
配置中的适当值。
+
+ - *启用 Checkpointing ：* 如果启用了 checkpointing ，那么当 checkpointing 完成时， Flink
Kafka Consumer 将提交的 offset 存储在 checkpoints 状态中。
+ 这确保 Kafka broker 中提交的 offset 与 checkpoints 状态中的 offset 一致。
+ 用户可以通过调用 consumer 上的 `setCommitOffsetsOnCheckpoints(boolean)` 方法来禁用或启用 offset
的提交(默认情况下，这个值是 true )。
+ 注意，在这个场景中，完全忽略 `Properties` 中的自动定期 offset 提交设置。
+
+### Kafka Consumers 和 Timestamp extractor 或者 Watermark Emission
Review comment:
```suggestion
### Kafka Consumers 和时间戳抽取以及 Watermark 发送
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312739165
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -511,15 +421,15 @@ FlinkKafkaConsumer08 myConsumer =
myConsumer.assignTimestampsAndWatermarks(new CustomWatermarkEmitter());
DataStream stream = env
- .addSource(myConsumer)
- .print();
+ .addSource(myConsumer)
+ .print();
{% endhighlight %}
{% highlight scala %}
val properties = new Properties()
properties.setProperty("bootstrap.servers", "localhost:9092")
-// only required for Kafka 0.8
+// 仅限于 Kafka 0.8 使用
Review comment:
```suggestion
// 仅 Kafka 0.8 需要
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737965
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
Review comment:
```suggestion
Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或适用于 Kafka 0.9.0.x 版本的
`FlinkKafkaConsumer09`，或适用于 Kafka >= 1.0.0 版本的`FlinkKafkaConsumer`）。它提供对一个或多个
Kafka topics 的访问。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312739155
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -448,61 +383,36 @@ val stream = env.addSource(myConsumer)
-In the above example, all topics with names that match the specified regular
expression
-(starting with `test-topic-` and ending with a single digit) will be
subscribed by the consumer
-when the job starts running.
-
-To allow the consumer to discover dynamically created topics after the job
started running,
-set a non-negative value for `flink.partition-discovery.interval-millis`. This
allows
-the consumer to discover partitions of new topics with names that also match
the specified
-pattern.
-
-### Kafka Consumers Offset Committing Behaviour Configuration
-
-The Flink Kafka Consumer allows configuring the behaviour of how offsets
-are committed back to Kafka brokers (or Zookeeper in 0.8). Note that the
-Flink Kafka Consumer does not rely on the committed offsets for fault
-tolerance guarantees. The committed offsets are only a means to expose
-the consumer's progress for monitoring purposes.
-
-The way to configure offset commit behaviour is different, depending on
-whether or not checkpointing is enabled for the job.
-
- - *Checkpointing disabled:* if checkpointing is disabled, the Flink Kafka
- Consumer relies on the automatic periodic offset committing capability
- of the internally used Kafka clients. Therefore, to disable or enable offset
- committing, simply set the `enable.auto.commit` (or `auto.commit.enable`
- for Kafka 0.8) / `auto.commit.interval.ms` keys to appropriate values
- in the provided `Properties` configuration.
-
- - *Checkpointing enabled:* if checkpointing is enabled, the Flink Kafka
- Consumer will commit the offsets stored in the checkpointed states when
- the checkpoints are completed. This ensures that the committed offsets
- in Kafka brokers is consistent with the offsets in the checkpointed states.
- Users can choose to disable or enable offset committing by calling the
- `setCommitOffsetsOnCheckpoints(boolean)` method on the consumer (by default,
- the behaviour is `true`).
- Note that in this scenario, the automatic periodic offset committing
- settings in `Properties` is completely ignored.
-
-### Kafka Consumers and Timestamp Extraction/Watermark Emission
-
-In many scenarios, the timestamp of a record is embedded (explicitly or
implicitly) in the record itself.
-In addition, the user may want to emit watermarks either periodically, or in
an irregular fashion, e.g. based on
-special records in the Kafka stream that contain the current event-time
watermark. For these cases, the Flink Kafka
-Consumer allows the specification of an `AssignerWithPeriodicWatermarks` or an
`AssignerWithPunctuatedWatermarks`.
-
-You can specify your custom timestamp extractor/watermark emitter as described
-[here]({{ site.baseurl }}/dev/event_timestamps_watermarks.html), or use one
from the
-[predefined ones]({{ site.baseurl }}/dev/event_timestamp_extractors.html).
After doing so, you
-can pass it to your consumer in the following way:
+在上面的例子中，当 Job 开始运行时， Consumer 将订阅名称与指定正则表达式匹配的所有主题（以 `test-topic` 开头并以单个数字结尾）。
+
+要允许 consumer 在作业开始运行后发现动态创建的主题，那么请为
`flink.partition-discovery.interval-millis` 设置非负值。 这允许 consumer
发现名称与指定模式匹配的新主题的分区。
+
+### Kafka Consumers 提交 Offset 的行为配置
+
+Flink Kafka Consumer 允许有配置如何将 offsets 提交回 Kafka broker （或 0.8 版本的 Zookeeper
）的行为。请注意： Flink Kafka Consumer 不依赖于提交的 offsets 来实现容错保证。提交的 offsets 只是一种方法，用于公开
consumer 的进度以便进行监控。
+
+配置 offset 提交行为的方法是否相同，取决于是否为 job 启用了 checkpointing 。
+
+ - *禁用 Checkpointing ：* 如果禁用了 checkpointing ，则 Flink Kafka Consumer 依赖于内部使用的
Kafka client 自动定期 offset 提交功能。
+ 因此，要禁用或启用 offset 的提交，只需将 `enable.auto.commit`（或 Kafka 0.8 的
`auto.commit.enable` ）或者 `auto.commit.interval.ms` 的Key 值设置为提供的 `Properties`
配置中的适当值。
+
+ - *启用 Checkpointing ：* 如果启用了 checkpointing ，那么当 checkpointing 完成时， Flink
Kafka Consumer 将提交的 offset 存储在 checkpoints 状态中。
+ 这确保 Kafka broker 中提交的 offset 与 checkpoints 状态中的 offset 一致。
+ 用户可以通过调用 consumer 上的 `setCommitOffsetsOnCheckpoints(boolean)` 方法来禁用或启用 offset
的提交(默认情况下，这个值是 true )。
+ 注意，在这个场景中，完全忽略 `Properties` 中的自动定期 offset 提交设置。
+
+### Kafka Consumers 和 Timestamp extractor 或者 Watermark Emission
+
+在许多场景中，记录的时间戳(显式或隐式)嵌入到记录本身中。此外，用户可能希望定期或以不规则的方式 Watermark Emission ，例如基于
Kafka 流中包含当前事件时间水位线的特殊记录。对于这些情况， Flink Kafka Consumer 允许指定
`AssignerWithPeriodicWatermarks` 或 `AssignerWithPunctuatedWatermarks`。
+
+你可以按照[此处]({{ site.baseurl }}/dev/event_timestamps_watermarks.html)的说明指定自定义
timestamp extractor 或者 Watermark Emission ，或者使用 [predefined ones]({{
site.baseurl }}/dev/event_timestamp_extractors.html)。你也可以通过以下方式将其传递给你的 consumer
：
Review comment:
```suggestion
你可以按照[此处]({{ site.baseurl
}}/zh/dev/event_timestamps_watermarks.html)的说明指定自定义的 timestamp 抽取和 watermark
发送 ，或者使用
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737523
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -54,31 +49,32 @@ For most users, the `FlinkKafkaConsumer08` (part of
`flink-connector-kafka`) is
FlinkKafkaConsumer08
FlinkKafkaProducer08
0.8.x
-Uses the https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example;>SimpleConsumer
API of Kafka internally. Offsets are committed to ZK by Flink.
+这个连接器在内部使用 Kafka 的 https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example;>SimpleConsumer
API 。偏移量由 Flink 提交给 ZK 。
+
flink-connector-kafka-0.9{{ site.scala_version_suffix }}
1.0.0
FlinkKafkaConsumer09
FlinkKafkaProducer09
0.9.x
-Uses the new http://kafka.apache.org/documentation.html#newconsumerapi;>Consumer
API Kafka.
+这个连接器使用新的 Kafka http://kafka.apache.org/documentation.html#newconsumerapi;>Consumer
API
flink-connector-kafka-0.10{{ site.scala_version_suffix }}
1.2.0
FlinkKafkaConsumer010
FlinkKafkaProducer010
0.10.x
-This connector supports https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message;>Kafka
messages with timestamps both for producing and consuming.
+这个连接器支持 https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message;>带有时间戳的
Kafka 消息，用于生产和消费。
flink-connector-kafka-0.11{{ site.scala_version_suffix }}
1.4.0
FlinkKafkaConsumer011
FlinkKafkaProducer011
0.11.x
-Since 0.11.x Kafka does not support scala 2.10. This connector
supports https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging;>Kafka
transactional messaging to provide exactly once semantic for the
producer.
+由于 Kafka 的 0.11.x 版本不支持 scala 2.10 ，所以此连接器通过支持 https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging;>Kafka
事务性的消息传递来为生产者提供 Exactly once 语义。
Review comment:
这里的 Since 应该不是“因为”的意思，和后半句话没有因果关系。
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737831
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -107,40 +100,36 @@ Then, import the connector in your maven project:
{% endhighlight %}
-Note that the streaming connectors are currently not part of the binary
distribution.
-See how to link with them for cluster execution [here]({{
site.baseurl}}/dev/projectsetup/dependencies.html).
+请注意：目前流连接器还不是二进制分发的一部分。
+[在此处]({{
site.baseurl}}/dev/projectsetup/dependencies.html)可以了解到如何链接它们以实现在集群中执行。
-## Installing Apache Kafka
+## 安装 Apache Kafka
-* Follow the instructions from [Kafka's
quickstart](https://kafka.apache.org/documentation.html#quickstart) to download
the code and launch a server (launching a Zookeeper and a Kafka server is
required every time before starting the application).
-* If the Kafka and Zookeeper servers are running on a remote machine, then the
`advertised.host.name` setting in the `config/server.properties` file must be
set to the machine's IP address.
+* 按照 [ Kafka
快速入门](https://kafka.apache.org/documentation.html#quickstart)的说明下载代码并启动 Kafka
服务器（每次启动应用程序之前都需要启动 Zookeeper 和 Kafka server）。
+* 如果 Kafka 和 Zookeeper 服务器运行在远端机器上，那么必须要将 `config/server.properties` 文件中的
`advertised.host.name`属性设置为远端设备的 IP 地址。
-## Kafka 1.0.0+ Connector
+## Kafka 1.0.0+ 连接器
-Starting with Flink 1.7, there is a new universal Kafka connector that does
not track a specific Kafka major version.
-Rather, it tracks the latest version of Kafka at the time of the Flink release.
+从 Flink 1.7 开始，有一个新的通用 Kafka 连接器，它不跟踪专用的 Kafka 主要版本。相反，它是在 Flink 发布时跟踪最新版本的
Kafka 。
+如果你的 Kafka broker 版本是 1.0.0 或 更新的版本，你应该使用这个 Kafka 连接器。
+如果你使用的是 Kafka 的旧版本( 0.11、0.10、0.9 或 0.8 )，那么你应该使用与 Kafka broker 版本相对应的连接器。
-If your Kafka broker version is 1.0.0 or newer, you should use this Kafka
connector.
-If you use an older version of Kafka (0.11, 0.10, 0.9, or 0.8), you should use
the connector corresponding to the broker version.
+### 兼容性
-### Compatibility
+通过 Kafka client API 和 broker 的兼容性保证，通用的 Kafka 连接器兼容较旧和较新的 Kafka broker 。
+它兼容 Kafka broker 0.11.0 或者更高版本，具体兼容性取决于所使用的功能。有关 Kafka 兼容性的详细信息，请参考 [ Kafka
文档](https://kafka.apache.org/protocol.html#protocol_compatibility)。
Review comment:
```suggestion
它兼容 Kafka broker 0.11.0 或者更高版本，具体兼容性取决于所使用的功能。有关 Kafka 兼容性的详细信息，请参考 [Kafka
文档](https://kafka.apache.org/protocol.html#protocol_compatibility)。
```
Remove additional space.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312739776
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -577,171 +474,101 @@ stream.addSink(myProducer);
val stream: DataStream[String] = ...
val myProducer = new FlinkKafkaProducer011[String](
-"localhost:9092", // broker list
-"my-topic", // target topic
-new SimpleStringSchema) // serialization schema
+"localhost:9092", // broker 列表
+"my-topic", // 目标 topic
+new SimpleStringSchema) // 序列化 schema
-// versions 0.10+ allow attaching the records' event timestamp when writing
them to Kafka;
-// this method is not available for earlier Kafka versions
+// 0.10+ 版本的 Kafka 允许在将记录写入 Kafka 时附加记录的事件时间戳；
+// 此方法不适用于早期版本的 Kafka
myProducer.setWriteTimestampToKafka(true)
stream.addSink(myProducer)
{% endhighlight %}
-The above examples demonstrate the basic usage of creating a Flink Kafka
Producer
-to write streams to a single Kafka target topic. For more advanced usages,
there
-are other constructor variants that allow providing the following:
-
- * *Providing custom properties*:
- The producer allows providing a custom properties configuration for the
internal `KafkaProducer`.
- Please refer to the [Apache Kafka
documentation](https://kafka.apache.org/documentation.html) for
- details on how to configure Kafka Producers.
- * *Custom partitioner*: To assign records to specific
- partitions, you can provide an implementation of a `FlinkKafkaPartitioner` to
the
- constructor. This partitioner will be called for each record in the stream
- to determine which exact partition of the target topic the record should be
sent to.
- Please see [Kafka Producer Partitioning
Scheme](#kafka-producer-partitioning-scheme) for more details.
- * *Advanced serialization schema*: Similar to the consumer,
- the producer also allows using an advanced serialization schema called
`KeyedSerializationSchema`,
- which allows serializing the key and value separately. It also allows to
override the target topic,
- so that one producer instance can send data to multiple topics.
-
-### Kafka Producer Partitioning Scheme
-
-By default, if a custom partitioner is not specified for the Flink Kafka
Producer, the producer will use
-a `FlinkFixedPartitioner` that maps each Flink Kafka Producer parallel subtask
to a single Kafka partition
-(i.e., all records received by a sink subtask will end up in the same Kafka
partition).
-
-A custom partitioner can be implemented by extending the
`FlinkKafkaPartitioner` class. All
-Kafka versions' constructors allow providing a custom partitioner when
instantiating the producer.
-Note that the partitioner implementation must be serializable, as they will be
transferred across Flink nodes.
-Also, keep in mind that any state in the partitioner will be lost on job
failures since the partitioner
-is not part of the producer's checkpointed state.
-
-It is also possible to completely avoid using and kind of partitioner, and
simply let Kafka partition
-the written records by their attached key (as determined for each record using
the provided serialization schema).
-To do this, provide a `null` custom partitioner when instantiating the
producer. It is important
-to provide `null` as the custom partitioner; as explained above, if a custom
partitioner is not specified
-the `FlinkFixedPartitioner` is used instead.
-
-### Kafka Producers and Fault Tolerance
+上面的例子演示了创建 Flink Kafka Producer 来将流消息写入单个 Kafka 目标 topic 的基本用法。
+对于更高级的用法，这还有其他构造函数变体允许提供以下内容：
+
+ * *提供自定义属性*： producer 允许为内部 `KafkaProducer` 提供自定义属性配置。 有关如何配置 Kafka Producers
的详细信息，请参阅 [Apache Kafka 文档](https://kafka.apache.org/documentation.html)。
+ * *自定义分区器*：要将消息分配给特定的分区，可以向构造函数提供一个 `FlinkKafkaPartitioner`
的实现。这个分区器将被流中的每条记录调用，以确定消息应该发送到目标 topic 的哪个具体分区里。有关详细信息，请参阅 [Kafka Producer
分区方案](#kafka-producer-partitioning-scheme)。
+ * *高级的序列化 schema* ：与 consumer 类似， producer 还允许使用名为 `KeyedSerializationSchema`
的高级序列化 schema ，该 schema 允许单独序列化 key 和 value 。它还允许覆盖目标 topic ，以便 producer
实例可以将数据发送到多个 topics 。
+
+### Kafka Producer 分区方案
+
+默认情况下，如果没有为 Flink Kafka Producer 指定自定义分区程序，则 producer 将使用
`FlinkFixedPartitioner` 为每个 Flink Kafka Producer 并行子任务映射到单个 Kafka
分区（即，接收子任务接收到的所有消息都将位于同一个 Kafka 分区中）。
+
+可以通过扩展 `FlinkKafkaPartitioner` 类来实现自定义分区程序。 所有 Kafka 版本的构造函数都允许在实例化 producer
时提供自定义分区程序。
+注意：分区器实现必须是可序列化的，因为它们将在 Flink 节点之间传输。此外，请记住分区器中的任何状态都将在作业失败时丢失，因为分区器不是
producer 的检查点状态的一部分。
+
+也可以完全避免使用分区器，并简单地让 Kafka 通过其附加 key 写入的消息进行分区（使用提供的序列化 schema 为每条记录确定分区）。
+为此，在实例化 producer 时提供 `null` 自定义分区程序，提供 `null` 作为自定义分区器是很重要的;
如上所述，如果未指定自定义分区程序，则默认使用 `FlinkFixedPartitioner` 。
+
+### Kafka Producers 和容错
Kafka 0.8
-Before 0.9 Kafka did not provide any mechanisms to guarantee at-least-once or
exactly-once semantics.
+在 0.9 版本之前， Kafka
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737997
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -150,44 +139,40 @@ To use the universal Kafka connector add a dependency to
it:
{% endhighlight %}
-Then instantiate the new source (`FlinkKafkaConsumer`) and sink
(`FlinkKafkaProducer`).
-The API is backward compatible with the Kafka 0.11 connector,
-except of dropping specific Kafka version from the module and class names.
+然后，实例化 source （ `FlinkKafkaConsumer` ） 和 sink （ `FlinkKafkaProducer` ）
。除了从模块和类名中删除了特定的 Kafka 版本外，这个 API 向后兼容 Kafka 0.11 版本的 connector 。
## Kafka Consumer
-Flink's Kafka consumer is called `FlinkKafkaConsumer08` (or 09 for Kafka
0.9.0.x versions, etc.
-or just `FlinkKafkaConsumer` for Kafka >= 1.0.0 versions). It provides access
to one or more Kafka topics.
+Flink 的 Kafka consumer 称为 `FlinkKafkaConsumer08` （或者 Kafka 0.9.0.x 版本的 Kafka
consumer 是 `FlinkKafkaConsumer09` ，或仅 FlinkKafkaConsumer 适用于 Kafka >= 1.0.0
的版本）。它提供对一个或多个 Kafka topics 的访问。
-The constructor accepts the following arguments:
+构造函数接受以下参数：
-1. The topic name / list of topic names
-2. A DeserializationSchema / KafkaDeserializationSchema for deserializing the
data from Kafka
-3. Properties for the Kafka consumer.
- The following properties are required:
- - "bootstrap.servers" (comma separated list of Kafka brokers)
- - "zookeeper.connect" (comma separated list of Zookeeper servers) (**only
required for Kafka 0.8**)
- - "group.id" the id of the consumer group
+1. Topic 名称或者 Topics 的名称列表
Review comment:
```suggestion
1. Topic 名称或者名称列表
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312738718 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -342,74 +300,51 @@ myConsumer.setStartFromSpecificOffsets(specificStartOffsets) -The above example configures the consumer to start from the specified offsets for -partitions 0, 1, and 2 of topic `myTopic`. The offset values should be the -next record that the consumer should read for each partition. Note that -if the consumer needs to read a partition which does not have a specified -offset within the provided offsets map, it will fallback to the default -group offsets behaviour (i.e. `setStartFromGroupOffsets()`) for that -particular partition. +上面的例子中使用的配置是指定从 `myTopic` 主题的 0 、1 和 2 分区的指定偏移量开始消费。 offset 值是 consumer 应该为每个分区读取的下一条消息。请注意：如果 consumer 需要读取在提供的 offsets 映射中没有指定 offsets 的分区，那么它将回退到该特定分区的默认组偏移行为（即 `setStartFromGroupOffsets()` ）。 + -Note that these start position configuration methods do not affect the start position when the job is -automatically restored from a failure or manually restored using a savepoint. -On restore, the start position of each Kafka partition is determined by the -offsets stored in the savepoint or checkpoint -(please see the next section for information about checkpointing to enable -fault tolerance for the consumer). +请注意：当 Job 从故障中自动恢复或使用 savepoint 手动恢复时，这些起始位置配置方法不会影响消费的起始位置。在恢复时，每个 Kafka 分区的起始位置由存储在 savepoint 或 checkpoint 中的 offsets 确定（有关 checkpointing 的信息，请参阅下一节，以便为 consumer 启用容错功能）。 -### Kafka Consumers and Fault Tolerance +### Kafka Consumers 和容错 -With Flink's checkpointing enabled, the Flink Kafka Consumer will consume records from a topic and periodically checkpoint all -its Kafka offsets, together with the state of other operations, in a consistent manner. In case of a job failure, Flink will restore -the streaming program to the state of the latest checkpoint and re-consume the records from Kafka, starting from the offsets that were -stored in the checkpoint. +伴随着启用 Flink 的 checkpointing 后，Flink Kafka Consumer 将使用 topic 中的记录，并以一致的方式定期检查其所有 Kafka offsets 和其他算子的状态。``如果 Job 失败， Flink 会将流式程序恢复到最新 checkpoint 的状态，并从存储在 checkpoint 中的 offsets 开始重新消费 Kafka 中的消息。 Review comment: ```suggestion 伴随着启用 Flink 的 checkpointing 后，Flink Kafka Consumer 将使用 topic 中的记录，并以一致的方式定期检查其所有 Kafka offsets 和其他算子的状态。如果 Job 失败， Flink 会将流式程序恢复到最新 checkpoint 的状态，并从存储在 checkpoint 中的 offsets 开始重新消费 Kafka 中的消息。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312739024
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -448,61 +383,36 @@ val stream = env.addSource(myConsumer)
-In the above example, all topics with names that match the specified regular
expression
-(starting with `test-topic-` and ending with a single digit) will be
subscribed by the consumer
-when the job starts running.
-
-To allow the consumer to discover dynamically created topics after the job
started running,
-set a non-negative value for `flink.partition-discovery.interval-millis`. This
allows
-the consumer to discover partitions of new topics with names that also match
the specified
-pattern.
-
-### Kafka Consumers Offset Committing Behaviour Configuration
-
-The Flink Kafka Consumer allows configuring the behaviour of how offsets
-are committed back to Kafka brokers (or Zookeeper in 0.8). Note that the
-Flink Kafka Consumer does not rely on the committed offsets for fault
-tolerance guarantees. The committed offsets are only a means to expose
-the consumer's progress for monitoring purposes.
-
-The way to configure offset commit behaviour is different, depending on
-whether or not checkpointing is enabled for the job.
-
- - *Checkpointing disabled:* if checkpointing is disabled, the Flink Kafka
- Consumer relies on the automatic periodic offset committing capability
- of the internally used Kafka clients. Therefore, to disable or enable offset
- committing, simply set the `enable.auto.commit` (or `auto.commit.enable`
- for Kafka 0.8) / `auto.commit.interval.ms` keys to appropriate values
- in the provided `Properties` configuration.
-
- - *Checkpointing enabled:* if checkpointing is enabled, the Flink Kafka
- Consumer will commit the offsets stored in the checkpointed states when
- the checkpoints are completed. This ensures that the committed offsets
- in Kafka brokers is consistent with the offsets in the checkpointed states.
- Users can choose to disable or enable offset committing by calling the
- `setCommitOffsetsOnCheckpoints(boolean)` method on the consumer (by default,
- the behaviour is `true`).
- Note that in this scenario, the automatic periodic offset committing
- settings in `Properties` is completely ignored.
-
-### Kafka Consumers and Timestamp Extraction/Watermark Emission
-
-In many scenarios, the timestamp of a record is embedded (explicitly or
implicitly) in the record itself.
-In addition, the user may want to emit watermarks either periodically, or in
an irregular fashion, e.g. based on
-special records in the Kafka stream that contain the current event-time
watermark. For these cases, the Flink Kafka
-Consumer allows the specification of an `AssignerWithPeriodicWatermarks` or an
`AssignerWithPunctuatedWatermarks`.
-
-You can specify your custom timestamp extractor/watermark emitter as described
-[here]({{ site.baseurl }}/dev/event_timestamps_watermarks.html), or use one
from the
-[predefined ones]({{ site.baseurl }}/dev/event_timestamp_extractors.html).
After doing so, you
-can pass it to your consumer in the following way:
+在上面的例子中，当 Job 开始运行时， Consumer 将订阅名称与指定正则表达式匹配的所有主题（以 `test-topic` 开头并以单个数字结尾）。
+
+要允许 consumer 在作业开始运行后发现动态创建的主题，那么请为
`flink.partition-discovery.interval-millis` 设置非负值。 这允许 consumer
发现名称与指定模式匹配的新主题的分区。
+
+### Kafka Consumers 提交 Offset 的行为配置
+
+Flink Kafka Consumer 允许有配置如何将 offsets 提交回 Kafka broker （或 0.8 版本的 Zookeeper
）的行为。请注意： Flink Kafka Consumer 不依赖于提交的 offsets 来实现容错保证。提交的 offsets 只是一种方法，用于公开
consumer 的进度以便进行监控。
+
+配置 offset 提交行为的方法是否相同，取决于是否为 job 启用了 checkpointing 。
+
+ - *禁用 Checkpointing ：* 如果禁用了 checkpointing ，则 Flink Kafka Consumer 依赖于内部使用的
Kafka client 自动定期 offset 提交功能。
+ 因此，要禁用或启用 offset 的提交，只需将 `enable.auto.commit`（或 Kafka 0.8 的
`auto.commit.enable` ）或者 `auto.commit.interval.ms` 的Key 值设置为提供的 `Properties`
配置中的适当值。
+
+ - *启用 Checkpointing ：* 如果启用了 checkpointing ，那么当 checkpointing 完成时， Flink
Kafka Consumer 将提交的 offset 存储在 checkpoints 状态中。
+ 这确保 Kafka broker 中提交的 offset 与 checkpoints 状态中的 offset 一致。
+ 用户可以通过调用 consumer 上的 `setCommitOffsetsOnCheckpoints(boolean)` 方法来禁用或启用 offset
的提交(默认情况下，这个值是 true )。
+ 注意，在这个场景中，完全忽略 `Properties` 中的自动定期 offset 提交设置。
+
+### Kafka Consumers 和 Timestamp extractor 或者 Watermark Emission
+
+在许多场景中，记录的时间戳(显式或隐式)嵌入到记录本身中。此外，用户可能希望定期或以不规则的方式 Watermark Emission ，例如基于
Kafka 流中包含当前事件时间水位线的特殊记录。对于这些情况， Flink Kafka Consumer 允许指定
`AssignerWithPeriodicWatermarks` 或 `AssignerWithPunctuatedWatermarks`。
Review comment:
```suggestion
在许多场景中，记录的时间戳(显式或隐式)嵌入到记录本身中。此外，用户可能希望定期或以不规则的方式发送 watermark ，例如基于 Kafka
流中包含当前事件时间水位线的特殊记录。对于这些情况， Flink Kafka Consumer 允许指定
`AssignerWithPeriodicWatermarks` 或 `AssignerWithPunctuatedWatermarks`。
```
This is an automated message from the Apache Git Service.
To respond to the
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737632
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -87,17 +83,14 @@ For most users, the `FlinkKafkaConsumer08` (part of
`flink-connector-kafka`) is
FlinkKafkaProducer
>= 1.0.0
-This universal Kafka connector attempts to track the latest version of
the Kafka client.
-The version of the client it uses may change between Flink releases.
Starting with Flink 1.9 release, it uses the Kafka 2.2.0 client.
-Modern Kafka clients are backwards compatible with broker versions
0.10.0 or later.
-However for Kafka 0.11.x and 0.10.x versions, we recommend using
dedicated
-flink-connector-kafka-0.11{{ site.scala_version_suffix }} and
flink-connector-kafka-0.10{{ site.scala_version_suffix }} respectively.
+这个通用的 Kafka 连接器企图与最新版本的 Kafka client 保持同步，连接器使用的 Kafka client 版本可能会在
Flink 版本之间发生变化。从 Flink 1.9 版本开始，它使用 Kafka 2.2.0 client 。当前 Kafka 客户端向后兼容 0.10.0
或更高版本的 Kafka broker 。
Review comment:
"企图"的翻译有一些贬义了。
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737704
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -107,40 +100,36 @@ Then, import the connector in your maven project:
{% endhighlight %}
-Note that the streaming connectors are currently not part of the binary
distribution.
-See how to link with them for cluster execution [here]({{
site.baseurl}}/dev/projectsetup/dependencies.html).
+请注意：目前流连接器还不是二进制分发的一部分。
+[在此处]({{
site.baseurl}}/dev/projectsetup/dependencies.html)可以了解到如何链接它们以实现在集群中执行。
Review comment:
```suggestion
[在此处]({{
site.baseurl}}/zh/dev/projectsetup/dependencies.html)可以了解到如何链接它们以实现在集群中执行。
```
Use a Chinese link here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738797
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -342,74 +300,51 @@
myConsumer.setStartFromSpecificOffsets(specificStartOffsets)
-The above example configures the consumer to start from the specified offsets
for
-partitions 0, 1, and 2 of topic `myTopic`. The offset values should be the
-next record that the consumer should read for each partition. Note that
-if the consumer needs to read a partition which does not have a specified
-offset within the provided offsets map, it will fallback to the default
-group offsets behaviour (i.e. `setStartFromGroupOffsets()`) for that
-particular partition.
+上面的例子中使用的配置是指定从 `myTopic` 主题的 0 、1 和 2 分区的指定偏移量开始消费。 offset 值是 consumer
应该为每个分区读取的下一条消息。请注意：如果 consumer 需要读取在提供的 offsets 映射中没有指定 offsets
的分区，那么它将回退到该特定分区的默认组偏移行为（即 `setStartFromGroupOffsets()` ）。
+
-Note that these start position configuration methods do not affect the start
position when the job is
-automatically restored from a failure or manually restored using a savepoint.
-On restore, the start position of each Kafka partition is determined by the
-offsets stored in the savepoint or checkpoint
-(please see the next section for information about checkpointing to enable
-fault tolerance for the consumer).
+请注意：当 Job 从故障中自动恢复或使用 savepoint 手动恢复时，这些起始位置配置方法不会影响消费的起始位置。在恢复时，每个 Kafka
分区的起始位置由存储在 savepoint 或 checkpoint 中的 offsets 确定（有关 checkpointing
的信息，请参阅下一节，以便为 consumer 启用容错功能）。
-### Kafka Consumers and Fault Tolerance
+### Kafka Consumers 和容错
-With Flink's checkpointing enabled, the Flink Kafka Consumer will consume
records from a topic and periodically checkpoint all
-its Kafka offsets, together with the state of other operations, in a
consistent manner. In case of a job failure, Flink will restore
-the streaming program to the state of the latest checkpoint and re-consume the
records from Kafka, starting from the offsets that were
-stored in the checkpoint.
+伴随着启用 Flink 的 checkpointing 后，Flink Kafka Consumer 将使用 topic
中的记录，并以一致的方式定期检查其所有 Kafka offsets 和其他算子的状态。``如果 Job 失败， Flink 会将流式程序恢复到最新
checkpoint 的状态，并从存储在 checkpoint 中的 offsets 开始重新消费 Kafka 中的消息。
-The interval of drawing checkpoints therefore defines how much the program may
have to go back at most, in case of a failure.
+因此，设置 checkpoints 的间隔定义了程序在发生故障时最多需要返回多少。
-To use fault tolerant Kafka Consumers, checkpointing of the topology needs to
be enabled at the execution environment:
+要使用容错的 Kafka Consumers ，需要在执行环境中启用拓扑的 checkpointing 。
{% highlight java %}
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
-env.enableCheckpointing(5000); // checkpoint every 5000 msecs
+env.enableCheckpointing(5000); // 每隔 5000 毫秒 执行一次 checkpoint
{% endhighlight %}
{% highlight scala %}
val env = StreamExecutionEnvironment.getExecutionEnvironment()
-env.enableCheckpointing(5000) // checkpoint every 5000 msecs
+env.enableCheckpointing(5000) // 每隔 5000 毫秒 执行一次 checkpoint
{% endhighlight %}
-Also note that Flink can only restart the topology if enough processing slots
are available to restart the topology.
-So if the topology fails due to loss of a TaskManager, there must still be
enough slots available afterwards.
-Flink on YARN supports automatic restart of lost YARN containers.
+另请注意，如果有足够的处理 solts 是可用于重新启动拓扑的，那么 Flink 只能重新启动拓扑计划。因此，如果拓扑由于丢失了 TaskManager
而失败，那么之后必须要一直有足够可用的 solts 。 Flink on YARN 支持自动重启丢失的 YARN 容器。
Review comment:
```suggestion
另请注意，Flink 只有在有足够的可用 solts 情况下才重新启动拓扑。因此，如果拓扑由于丢失了 TaskManager
而失败，那么之后必须要有足够可用的 solts 。 Flink on YARN 支持自动重启丢失的 YARN 容器。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312739365 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -532,41 +442,28 @@ stream = env -Internally, an instance of the assigner is executed per Kafka partition. -When such an assigner is specified, for each record read from Kafka, the -`extractTimestamp(T element, long previousElementTimestamp)` is called to assign a timestamp to the record and -the `Watermark getCurrentWatermark()` (for periodic) or the -`Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` (for punctuated) is called to determine -if a new watermark should be emitted and with which timestamp. - -**Note**: If a watermark assigner depends on records read from Kafka to advance its watermarks -(which is commonly the case), all topics and partitions need to have a continuous stream of records. -Otherwise, the watermarks of the whole application cannot advance and all time-based operations, -such as time windows or functions with timers, cannot make progress. A single idle Kafka partition causes this behavior. -A Flink improvement is planned to prevent this from happening -(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions]( -https://issues.apache.org/jira/browse/FLINK-5479)). -In the meanwhile, a possible workaround is to send *heartbeat messages* to all consumed partitions that advance the watermarks of idle partitions. +在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （用于标点符号）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 + +**请注意**： 如果水位线 assigner 依赖于从 Kafka 读取的消息来上涨其水位(通常就是这种情况)，那么所有主题和分区都需要有连续的消息流。否则，整个应用程序的水位将无法上涨，所有基于时间的算子(例如时间窗口或带有计时器的函数)也无法运行。单个的 Kafka 分区也会导致这种反应。这是一个已在计划中的 Flink improvement ，目的是为了防止这种情况发生（请见[FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions](https://issues.apache.org/jira/browse/FLINK-5479)）。同时，可能的解决方法是将*心跳消息*发送到所有 consumer 的分区里，从而上涨空闲分区的水位。 Review comment: ```suggestion **请注意**： 如果 watermark assigner 依赖于从 Kafka 读取的消息来上涨其 watermark (通常就是这种情况)，那么所有主题和分区都需要有连续的消息流。否则，整个应用程序的 watermark 将无法上涨，所有基于时间的算子(例如时间窗口或带有计时器的函数)也无法运行。单个的 Kafka 分区也会导致这种反应。这是一个已在计划中的 Flink 改进，目的是为了防止这种情况发生（请见[FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions](https://issues.apache.org/jira/browse/FLINK-5479)）。同时，可能的解决方法是将*心跳消息*发送到所有 consumer 的分区里，从而上涨空闲分区的 watermark。 ``` 我更倾向于对 watermark 保持同一种翻译，或者都不翻译。 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312739219 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -532,41 +442,28 @@ stream = env -Internally, an instance of the assigner is executed per Kafka partition. -When such an assigner is specified, for each record read from Kafka, the -`extractTimestamp(T element, long previousElementTimestamp)` is called to assign a timestamp to the record and -the `Watermark getCurrentWatermark()` (for periodic) or the -`Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` (for punctuated) is called to determine -if a new watermark should be emitted and with which timestamp. - -**Note**: If a watermark assigner depends on records read from Kafka to advance its watermarks -(which is commonly the case), all topics and partitions need to have a continuous stream of records. -Otherwise, the watermarks of the whole application cannot advance and all time-based operations, -such as time windows or functions with timers, cannot make progress. A single idle Kafka partition causes this behavior. -A Flink improvement is planned to prevent this from happening -(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions]( -https://issues.apache.org/jira/browse/FLINK-5479)). -In the meanwhile, a possible workaround is to send *heartbeat messages* to all consumed partitions that advance the watermarks of idle partitions. +在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （用于标点符号）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 Review comment: ```suggestion 在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期形式）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （打点形式）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737335
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -26,25 +26,20 @@ under the License.
* This will be replaced by the TOC
{:toc}
-This connector provides access to event streams served by [Apache
Kafka](https://kafka.apache.org/).
+此连接器提供了访问 [Apache Kafka](https://kafka.apache.org/) 事件流的服务。
-Flink provides special Kafka Connectors for reading and writing data from/to
Kafka topics.
-The Flink Kafka Consumer integrates with Flink's checkpointing mechanism to
provide
-exactly-once processing semantics. To achieve that, Flink does not purely rely
on Kafka's consumer group
-offset tracking, but tracks and checkpoints these offsets internally as well.
+Flink 提供了专门的 Kafka 连接器，向 Kafka topics 中读取或者写入数据。 Flink Kafka Consumer 集成了
Flink 的 Checkpoint 机制，可提供 exactly-once 的处理语义。为此， Flink 并不完全依赖于跟踪 Kafka
消费组的偏移量，而是在内部跟踪和检查偏移量。
Review comment:
```suggestion
Flink 提供了专门的 Kafka 连接器，向 Kafka topic 中读取或者写入数据。 Flink Kafka Consumer 集成了
Flink 的 Checkpoint 机制，可提供 exactly-once 的处理语义。为此， Flink 并不完全依赖于跟踪 Kafka
消费组的偏移量，而是在内部跟踪和检查偏移量。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312738167 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -197,41 +182,31 @@ stream = env -### The `DeserializationSchema` +### `DeserializationSchema` -The Flink Kafka Consumer needs to know how to turn the binary data in Kafka into Java/Scala objects. The -`DeserializationSchema` allows users to specify such a schema. The `T deserialize(byte[] message)` -method gets called for each Kafka message, passing the value from Kafka. +Flink Kafka Consumer 需要知道如何将 Kafka 中的二进制数据转换为 Java 或者 Scala 对象。 `DeserializationSchema` 允许用户指定这样的 schema ，为每条 Kafka 消息调用 `T deserialize(byte[] message)` 方法，从 Kafka 中传递值。 -It is usually helpful to start from the `AbstractDeserializationSchema`, which takes care of describing the -produced Java/Scala type to Flink's type system. Users that implement a vanilla `DeserializationSchema` need -to implement the `getProducedType(...)` method themselves. +从 `AbstractDeserializationSchema` 开始通常很有帮助，它负责将生成的 Java 或 Scala 类型描述为 Flink 的类型系统。 +实现带有 vanilla `DeserializationSchema` 的用户需要自己实现 `getProducedType（...）` 方法。 Review comment: ```suggestion 用户如果要自己去实现一个`DeserializationSchema`，需要自己去实现 `getProducedType(...)`方法。 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737779
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -107,40 +100,36 @@ Then, import the connector in your maven project:
{% endhighlight %}
-Note that the streaming connectors are currently not part of the binary
distribution.
-See how to link with them for cluster execution [here]({{
site.baseurl}}/dev/projectsetup/dependencies.html).
+请注意：目前流连接器还不是二进制分发的一部分。
+[在此处]({{
site.baseurl}}/dev/projectsetup/dependencies.html)可以了解到如何链接它们以实现在集群中执行。
-## Installing Apache Kafka
+## 安装 Apache Kafka
-* Follow the instructions from [Kafka's
quickstart](https://kafka.apache.org/documentation.html#quickstart) to download
the code and launch a server (launching a Zookeeper and a Kafka server is
required every time before starting the application).
-* If the Kafka and Zookeeper servers are running on a remote machine, then the
`advertised.host.name` setting in the `config/server.properties` file must be
set to the machine's IP address.
+* 按照 [ Kafka
快速入门](https://kafka.apache.org/documentation.html#quickstart)的说明下载代码并启动 Kafka
服务器（每次启动应用程序之前都需要启动 Zookeeper 和 Kafka server）。
+* 如果 Kafka 和 Zookeeper 服务器运行在远端机器上，那么必须要将 `config/server.properties` 文件中的
`advertised.host.name`属性设置为远端设备的 IP 地址。
-## Kafka 1.0.0+ Connector
+## Kafka 1.0.0+ 连接器
-Starting with Flink 1.7, there is a new universal Kafka connector that does
not track a specific Kafka major version.
-Rather, it tracks the latest version of Kafka at the time of the Flink release.
+从 Flink 1.7 开始，有一个新的通用 Kafka 连接器，它不跟踪专用的 Kafka 主要版本。相反，它是在 Flink 发布时跟踪最新版本的
Kafka 。
Review comment:
```suggestion
从 Flink 1.7 开始，有一个新的通用 Kafka 连接器，它不跟踪特定的 Kafka 主版本。相反，它是在 Flink 发布时跟踪最新版本的
Kafka 。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312738501
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -197,41 +182,31 @@ stream = env
-### The `DeserializationSchema`
+### `DeserializationSchema`
-The Flink Kafka Consumer needs to know how to turn the binary data in Kafka
into Java/Scala objects. The
-`DeserializationSchema` allows users to specify such a schema. The `T
deserialize(byte[] message)`
-method gets called for each Kafka message, passing the value from Kafka.
+Flink Kafka Consumer 需要知道如何将 Kafka 中的二进制数据转换为 Java 或者 Scala 对象。
`DeserializationSchema` 允许用户指定这样的 schema ，为每条 Kafka 消息调用 `T deserialize(byte[]
message)` 方法，从 Kafka 中传递值。
-It is usually helpful to start from the `AbstractDeserializationSchema`, which
takes care of describing the
-produced Java/Scala type to Flink's type system. Users that implement a
vanilla `DeserializationSchema` need
-to implement the `getProducedType(...)` method themselves.
+从 `AbstractDeserializationSchema` 开始通常很有帮助，它负责将生成的 Java 或 Scala 类型描述为 Flink
的类型系统。
+实现带有 vanilla `DeserializationSchema` 的用户需要自己实现 `getProducedType（...）` 方法。
-For accessing the key, value and metadata of the Kafka message, the
`KafkaDeserializationSchema` has
-the following deserialize method `T deserialize(ConsumerRecord
record)`.
+为了访问 Kafka 消息的 key 、 value 和元数据， `KafkaDeserializationSchema` 具有以下反序列化方法 `T
deserialize(ConsumerRecord record)` 。
-For convenience, Flink provides the following schemas:
+为了方便使用， Flink 提供了以下几种 schemas ：
-1. `TypeInformationSerializationSchema` (and
`TypeInformationKeyValueSerializationSchema`) which creates
-a schema based on a Flink's `TypeInformation`. This is useful if the data
is both written and read by Flink.
-This schema is a performant Flink-specific alternative to other generic
serialization approaches.
+1. `TypeInformationSerializationSchema` （和
`TypeInformationKeyValueSerializationSchema`) 基于 Flink 的 `TypeInformation` 创建
`schema` 。
+如果 Flink 既负责数据的读也负责写，那么这将是非常有用的。此 schema 是其他通用序列化方法的高性能 Flink 替代方案。
-2. `JsonDeserializationSchema` (and `JSONKeyValueDeserializationSchema`) which
turns the serialized JSON
-into an ObjectNode object, from which fields can be accessed using
`objectNode.get("field").as(Int/String/...)()`.
-The KeyValue objectNode contains a "key" and "value" field which contain
all fields, as well as
-an optional "metadata" field that exposes the offset/partition/topic for
this message.
+2. `JsonDeserializationSchema` （和 `JSONKeyValueDeserializationSchema`） 将序列化的
JSON 转化为 ObjectNode 对象，可以使用 `objectNode.get("field").as(Int/String/...)()`
来访问某个字段。
+KeyValue objectNode 包含一个含所有字段的 key 和 values 字段，以及公开此消息的 offset 或 partition
或 topic 的可选“元数据”字段。
Review comment:
```suggestion
KeyValue objectNode 包含一个含所有字段的 key 和 values
字段，以及一个可选的"metadata"字段，可以访问到消息的 offset、partition、topic 等信息。
```
这里 "metadata" 是需要作为 key 去访问的，所以不建议翻译。
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C… URL: https://github.com/apache/flink/pull/9415#discussion_r312739429 ## File path: docs/dev/connectors/kafka.zh.md ## @@ -532,41 +442,28 @@ stream = env -Internally, an instance of the assigner is executed per Kafka partition. -When such an assigner is specified, for each record read from Kafka, the -`extractTimestamp(T element, long previousElementTimestamp)` is called to assign a timestamp to the record and -the `Watermark getCurrentWatermark()` (for periodic) or the -`Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` (for punctuated) is called to determine -if a new watermark should be emitted and with which timestamp. - -**Note**: If a watermark assigner depends on records read from Kafka to advance its watermarks -(which is commonly the case), all topics and partitions need to have a continuous stream of records. -Otherwise, the watermarks of the whole application cannot advance and all time-based operations, -such as time windows or functions with timers, cannot make progress. A single idle Kafka partition causes this behavior. -A Flink improvement is planned to prevent this from happening -(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions]( -https://issues.apache.org/jira/browse/FLINK-5479)). -In the meanwhile, a possible workaround is to send *heartbeat messages* to all consumed partitions that advance the watermarks of idle partitions. +在内部，每个 Kafka 分区执行一个 assigner 实例。当指定了这样的 assigner 时，对于从 Kafka 读取的每条消息，调用 `extractTimestamp(T element, long previousElementTimestamp)` 来为记录分配时间戳，并为 `Watermark getCurrentWatermark()` （定期）或 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` （用于标点符号）以确定是否应该发出新的 watermark 以及使用哪个时间戳。 + +**请注意**： 如果水位线 assigner 依赖于从 Kafka 读取的消息来上涨其水位(通常就是这种情况)，那么所有主题和分区都需要有连续的消息流。否则，整个应用程序的水位将无法上涨，所有基于时间的算子(例如时间窗口或带有计时器的函数)也无法运行。单个的 Kafka 分区也会导致这种反应。这是一个已在计划中的 Flink improvement ，目的是为了防止这种情况发生（请见[FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions](https://issues.apache.org/jira/browse/FLINK-5479)）。同时，可能的解决方法是将*心跳消息*发送到所有 consumer 的分区里，从而上涨空闲分区的水位。 ## Kafka Producer -Flink’s Kafka Producer is called `FlinkKafkaProducer011` (or `010` for Kafka 0.10.0.x versions, etc. or just `FlinkKafkaProducer` for Kafka >= 1.0.0 versions). -It allows writing a stream of records to one or more Kafka topics. +Flink Kafka Producer 被称为 `FlinkKafkaProducer011` （或者 `010` 只适合 Kafka 0.10.0.x 的版本使用 ，或 `FlinkKafkaProducer` 只适用于 Kafka> = 1.0.0 的版本使用）。它允许将消息流写入一个或多个 Kafka topics 。 -Example: +举例： Review comment: ```suggestion 示例： ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh] Translate "Apache Kafka Connector" page into C…
wuchong commented on a change in pull request #9415: [FLINK-12939][docs-zh]
Translate "Apache Kafka Connector" page into C…
URL: https://github.com/apache/flink/pull/9415#discussion_r312737610
##
File path: docs/dev/connectors/kafka.zh.md
##
@@ -87,17 +83,14 @@ For most users, the `FlinkKafkaConsumer08` (part of
`flink-connector-kafka`) is
FlinkKafkaProducer
>= 1.0.0
-This universal Kafka connector attempts to track the latest version of
the Kafka client.
-The version of the client it uses may change between Flink releases.
Starting with Flink 1.9 release, it uses the Kafka 2.2.0 client.
-Modern Kafka clients are backwards compatible with broker versions
0.10.0 or later.
-However for Kafka 0.11.x and 0.10.x versions, we recommend using
dedicated
-flink-connector-kafka-0.11{{ site.scala_version_suffix }} and
flink-connector-kafka-0.10{{ site.scala_version_suffix }} respectively.
+这个通用的 Kafka 连接器企图与最新版本的 Kafka client 保持同步，连接器使用的 Kafka client 版本可能会在
Flink 版本之间发生变化。从 Flink 1.9 版本开始，它使用 Kafka 2.2.0 client 。当前 Kafka 客户端向后兼容 0.10.0
或更高版本的 Kafka broker 。
Review comment:
```suggestion
这个通用的 Kafka 连接器尽力与 Kafka client 的最新版本保持同步。该连接器使用的 Kafka client
版本可能会在 Flink 版本之间发生变化。从 Flink 1.9 版本开始，它使用 Kafka 2.2.0 client 。当前 Kafka 客户端向后兼容
0.10.0 或更高版本的 Kafka broker 。
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-13678) Translate "Code Style - Preamble" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904642#comment-16904642 ] WangHengWei commented on FLINK-13678: - I want to work on this. would you please assign it to me? > Translate "Code Style - Preamble" page into Chinese > --- > > Key: FLINK-13678 > URL: https://issues.apache.org/jira/browse/FLINK-13678 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Priority: Major > > Translate page > https://flink.apache.org/zh/contributing/code-style-and-quality-preamble.html > into Chinese. The page is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/code-style-and-quality-scala.zh.md. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] YngwieWang commented on a change in pull request #9348: [FLINK-13505][docs-zh] Translate "Java Lambda Expressions" page into Chinese
YngwieWang commented on a change in pull request #9348: [FLINK-13505][docs-zh]
Translate "Java Lambda Expressions" page into Chinese
URL: https://github.com/apache/flink/pull/9348#discussion_r312737566
##
File path: docs/dev/java_lambdas.zh.md
##
@@ -22,32 +22,28 @@ specific language governing permissions and limitations
under the License.
-->
-Java 8 introduced several new language features designed for faster and
clearer coding. With the most important feature,
-the so-called "Lambda Expressions", it opened the door to functional
programming. Lambda expressions allow for implementing and
-passing functions in a straightforward way without having to declare
additional (anonymous) classes.
+Java 8 引入了几种新的语言特性，旨在实现更快、更清晰的编码。 作为最重要的特性，即所谓的“Lambda
表达式”，它开启了函数式编程的大门。Lambda 表达式允许以简捷的方式实现和传递函数，而无需声明额外的（匿名）类。
-Attention Flink supports the usage of
lambda expressions for all operators of the Java API, however, whenever a
lambda expression uses Java generics you need to declare type information
*explicitly*.
+注意 Flink 支持对 Java API 的所有算子使用 Lambda
表达式，但是，当 Lambda 表达式使用 Java 泛型时，你需要 *显式* 声明类型信息。
-This document shows how to use lambda expressions and describes current
limitations. For a general introduction to the
-Flink API, please refer to the [Programming Guide]({{ site.baseurl
}}/dev/api_concepts.html)
+本文档介绍了如何使用 Lambda 表达式并描述了其在当前应用中的限制。有关 Flink API 的一般性介绍， 请参阅[编程指南]({{
site.baseurl }}/zh/dev/api_concepts.html)。
Review comment:
OK, thanks very much. It's my pleasure to make contributes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9348: [FLINK-13505][docs-zh] Translate "Java Lambda Expressions" page into Chinese
wuchong commented on a change in pull request #9348: [FLINK-13505][docs-zh]
Translate "Java Lambda Expressions" page into Chinese
URL: https://github.com/apache/flink/pull/9348#discussion_r312737178
##
File path: docs/dev/java_lambdas.zh.md
##
@@ -22,32 +22,28 @@ specific language governing permissions and limitations
under the License.
-->
-Java 8 introduced several new language features designed for faster and
clearer coding. With the most important feature,
-the so-called "Lambda Expressions", it opened the door to functional
programming. Lambda expressions allow for implementing and
-passing functions in a straightforward way without having to declare
additional (anonymous) classes.
+Java 8 引入了几种新的语言特性，旨在实现更快、更清晰的编码。 作为最重要的特性，即所谓的“Lambda
表达式”，它开启了函数式编程的大门。Lambda 表达式允许以简捷的方式实现和传递函数，而无需声明额外的（匿名）类。
-Attention Flink supports the usage of
lambda expressions for all operators of the Java API, however, whenever a
lambda expression uses Java generics you need to declare type information
*explicitly*.
+注意 Flink 支持对 Java API 的所有算子使用 Lambda
表达式，但是，当 Lambda 表达式使用 Java 泛型时，你需要 *显式* 声明类型信息。
-This document shows how to use lambda expressions and describes current
limitations. For a general introduction to the
-Flink API, please refer to the [Programming Guide]({{ site.baseurl
}}/dev/api_concepts.html)
+本文档介绍了如何使用 Lambda 表达式并描述了其在当前应用中的限制。有关 Flink API 的一般性介绍， 请参阅[编程指南]({{
site.baseurl }}/zh/dev/api_concepts.html)。
Review comment:
No, you don't need. This PR is merged by me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] YngwieWang commented on a change in pull request #9348: [FLINK-13505][docs-zh] Translate "Java Lambda Expressions" page into Chinese
YngwieWang commented on a change in pull request #9348: [FLINK-13505][docs-zh]
Translate "Java Lambda Expressions" page into Chinese
URL: https://github.com/apache/flink/pull/9348#discussion_r312736946
##
File path: docs/dev/java_lambdas.zh.md
##
@@ -22,32 +22,28 @@ specific language governing permissions and limitations
under the License.
-->
-Java 8 introduced several new language features designed for faster and
clearer coding. With the most important feature,
-the so-called "Lambda Expressions", it opened the door to functional
programming. Lambda expressions allow for implementing and
-passing functions in a straightforward way without having to declare
additional (anonymous) classes.
+Java 8 引入了几种新的语言特性，旨在实现更快、更清晰的编码。 作为最重要的特性，即所谓的“Lambda
表达式”，它开启了函数式编程的大门。Lambda 表达式允许以简捷的方式实现和传递函数，而无需声明额外的（匿名）类。
-Attention Flink supports the usage of
lambda expressions for all operators of the Java API, however, whenever a
lambda expression uses Java generics you need to declare type information
*explicitly*.
+注意 Flink 支持对 Java API 的所有算子使用 Lambda
表达式，但是，当 Lambda 表达式使用 Java 泛型时，你需要 *显式* 声明类型信息。
-This document shows how to use lambda expressions and describes current
limitations. For a general introduction to the
-Flink API, please refer to the [Programming Guide]({{ site.baseurl
}}/dev/api_concepts.html)
+本文档介绍了如何使用 Lambda 表达式并描述了其在当前应用中的限制。有关 Flink API 的一般性介绍， 请参阅[编程指南]({{
site.baseurl }}/zh/dev/api_concepts.html)。
Review comment:
This PR is closed. I don't know why. should I open a new one?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9336: [FLINK-13548][Deployment/YARN]Support priority of the Flink YARN application
flinkbot edited a comment on issue #9336: [FLINK-13548][Deployment/YARN]Support priority of the Flink YARN application URL: https://github.com/apache/flink/pull/9336#issuecomment-517610510 ## CI report: * 4fe9e1ba5707fb4d208290116bc172142e6be08a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121719017) * 346ed33756127b27aed16fc91d8ce81048186c06 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/121827648) * d9b31af0157fe9b2adf080575272502b6f2e0cb5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122217463) * eb8a3c29bbb32a99d5d8e5e8baaabc390f123663 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122378181) * ef702e9abf75ba1a8d2eed162ba836febb8e7aab : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122707201) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9336: [FLINK-13548][Deployment/YARN]Support priority of the Flink YARN application
flinkbot edited a comment on issue #9336: [FLINK-13548][Deployment/YARN]Support priority of the Flink YARN application URL: https://github.com/apache/flink/pull/9336#issuecomment-517610510 ## CI report: * 4fe9e1ba5707fb4d208290116bc172142e6be08a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121719017) * 346ed33756127b27aed16fc91d8ce81048186c06 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/121827648) * d9b31af0157fe9b2adf080575272502b6f2e0cb5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122217463) * eb8a3c29bbb32a99d5d8e5e8baaabc390f123663 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122378181) * ef702e9abf75ba1a8d2eed162ba836febb8e7aab : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/122707201) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
