Github user zhangminglei commented on the issue: https://github.com/apache/flink/pull/5761 Hi, @tzulitai Could you take another look on this PR ? It's the time for having another review. Update the code according to your suggestions and I ran the test on my local machine and below are the tests results. Looks good from these results. Thank you very much. ``` Flink dist directory: /Users/zhangminglei/projects/flink/build-target TEST_DATA_DIR: /Users/zhangminglei/projects/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-20N Downloading Elasticsearch1 from https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz > Elasticsearch node is running. Starting cluster. Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local. Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local. Waiting for dispatcher REST endpoint to come up... Waiting for dispatcher REST endpoint to come up... Waiting for dispatcher REST endpoint to come up... Waiting for dispatcher REST endpoint to come up... Dispatcher REST endpoint is up. Starting execution of program Program execution finished Job with JobID dbd0dcbf09127ca1715da4bb3f4ca7fa has finished. Job Runtime: 1791 ms > Elasticsearch end to end test pass. Stopping taskexecutor daemon (pid: 64950) on host zhangmingleideMacBook-Pro.local. Stopping standalonesession daemon (pid: 64658) on host zhangmingleideMacBook-Pro.local. No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local. All tests PASS Downloading Elasticsearch2 from https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz > Elasticsearch node is running. Starting cluster. Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local. [2018-05-22 09:16:16,773][INFO ][node ] [Orb] version[2.3.5], pid[66176], build[90f439f/2016-07-27T10:36:52Z] [2018-05-22 09:16:16,773][INFO ][node ] [Orb] initializing ... Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local. Waiting for dispatcher REST endpoint to come up... [2018-05-22 09:16:17,502][INFO ][plugins ] [Orb] modules [reindex, lang-expression, lang-groovy], plugins [], sites [] [2018-05-22 09:16:17,533][INFO ][env ] [Orb] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [287.8gb], net total_space [464.6gb], spins? [unknown], types [hfs] [2018-05-22 09:16:17,533][INFO ][env ] [Orb] heap size [989.8mb], compressed ordinary object pointers [true] [2018-05-22 09:16:17,534][WARN ][env ] [Orb] max file descriptors [10240] for elasticsearch process likely too low, consider increasing to at least [65536] Waiting for dispatcher REST endpoint to come up... Waiting for dispatcher REST endpoint to come up... [2018-05-22 09:16:19,858][INFO ][node ] [Orb] initialized [2018-05-22 09:16:19,858][INFO ][node ] [Orb] starting ... [2018-05-22 09:16:19,963][INFO ][transport ] [Orb] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300} [2018-05-22 09:16:19,967][INFO ][discovery ] [Orb] elasticsearch/VmnVQsJKSDKtkXpxZY11Tg Waiting for dispatcher REST endpoint to come up... Dispatcher REST endpoint is up. Starting execution of program [2018-05-22 09:16:23,008][INFO ][cluster.service ] [Orb] new_master {Orb}{VmnVQsJKSDKtkXpxZY11Tg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received) [2018-05-22 09:16:23,023][INFO ][http ] [Orb] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200} [2018-05-22 09:16:23,023][INFO ][node ] [Orb] started [2018-05-22 09:16:23,047][INFO ][gateway ] [Orb] recovered [0] indices into cluster_state [2018-05-22 09:16:24,579][INFO ][cluster.metadata ] [Orb] [index] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [type] [2018-05-22 09:16:24,921][INFO ][cluster.routing.allocation] [Orb] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[index][4]] ...]). [2018-05-22 09:16:24,940][INFO ][cluster.metadata ] [Orb] [index] update_mapping [type] Program execution finished Job with JobID ac42b71aeb6138100d9357b72bcf94f5 has finished. Job Runtime: 1726 ms > Elasticsearch end to end test pass. [2018-05-22 09:16:26,552][INFO ][node ] [Orb] stopping ... [2018-05-22 09:16:26,637][INFO ][node ] [Orb] stopped [2018-05-22 09:16:26,637][INFO ][node ] [Orb] closing ... [2018-05-22 09:16:26,643][INFO ][node ] [Orb] closed Stopping taskexecutor daemon (pid: 66886) on host zhangmingleideMacBook-Pro.local. Stopping standalonesession daemon (pid: 66594) on host zhangmingleideMacBook-Pro.local. No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local. All tests PASS Downloading Elasticsearch5 from https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz > Elasticsearch node is running. Starting cluster. Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local. Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local. Waiting for dispatcher REST endpoint to come up... [2018-05-22T09:19:01,590][INFO ][o.e.n.Node ] [] initializing ... [2018-05-22T09:19:01,702][INFO ][o.e.e.NodeEnvironment ] [1m3UnQf] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [287.8gb], net total_space [464.6gb], spins? [unknown], types [hfs] [2018-05-22T09:19:01,702][INFO ][o.e.e.NodeEnvironment ] [1m3UnQf] heap size [1.9gb], compressed ordinary object pointers [true] [2018-05-22T09:19:01,704][INFO ][o.e.n.Node ] node name [1m3UnQf] derived from node ID [1m3UnQfqRESzEuHGmXtWXw]; set [node.name] to override [2018-05-22T09:19:01,708][INFO ][o.e.n.Node ] version[5.1.2], pid[68158], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Mac OS X/10.12.3/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12] Waiting for dispatcher REST endpoint to come up... [2018-05-22T09:19:03,051][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [aggs-matrix-stats] [2018-05-22T09:19:03,051][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [ingest-common] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [lang-expression] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [lang-groovy] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [lang-mustache] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [lang-painless] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [percolator] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [reindex] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [transport-netty3] [2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf] loaded module [transport-netty4] [2018-05-22T09:19:03,053][INFO ][o.e.p.PluginsService ] [1m3UnQf] no plugins loaded Waiting for dispatcher REST endpoint to come up... Waiting for dispatcher REST endpoint to come up... [2018-05-22T09:19:05,763][INFO ][o.e.n.Node ] initialized [2018-05-22T09:19:05,763][INFO ][o.e.n.Node ] [1m3UnQf] starting ... Dispatcher REST endpoint is up. [2018-05-22T09:19:05,973][INFO ][o.e.t.TransportService ] [1m3UnQf] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300} Starting execution of program [2018-05-22T09:19:09,041][INFO ][o.e.c.s.ClusterService ] [1m3UnQf] new_master {1m3UnQf}{1m3UnQfqRESzEuHGmXtWXw}{Hk79-3qVQCGN5rH2PSvsQA}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined) [2018-05-22T09:19:09,064][INFO ][o.e.h.HttpServer ] [1m3UnQf] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200} [2018-05-22T09:19:09,064][INFO ][o.e.n.Node ] [1m3UnQf] started [2018-05-22T09:19:09,073][INFO ][o.e.g.GatewayService ] [1m3UnQf] recovered [0] indices into cluster_state [2018-05-22T09:19:09,134][INFO ][o.e.c.m.MetaDataCreateIndexService] [1m3UnQf] [index] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [] [2018-05-22T09:19:09,401][INFO ][o.e.c.m.MetaDataMappingService] [1m3UnQf] [index/cPsn88XrR0yimNn648GNlw] create_mapping [type] Program execution finished Job with JobID d5506334c212280afaf51f6f04ef5613 has finished. Job Runtime: 1971 ms > Elasticsearch end to end test pass. [2018-05-22T09:19:10,749][INFO ][o.e.n.Node ] [1m3UnQf] stopping ... [2018-05-22T09:19:10,818][INFO ][o.e.n.Node ] [1m3UnQf] stopped [2018-05-22T09:19:10,818][INFO ][o.e.n.Node ] [1m3UnQf] closing ... [2018-05-22T09:19:10,825][INFO ][o.e.n.Node ] [1m3UnQf] closed Stopping taskexecutor daemon (pid: 68873) on host zhangmingleideMacBook-Pro.local. Stopping standalonesession daemon (pid: 68581) on host zhangmingleideMacBook-Pro.local. No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local. All tests PASS ```
---