[
https://issues.apache.org/jira/browse/FLINK-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483393#comment-16483393
]
ASF GitHub Bot commented on FLINK-8989:
---------------------------------------
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/5761
Hi, @tzulitai Could you take another look on this PR ? It's the time for
having another review. Update the code according to your suggestions and I ran
the test on my local machine and below are the tests results. Looks good from
these results. Thank you very much.
```
Flink dist directory: /Users/zhangminglei/projects/flink/build-target
TEST_DATA_DIR:
/Users/zhangminglei/projects/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-20N
Downloading Elasticsearch1 from
https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz
> Elasticsearch node is running.
Starting cluster.
Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local.
Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local.
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
Dispatcher REST endpoint is up.
Starting execution of program
Program execution finished
Job with JobID dbd0dcbf09127ca1715da4bb3f4ca7fa has finished.
Job Runtime: 1791 ms
> Elasticsearch end to end test pass.
Stopping taskexecutor daemon (pid: 64950) on host
zhangmingleideMacBook-Pro.local.
Stopping standalonesession daemon (pid: 64658) on host
zhangmingleideMacBook-Pro.local.
No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local.
All tests PASS
Downloading Elasticsearch2 from
https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz
> Elasticsearch node is running.
Starting cluster.
Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local.
[2018-05-22 09:16:16,773][INFO ][node ] [Orb]
version[2.3.5], pid[66176], build[90f439f/2016-07-27T10:36:52Z]
[2018-05-22 09:16:16,773][INFO ][node ] [Orb]
initializing ...
Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local.
Waiting for dispatcher REST endpoint to come up...
[2018-05-22 09:16:17,502][INFO ][plugins ] [Orb] modules
[reindex, lang-expression, lang-groovy], plugins [], sites []
[2018-05-22 09:16:17,533][INFO ][env ] [Orb] using [1]
data paths, mounts [[/ (/dev/disk1)]], net usable_space [287.8gb], net
total_space [464.6gb], spins? [unknown], types [hfs]
[2018-05-22 09:16:17,533][INFO ][env ] [Orb] heap size
[989.8mb], compressed ordinary object pointers [true]
[2018-05-22 09:16:17,534][WARN ][env ] [Orb] max file
descriptors [10240] for elasticsearch process likely too low, consider
increasing to at least [65536]
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
[2018-05-22 09:16:19,858][INFO ][node ] [Orb]
initialized
[2018-05-22 09:16:19,858][INFO ][node ] [Orb] starting
...
[2018-05-22 09:16:19,963][INFO ][transport ] [Orb]
publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300},
{[::1]:9300}, {127.0.0.1:9300}
[2018-05-22 09:16:19,967][INFO ][discovery ] [Orb]
elasticsearch/VmnVQsJKSDKtkXpxZY11Tg
Waiting for dispatcher REST endpoint to come up...
Dispatcher REST endpoint is up.
Starting execution of program
[2018-05-22 09:16:23,008][INFO ][cluster.service ] [Orb]
new_master {Orb}{VmnVQsJKSDKtkXpxZY11Tg}{127.0.0.1}{127.0.0.1:9300}, reason:
zen-disco-join(elected_as_master, [0] joins received)
[2018-05-22 09:16:23,023][INFO ][http ] [Orb]
publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200},
{[::1]:9200}, {127.0.0.1:9200}
[2018-05-22 09:16:23,023][INFO ][node ] [Orb] started
[2018-05-22 09:16:23,047][INFO ][gateway ] [Orb] recovered
[0] indices into cluster_state
[2018-05-22 09:16:24,579][INFO ][cluster.metadata ] [Orb] [index]
creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings
[type]
[2018-05-22 09:16:24,921][INFO ][cluster.routing.allocation] [Orb] Cluster
health status changed from [RED] to [YELLOW] (reason: [shards started
[[index][4]] ...]).
[2018-05-22 09:16:24,940][INFO ][cluster.metadata ] [Orb] [index]
update_mapping [type]
Program execution finished
Job with JobID ac42b71aeb6138100d9357b72bcf94f5 has finished.
Job Runtime: 1726 ms
> Elasticsearch end to end test pass.
[2018-05-22 09:16:26,552][INFO ][node ] [Orb] stopping
...
[2018-05-22 09:16:26,637][INFO ][node ] [Orb] stopped
[2018-05-22 09:16:26,637][INFO ][node ] [Orb] closing
...
[2018-05-22 09:16:26,643][INFO ][node ] [Orb] closed
Stopping taskexecutor daemon (pid: 66886) on host
zhangmingleideMacBook-Pro.local.
Stopping standalonesession daemon (pid: 66594) on host
zhangmingleideMacBook-Pro.local.
No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local.
All tests PASS
Downloading Elasticsearch5 from
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz
> Elasticsearch node is running.
Starting cluster.
Starting standalonesession daemon on host zhangmingleideMacBook-Pro.local.
Starting taskexecutor daemon on host zhangmingleideMacBook-Pro.local.
Waiting for dispatcher REST endpoint to come up...
[2018-05-22T09:19:01,590][INFO ][o.e.n.Node ] [] initializing
...
[2018-05-22T09:19:01,702][INFO ][o.e.e.NodeEnvironment ] [1m3UnQf] using
[1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [287.8gb], net
total_space [464.6gb], spins? [unknown], types [hfs]
[2018-05-22T09:19:01,702][INFO ][o.e.e.NodeEnvironment ] [1m3UnQf] heap
size [1.9gb], compressed ordinary object pointers [true]
[2018-05-22T09:19:01,704][INFO ][o.e.n.Node ] node name
[1m3UnQf] derived from node ID [1m3UnQfqRESzEuHGmXtWXw]; set [node.name] to
override
[2018-05-22T09:19:01,708][INFO ][o.e.n.Node ] version[5.1.2],
pid[68158], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Mac OS
X/10.12.3/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server
VM/1.8.0_161/25.161-b12]
Waiting for dispatcher REST endpoint to come up...
[2018-05-22T09:19:03,051][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [aggs-matrix-stats]
[2018-05-22T09:19:03,051][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [ingest-common]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [lang-expression]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [lang-groovy]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [lang-mustache]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [lang-painless]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [percolator]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [reindex]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [transport-netty3]
[2018-05-22T09:19:03,052][INFO ][o.e.p.PluginsService ] [1m3UnQf]
loaded module [transport-netty4]
[2018-05-22T09:19:03,053][INFO ][o.e.p.PluginsService ] [1m3UnQf] no
plugins loaded
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
[2018-05-22T09:19:05,763][INFO ][o.e.n.Node ] initialized
[2018-05-22T09:19:05,763][INFO ][o.e.n.Node ] [1m3UnQf]
starting ...
Dispatcher REST endpoint is up.
[2018-05-22T09:19:05,973][INFO ][o.e.t.TransportService ] [1m3UnQf]
publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300},
{[::1]:9300}, {127.0.0.1:9300}
Starting execution of program
[2018-05-22T09:19:09,041][INFO ][o.e.c.s.ClusterService ] [1m3UnQf]
new_master
{1m3UnQf}{1m3UnQfqRESzEuHGmXtWXw}{Hk79-3qVQCGN5rH2PSvsQA}{127.0.0.1}{127.0.0.1:9300},
reason: zen-disco-elected-as-master ([0] nodes joined)
[2018-05-22T09:19:09,064][INFO ][o.e.h.HttpServer ] [1m3UnQf]
publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200},
{[::1]:9200}, {127.0.0.1:9200}
[2018-05-22T09:19:09,064][INFO ][o.e.n.Node ] [1m3UnQf]
started
[2018-05-22T09:19:09,073][INFO ][o.e.g.GatewayService ] [1m3UnQf]
recovered [0] indices into cluster_state
[2018-05-22T09:19:09,134][INFO ][o.e.c.m.MetaDataCreateIndexService]
[1m3UnQf] [index] creating index, cause [auto(bulk api)], templates [], shards
[5]/[1], mappings []
[2018-05-22T09:19:09,401][INFO ][o.e.c.m.MetaDataMappingService] [1m3UnQf]
[index/cPsn88XrR0yimNn648GNlw] create_mapping [type]
Program execution finished
Job with JobID d5506334c212280afaf51f6f04ef5613 has finished.
Job Runtime: 1971 ms
> Elasticsearch end to end test pass.
[2018-05-22T09:19:10,749][INFO ][o.e.n.Node ] [1m3UnQf]
stopping ...
[2018-05-22T09:19:10,818][INFO ][o.e.n.Node ] [1m3UnQf]
stopped
[2018-05-22T09:19:10,818][INFO ][o.e.n.Node ] [1m3UnQf]
closing ...
[2018-05-22T09:19:10,825][INFO ][o.e.n.Node ] [1m3UnQf] closed
Stopping taskexecutor daemon (pid: 68873) on host
zhangmingleideMacBook-Pro.local.
Stopping standalonesession daemon (pid: 68581) on host
zhangmingleideMacBook-Pro.local.
No zookeeper daemon to stop on host zhangmingleideMacBook-Pro.local.
All tests PASS
```
> End-to-end test: ElasticSearch connector
> ----------------------------------------
>
> Key: FLINK-8989
> URL: https://issues.apache.org/jira/browse/FLINK-8989
> Project: Flink
> Issue Type: Sub-task
> Components: ElasticSearch Connector, Tests
> Reporter: Till Rohrmann
> Assignee: mingleizhang
> Priority: Major
>
> Similar to FLINK-8988 we should add a end-to-end test which tests the
> {{ElasticSearch}} connector. We should run against all three supported
> ElasticSearch versions: 1.x, 2.x and 5.x.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
