[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686569#comment-16686569 ] Etienne Chauchot commented on BEAM-5925: [~wouts]assigning to you as you started to work on that. Tell me is you need help. I'll take a look at the other test flakes > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Wout Scheepers >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680186#comment-16680186 ] Chamikara Jayalath commented on BEAM-5925: -- Another timeout here but from a different location: https://scans.gradle.com/s/33iongbzxsfy4/tests/fo2bghfaj5ysq-nxssse2hlcccy?openStackTraces=WzEsMl0 > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678166#comment-16678166 ] Wout Scheepers commented on BEAM-5925: -- I'm certain the retryConfiguration does not take care of RestClient listener timeouts, as they still show up when trying to stream 40M documents with a retryConfig set. [~timrobertson100] suggested me to try again with an increased timeout of the RestClient: ``` RestClient createClient() throws IOException { ... ... return restClientBuilder .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() { @Override public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) { return requestConfigBuilder.setConnectTimeout(60) .setSocketTimeout(60); } }) .setMaxRetryTimeoutMillis(60) .build(); } ``` This worked for me. My elastic dev cluster consists of one node, and it took my dataflow job 1h45 min to stream 40M docs with 5 workers. Hope this helps > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677889#comment-16677889 ] Tim Robertson commented on BEAM-5925: - CC [~wouts] too, a new ES contributor who is also looking at timeouts in his environment > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] >
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677881#comment-16677881 ] Dat Tran commented on BEAM-5925: Let me check [~echauchot] > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [darwin/g_sIfS5aQwi6BAXw_--vgw] create_mapping [TYPE_1] > [2018-10-31T04:06:39,201][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677858#comment-16677858 ] Etienne Chauchot commented on BEAM-5925: I have very little availability to work on ES6 [~dattran.vn01] can you take a look at this flaky test? I cannot assign you the ticket. > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670115#comment-16670115 ] Etienne Chauchot commented on BEAM-5925: Thanks Kenn for pointing out. It has been a lot of flakiness lately with ESIO v6. I'll take a look at them > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] >
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669429#comment-16669429 ] Kenneth Knowles commented on BEAM-5925: --- Looked into it, and I couldn't quite figure out what is going on, but I don't think adding a timeout (which would be to the ES RetryConfiguration I think) is the issue. Do {{:beam-sdks-java-io-elasticsearch-tests-2:test}}, {{:beam-sdks-java-io-elasticsearch-tests-5:test}}, and {{:beam-sdks-java-io-elasticsearch-tests-6:test}} have some accidental conflict in their clusters or teardown? I see they use different indices... > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669346#comment-16669346 ] Kenneth Knowles commented on BEAM-5925: --- I'm going to take a minute to see if I can just increase the timeout. > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [darwin/g_sIfS5aQwi6BAXw_--vgw]
[jira] [Commented] (BEAM-5925) Test flake in ElasticsearchIOTest.testWriteFullAddressing
[ https://issues.apache.org/jira/browse/BEAM-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669345#comment-16669345 ] Kenneth Knowles commented on BEAM-5925: --- To be clear, this is v6: {{:beam-sdks-java-io-elasticsearch-tests-6:test}} > Test flake in ElasticsearchIOTest.testWriteFullAddressing > - > > Key: BEAM-5925 > URL: https://issues.apache.org/jira/browse/BEAM-5925 > Project: Beam > Issue Type: Bug > Components: io-java-elasticsearch >Reporter: Kenneth Knowles >Assignee: Etienne Chauchot >Priority: Critical > > https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1789/ > https://scans.gradle.com/s/j42mwdsn5svcs > {code} > org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: > listener timeout after waiting for [3] ms > {code} > Log looks like this: > {code} > [2018-10-31T04:06:07,571][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [testWriteFullAddressing]: before test > [2018-10-31T04:06:07,572][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: setting up test > [2018-10-31T04:06:07,589][INFO ][o.e.c.m.MetaDataIndexTemplateService] > [node_s0] adding template [random_index_template] for index patterns [*] > [2018-10-31T04:06:07,645][INFO ][o.a.b.s.i.e.ElasticsearchIOTest] > [ElasticsearchIOTest#testWriteFullAddressing]: all set up test > [2018-10-31T04:06:10,536][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [galilei] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:33,963][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [curie] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,034][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [darwin] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,050][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [copernicus] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,075][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [faraday] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,095][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [bohr] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,113][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [pasteur] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,142][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [einstein] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,205][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [maxwell] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:34,226][INFO ][o.e.c.m.MetaDataCreateIndexService] > [node_s0] [newton] creating index, cause [auto(bulk api)], templates > [random_index_template], shards [6]/[0], mappings [] > [2018-10-31T04:06:36,914][INFO ][o.e.c.r.a.AllocationService] [node_s0] > Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards > started [[galilei][4], [galilei][5]] ...]). > [2018-10-31T04:06:36,970][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [galilei/Vn1b8XXVSAmrTb5BVe2IJQ] create_mapping [TYPE_1] > [2018-10-31T04:06:37,137][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [newton/bjnImLt_QguBGEFH9lBJ6Q] create_mapping [TYPE_-1] > [2018-10-31T04:06:37,385][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [maxwell/-RZ32NbRRZWaGaVfaptFIA] create_mapping [TYPE_0] > [2018-10-31T04:06:37,636][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [einstein/2lgF5Vj6Ti2KTS-pYSzv3Q] create_mapping [TYPE_1] > [2018-10-31T04:06:37,806][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [pasteur/832OwzleRSOHsWx85vOH-w] create_mapping [TYPE_0] > [2018-10-31T04:06:38,103][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [bohr/9YTwB1yvTYKf9YjYCmHjwg] create_mapping [TYPE_1] > [2018-10-31T04:06:38,229][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [faraday/vIMYG8vpTQKqNkyajcFOxw] create_mapping [TYPE_0] > [2018-10-31T04:06:38,576][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [copernicus/NzCZssInSiOdZKTmLCoXRw] create_mapping [TYPE_1] > [2018-10-31T04:06:38,890][INFO ][o.e.c.m.MetaDataMappingService] [node_s0] > [darwin/g_sIfS5aQwi6BAXw_--vgw]