[
https://issues.apache.org/jira/browse/BEAM-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rik Nauta updated BEAM-2541:
----------------------------
Description:
We have a ES (2.4) database running in production that is not accessible from
dev-machines because of VPS firewall rules. However, we do create the Beam
graph locally on our machines and submit it to Google Dataflow for execution.
However because of this check (line 213)
{code:java}
private static void checkVersion(ConnectionConfiguration
connectionConfiguration)
throws IOException {
RestClient restClient = connectionConfiguration.createClient();
Response response = restClient.performRequest("GET", "", new
BasicHeader("", ""));
JsonNode jsonNode = parseResponse(response);
String version = jsonNode.path("version").path("number").asText();
boolean version2x = version.startsWith("2.");
restClient.close();
checkArgument(
version2x,
"ConnectionConfiguration.create(addresses, index, type): "
+ "the Elasticsearch version to connect to is different of 2.x. "
+ "This version of the ElasticsearchIO is only compatible with
Elasticsearch v2.x");
}
{code}
the creation of the graph fails because it can't complete the restClient
request from our dev machines. This check should probably not happen this early
since there is no guarantee that the database is reachable when building the
graph
{code:java}
exception in thread "main" java.net.ConnectException
at
org.apache.http.nio.pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168)
at
org.apache.http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIOConnPool.java:561)
at
org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.timeout(AbstractNIOConnPool.java:822)
at
org.apache.http.impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(DefaultConnectingIOReactor.java:210)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
at
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
at
org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
at java.lang.Thread.run(Thread.java:745)
{code}
was:
We have a ES (2.4) database running in production that is not accessible from
dev-machines because of VPS firewall rules. However, we do create the Beam
graph locally on our machines and submit it to Google Dataflow for execution.
However because of this check (line 213)
{code:java}
private static void checkVersion(ConnectionConfiguration
connectionConfiguration)
throws IOException {
RestClient restClient = connectionConfiguration.createClient();
Response response = restClient.performRequest("GET", "", new
BasicHeader("", ""));
JsonNode jsonNode = parseResponse(response);
String version = jsonNode.path("version").path("number").asText();
boolean version2x = version.startsWith("2.");
restClient.close();
checkArgument(
version2x,
"ConnectionConfiguration.create(addresses, index, type): "
+ "the Elasticsearch version to connect to is different of 2.x. "
+ "This version of the ElasticsearchIO is only compatible with
Elasticsearch v2.x");
}
{code}
the creation of the graph fails because it can't complete the restClient
request from our dev machines. This check should probably not happen this early
since there is no guarantee that the database is reachable when building the
graph
> ElasticsearchIO breaks for non locally accessible DB
> ----------------------------------------------------
>
> Key: BEAM-2541
> URL: https://issues.apache.org/jira/browse/BEAM-2541
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Rik Nauta
> Assignee: Davor Bonaci
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> We have a ES (2.4) database running in production that is not accessible from
> dev-machines because of VPS firewall rules. However, we do create the Beam
> graph locally on our machines and submit it to Google Dataflow for execution.
> However because of this check (line 213)
> {code:java}
> private static void checkVersion(ConnectionConfiguration
> connectionConfiguration)
> throws IOException {
> RestClient restClient = connectionConfiguration.createClient();
> Response response = restClient.performRequest("GET", "", new
> BasicHeader("", ""));
> JsonNode jsonNode = parseResponse(response);
> String version = jsonNode.path("version").path("number").asText();
> boolean version2x = version.startsWith("2.");
> restClient.close();
> checkArgument(
> version2x,
> "ConnectionConfiguration.create(addresses, index, type): "
> + "the Elasticsearch version to connect to is different of 2.x.
> "
> + "This version of the ElasticsearchIO is only compatible with
> Elasticsearch v2.x");
> }
> {code}
> the creation of the graph fails because it can't complete the restClient
> request from our dev machines. This check should probably not happen this
> early since there is no guarantee that the database is reachable when
> building the graph
> {code:java}
> exception in thread "main" java.net.ConnectException
> at
> org.apache.http.nio.pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168)
> at
> org.apache.http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIOConnPool.java:561)
> at
> org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.timeout(AbstractNIOConnPool.java:822)
> at
> org.apache.http.impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183)
> at
> org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(DefaultConnectingIOReactor.java:210)
> at
> org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
> at
> org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
> at
> org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
