[
https://issues.apache.org/jira/browse/FLINK-8249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joao Boto updated FLINK-8249:
-----------------------------
Description:
Hi,
setting this configurations to FlinkKinesisProducer:
{code}
properties.put(AWSConfigConstants.AWS_REGION, "eu-west-1");
properties.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "accessKey");
properties.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "secretKey");
{code}
is throwing this error:
{code}
17/12/13 10:50:11 ERROR LogInputStreamReader: [2017-12-13 10:50:11.290786]
[0x000057ba][0x00007f31cbce5780] [error] [main.cc:266] Could not configure the
region. It was not given in the config and we were unable to retrieve it from
EC2 metadata.
17/12/13 10:50:12 ERROR KinesisProducer: Error in child process
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.IrrecoverableError:
Child process exited with code 1
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:475)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700441]
[0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](CurlHttpClient)Curl
returned error code 28
17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700521]
[0x000057c4][0x00007ffb152b5780] [error] [AWS Log:
ERROR](EC2MetadataClient)Http request to Ec2MetadataService failed.
{code}
making some investigations the region is never setted and i think this is the
reason:
in this commit:
https://github.com/apache/flink/commit/9ed5d9a180dcd871e33bf8982434e3afd90ed295#diff-f3c6c35f3b045df8408b310f8f8a6bc7
{code}
- KinesisProducerConfiguration producerConfig = new
KinesisProducerConfiguration();
-
producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
+ // check and pass the configuration properties
+ KinesisProducerConfiguration producerConfig =
KinesisConfigUtil.validateProducerConfiguration(configProps);
producerConfig.setCredentialsProvider(AWSUtil.getCredentialsProvider(configProps));
{code}
this line was removed
{code}
producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
{code}
cc [~tzulitai], [~phoenixjiangnan]
was:
Hi,
setting this configurations to FlinkKinesisProducer:
{code}
properties.put(AWSConfigConstants.AWS_REGION, "eu-west-1");
properties.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "accessKey");
properties.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "secretKey");
{code}
is throwing this error:
{code}
17/12/13 10:50:11 ERROR LogInputStreamReader: [2017-12-13 10:50:11.290786]
[0x000057ba][0x00007f31cbce5780] [error] [main.cc:266] Could not configure the
region. It was not given in the config and we were unable to retrieve it from
EC2 metadata.
17/12/13 10:50:12 ERROR KinesisProducer: Error in child process
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.IrrecoverableError:
Child process exited with code 1
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:475)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
at
org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700441]
[0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](CurlHttpClient)Curl
returned error code 28
17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700521]
[0x000057c4][0x00007ffb152b5780] [error] [AWS Log:
ERROR](EC2MetadataClient)Http request to Ec2MetadataService failed.
{code}
making some investigations the region is never setted and i think this is the
reason:
in this commit:
https://github.com/apache/flink/commit/9ed5d9a180dcd871e33bf8982434e3afd90ed295#diff-f3c6c35f3b045df8408b310f8f8a6bc7
{code}
- KinesisProducerConfiguration producerConfig = new
KinesisProducerConfiguration();
-
producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
+ // check and pass the configuration properties
+ KinesisProducerConfiguration producerConfig =
KinesisConfigUtil.validateProducerConfiguration(configProps);
producerConfig.setCredentialsProvider(AWSUtil.getCredentialsProvider(configProps));
{code}
this line was removed
{code}
producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
{code}
cc [~tzulitai]
> Kinesis Producer didnt configure region
> ---------------------------------------
>
> Key: FLINK-8249
> URL: https://issues.apache.org/jira/browse/FLINK-8249
> Project: Flink
> Issue Type: Bug
> Components: Kinesis Connector
> Affects Versions: 1.4.0
> Reporter: Joao Boto
>
> Hi,
> setting this configurations to FlinkKinesisProducer:
> {code}
> properties.put(AWSConfigConstants.AWS_REGION, "eu-west-1");
> properties.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "accessKey");
> properties.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "secretKey");
> {code}
> is throwing this error:
> {code}
> 17/12/13 10:50:11 ERROR LogInputStreamReader: [2017-12-13 10:50:11.290786]
> [0x000057ba][0x00007f31cbce5780] [error] [main.cc:266] Could not configure
> the region. It was not given in the config and we were unable to retrieve it
> from EC2 metadata.
> 17/12/13 10:50:12 ERROR KinesisProducer: Error in child process
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.IrrecoverableError:
> Child process exited with code 1
> at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525)
> at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
> at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:475)
> at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
> at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700441]
> [0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](CurlHttpClient)Curl
> returned error code 28
> 17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700521]
> [0x000057c4][0x00007ffb152b5780] [error] [AWS Log:
> ERROR](EC2MetadataClient)Http request to Ec2MetadataService failed.
> {code}
> making some investigations the region is never setted and i think this is the
> reason:
> in this commit:
> https://github.com/apache/flink/commit/9ed5d9a180dcd871e33bf8982434e3afd90ed295#diff-f3c6c35f3b045df8408b310f8f8a6bc7
> {code}
> - KinesisProducerConfiguration producerConfig = new
> KinesisProducerConfiguration();
> -
> producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
> + // check and pass the configuration properties
> + KinesisProducerConfiguration producerConfig =
> KinesisConfigUtil.validateProducerConfiguration(configProps);
>
> producerConfig.setCredentialsProvider(AWSUtil.getCredentialsProvider(configProps));
> {code}
> this line was removed
> {code}
> producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
> {code}
> cc [~tzulitai], [~phoenixjiangnan]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
