[
https://issues.apache.org/jira/browse/LIVY-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
poonam chaudhari updated LIVY-740:
----------------------------------
Description:
curl -X POST --data '{"file": "s3://livy-test/spark-examples_2.11-2.4.4.jar",
"className":
"org.apache.spark.examples.SparkPi","conf":{"spark.hadoop.fs.s3.awsAccessKeyId":"***","spark.hadoop.fs.s3.awsSecretAccessKey":"***","spark.executor.extraJavaOptions":"-Dcom.amazonaws.services.s3.enableV4=true","spark.driver.extraJavaOptions":"Dcom.amazonaws.services.s3.enableV4=true"}}'
-H "Content-Type: application/json" -H "X-Requested-By: admin"
[http://hostname:8999/batches|http://172.16.7.203:8999/batches]
I am trying to submit spark job to like this but it fails with below error:
!image-2020-01-10-09-56-27-060.png!
AM I missing something here? I have verified aws secret many times.The same
secret is working with aws cli. Can you please guide me here?
was:
curl -X POST --data '\{"file": "s3://livy-test/spark-examples_2.11-2.4.4.jar",
"className":
"org.apache.spark.examples.SparkPi","conf":{"spark.hadoop.fs.s3.awsAccessKeyId":"***","spark.hadoop.fs.s3.awsSecretAccessKey":"***","spark.executor.extraJavaOptions":"-Dcom.amazonaws.services.s3.enableV4=true","spark.driver.extraJavaOptions":"Dcom.amazonaws.services.s3.enableV4=true"}}'
-H "Content-Type: application/json" -H "X-Requested-By: admin"
[http://172.16.7.203:8999/batches]
I am trying to submit spark job to like this but it fails with below error:
!image-2020-01-10-09-56-27-060.png!
AM I missing something here? I have verified aws secret many times.The same
secret is working with aws cli. Can you please guide me here?
> Trying to access jar on s3 using livy
> -------------------------------------
>
> Key: LIVY-740
> URL: https://issues.apache.org/jira/browse/LIVY-740
> Project: Livy
> Issue Type: Question
> Components: API
> Environment: Spark - 2.3.0.2.6.5.1175-1
> hdp - 2.6.5.1175-1
> AWS jar versions in spark:
> /aws-java-sdk-core-1.10.6.jar
> /aws-java-sdk-kms-1.10.6.jar
> /aws-java-sdk-s3-1.10.6.jar
> /hadoop-aws-2.7.3.2.6.5.1175-1.jar
> Reporter: poonam chaudhari
> Priority: Major
> Labels: beginner
> Attachments: image-2020-01-10-09-56-27-060.png
>
>
> curl -X POST --data '{"file": "s3://livy-test/spark-examples_2.11-2.4.4.jar",
> "className":
> "org.apache.spark.examples.SparkPi","conf":{"spark.hadoop.fs.s3.awsAccessKeyId":"***","spark.hadoop.fs.s3.awsSecretAccessKey":"***","spark.executor.extraJavaOptions":"-Dcom.amazonaws.services.s3.enableV4=true","spark.driver.extraJavaOptions":"Dcom.amazonaws.services.s3.enableV4=true"}}'
> -H "Content-Type: application/json" -H "X-Requested-By: admin"
> [http://hostname:8999/batches|http://172.16.7.203:8999/batches]
>
> I am trying to submit spark job to like this but it fails with below error:
> !image-2020-01-10-09-56-27-060.png!
> AM I missing something here? I have verified aws secret many times.The same
> secret is working with aws cli. Can you please guide me here?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
