[
https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510875#comment-16510875
]
SHAILENDRA SHAHANE commented on SPARK-24514:
--------------------------------------------
Attached Source code and Exception trace
files[^MongoSparkTest.java][^SparkMongoException.txt]
</dependency>
<dependency> <!-- Spark SQL dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.3.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.3.0</version>
</dependency>
<dependency>
<groupId>org.mongodb.spark</groupId>
<artifactId>mongo-spark-connector_2.11</artifactId>
<version>2.2.2</version>
</dependency>
----------------------------------
Would like to get this resolved.
> Exception while converting RDD to DataFrame
> -------------------------------------------
>
> Key: SPARK-24514
> URL: https://issues.apache.org/jira/browse/SPARK-24514
> Project: Spark
> Issue Type: Bug
> Components: Java API, Spark Core, SQL
> Affects Versions: 2.3.0
> Environment: Spark version 2.3
> JDK 1.8
> MongoDB Version - 3.4 and 3.6
> Sample Data -
> {"_id":"5b0d31f892549e10b61d962a","RSEG_MANDT":"800","RSEG_EBELN":"4500017749","RSEG_EBELP":"00020","RSEG_BELNR":"1000000001","RSEG_BUZEI":"000002","RSEG_GJAHR":"2013","RBKP_BUDAT":"2013-10-04","RSEG_MENGE":{"$numberDecimal":"30.000"},"RSEG_LFBNR":"5000000472","RSEG_LFGJA":"2013","RSEG_LFPOS":"0002","NOT_ACCOUNT_MAINTENANCE":{"$numberDecimal":"1.0000000000"},"RBKP_CPUTIMESTAMP":"2013-10-04T10:32:02.000Z","RBKP_WAERS":"USD","RSEG_BNKAN":{"$numberDecimal":"0.00"},"RSEG_WRBTR":{"$numberDecimal":"2340.00"},"RSEG_SHKZG":"S"}
> Reporter: SHAILENDRA SHAHANE
> Priority: Major
> Attachments: MongoSparkTest.java, SparkJiraData.txt,
> SparkMongoException.txt
>
>
> I tried to fetch data from MongoDB and got the following exception while
> converting the RDD to DF.
> -----------------Code --------------
> SparkSession spark = sparkSessionBuilder.appName("VendorProcessor")
> .config("spark.mongodb.input.uri","myMongoUrl")
> .config("spark.mongodb.output.uri","myMongoUrl")
> .getOrCreate();
> JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
> JavaMongoRDD<Document> rdd = MongoSpark.load(jsc);
> Dataset<Row> rbkp = rdd.toDF();
> ------------------ OR ------------------------
> DataFrameReader dfr = spark.read()
> .format("com.mongodb.spark.sql")
> .option("floatAsBigDecimal", "true");
> Dataset<Row> rbkp = dfr.load();
> --------------------
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
