[ https://issues.apache.org/jira/browse/SPARK-10805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-10805: ------------------------------ Target Version/s: (was: 1.4.1) Priority: Minor (was: Critical) > JSON Data Frame does not return correct string lengths > ------------------------------------------------------ > > Key: SPARK-10805 > URL: https://issues.apache.org/jira/browse/SPARK-10805 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 1.4.1 > Reporter: Jeff Li > Priority: Minor > > Here is the sample code to run the test > @Test > public void runSchemaTest() throws Exception { > DataFrame jsonDataFrame = > sqlContext.jsonFile("src/test/resources/jsontransform/json.sampledata.json"); > jsonDataFrame.printSchema(); > StructType jsonSchema = jsonDataFrame.schema(); > StructField[] dataFields = jsonSchema.fields(); > for ( int fieldIndex = 0; fieldIndex < dataFields.length; > fieldIndex++) { > StructField aField = dataFields[fieldIndex]; > DataType aType = aField.dataType(); > System.out.println("name: " + aField.name() + " type: " > + aType.typeName() > + " size: " +aType.defaultSize()); > } > } > name: _id type: string size: 4096 > name: firstName type: string size: 4096 > name: lastName type: string size: 4096 > In my case, the _id: 1 character, first name: 4 characters, and last name: 7 > characters). > The Spark JSON Data frame should have a way to tell the maximum length of > each JSON String elements in the JSON document. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org