Re: Reading as Parquet a directory created by Spark Structured Streaming - problems

[ddebarbieux]

cala> spark.read.schema(StructType(Seq(StructField("_1",StringType,false), StructField("_2",StringType,true)))).parque ("hdfs://---/MY_DIRECTORY/*_1=201812030900*").show()

+----+--------------------+
|  _1| _2|
+----+--------------------+
|null|ba1ca2dc033440125...|
|null|ba1ca2dc033440125...|
|null|ba1ca2dc033440125...|

Is the column '_1' a String or a Number?

Denis