Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r224044435 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,57 @@ * limitations under the License. */ -package org.apache.spark.sql.execution.datasources +package org.apache.spark.sql.catalyst.util import org.apache.spark.SparkException import org.apache.spark.sql.catalyst.InternalRow import org.apache.spark.sql.catalyst.expressions.GenericInternalRow -import org.apache.spark.sql.catalyst.util._ -import org.apache.spark.sql.internal.SQLConf -import org.apache.spark.sql.types.StructType +import org.apache.spark.sql.types.{DataType, StructType} import org.apache.spark.unsafe.types.UTF8String class FailureSafeParser[IN]( --- End diff -- Frankly speaking I don't fully understand the idea. Let's look at an example. We should parser JSON arrays (one array per row) like: ``` [1, 2, 3] [4, 5] ``` and an user provided the schema `ArrayType(IntegerType, true)`. So, you propose to wrap the array by `StructType(Seq(StructField(ArrayType(IntegerType, ...))))`, right? And use the code inside of `JacksonParser` which we disabled by `allowArrayAsStructs` for now?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org