HyukjinKwon commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368388820
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##########
@@ -203,7 +202,11 @@ class UnivocityParser(
}
}
- private val doParse = if (options.columnPruning && requiredSchema.isEmpty) {
+ /**
+ * Parses a single CSV string and turns it into either one resulting row or
no row (if the
+ * the record is malformed).
+ */
+ val parse: String => Option[InternalRow] = if (options.columnPruning &&
requiredSchema.isEmpty) {
Review comment:
In the current way, it saves one comparison for each record vs the
suggestion needs to compare for every record. It's maybe no big deal but let me
just don't change any behaviours for now in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
