Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21601 I read your changes and in the test I was searching for an check/assert but found none. I understand it is about checking no exception is thrown during the directory content reading but I still missed some asserts and covering more cases, at least: - min split size per node < maxSplitSize && min split size per rack < maxSplitSize - min split size per node > maxSplitSize && min split size per rack < maxSplitSize - min split size per node < maxSplitSize && min split size per rack > maxSplitSize As I see it is hard to add checks/asserts but what about testing WholeTextFileInputFormat directly? In your test you could inherit from WholeTextFileInputFormat and override the protected setters for maxSplitSize, minSplitSizeNode, minSplitSizeRack and and store the values in your new test class so asserts and checks can be added.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org