jiayuasu commented on issue #814: URL: https://github.com/apache/sedona/issues/814#issuecomment-1500977146
@conect This is in fact not possible. JSON in general is a very bad format for big data. Parsing a large json dataset is known to be very slow due to (1) json string parsing is slow anyway (2) multi-line json cannot be chopped to multiple partitions and read in parallel. The reader has to figure out a way to honor the json that crosses multiple partitions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
