Github user witgo commented on the issue: https://github.com/apache/spark/pull/14995 This PR is Test only, it used to 1. verify code through CI 2. verify the effectiveness of the solution includes two underlying API changes. 1. Replace ByteBuffer with ChunkedByteBuffer. 2. Replace ByteBuf with InputStream. There should not be much debate about 1.(Master branch has done some of the relevant changes), But @rxin has a different idea for 2. We should reach a consensus on the above two underlying changes, and then do the next step. @srowen What do you think of the above two changes? @opme Have you done more test on large scale data shuflle?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org