Ted Yu created SPARK-23347: ------------------------------ Summary: Introduce buffer between Java data stream and gzip stream Key: SPARK-23347 URL: https://issues.apache.org/jira/browse/SPARK-23347 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.2.0 Reporter: Ted Yu
Currently GZIPOutputStream is used directly around ByteArrayOutputStream e.g. from KVStoreSerializer : {code} ByteArrayOutputStream bytes = new ByteArrayOutputStream(); GZIPOutputStream out = new GZIPOutputStream(bytes); {code} This seems inefficient. GZIPOutputStream does not implement the write(byte) method. It only provides a write(byte[], offset, len) method, which calls the corresponding JNI zlib function. BufferedOutputStream can be introduced wrapping GZIPOutputStream for better performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org