Jakub Dubovsky created SPARK-3121: ------------------------------------- Summary: Wrong implementation of implicit bytesWritableConverter Key: SPARK-3121 URL: https://issues.apache.org/jira/browse/SPARK-3121 Project: Spark Issue Type: Bug Affects Versions: 1.0.2 Reporter: Jakub Dubovsky Priority: Minor
val path = ... //path to seq file with BytesWritable as type of both key and value val file = sc.sequenceFile[Array[Byte],Array[Byte]](path) file.take(1)(0)._1 This prints incorrect content of byte array. Actual content starts with correct one and some "random" bytes and zeros are appended. BytesWritable has two methods: getBytes() - return content of all internal array which is often longer then actual value stored. It usually contains the rest of previous longer values copyBytes() - return just begining of internal array determined by internal length property It looks like in implicit conversion between BytesWritable and Array[byte] getBytes is used instead of correct copyBytes. -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org