How to write value only using flink's SequenceFileWriter?

2019-07-26 Thread Liu Bo
Dear flink users, We're trying to switch from StringWriter to SequenceFileWriter to turn on compression. StringWriter writes value only and we want to keep that way. AFAIK, you can use NullWritable in Hadoop writers to escape key so you only write the values. So I tried with NullWritable as foll

Re: How to write value only using flink's SequenceFileWriter?

2019-07-26 Thread Liu Bo
The file header says key is NullWritable: SEQ^F!org.apache.hadoop.io.NullWritable^Yorg.apache.hadoop.io.Text^A^A)org.apache.hadoop.io.compress.SnappyCodec Might be a hadoop -text problem? On Sat, 27 Jul 2019 at 11:07, Liu Bo wrote: > Dear flink users, > > We're trying to switch from StringWrit

Re:Re: How to write value only using flink's SequenceFileWriter?

2019-08-06 Thread Haibo Sun
Hi Liu Bo, If you haven't customize serializations through the configuration item "io.serializations", the default serializer for Writable objects is org.apache.hadoop.io.serializer.WritableSerialization.WritableSerializer. As you said, when WritableSerializer serialize the NullWritable object