Re: BucketAssigner - Confusion

2019-04-03 Thread Chesnay Schepler
BucketID is a variable type, and conceptually you can use any type so long as you can provide a serializer for it (BucketAssigner#getSerializer). The documentation is wrong in this instance. The convenience Flink APIs (StreamingFileSink#forRowFormat/StreamingFileSink#forBulkFormat) default

Re: BucketAssigner - Confusion

2019-04-02 Thread Jeff Crane
According to my IDE (Jetbrains), I get an error with getBucketID(IN, Context) signature requiring a return of string (Flink 1.7 libs), so I still don't think the BucketID is a variable type. I still don't understand the role of the:public SimpleVersionedSerializer getSerializer() {

BucketAssigner - Confusion

2019-04-01 Thread Jeff Crane
I have had an issue understanding the documentation, in regard to BucketAssigner.BucketID getBucketId(IN element, BucketAssigner.Context  context)SimpleVersionedSerializer getSerializer()First of all, I don't understand what type of "BucketID" means. I assume that's the