Hi guys! I'm currently working with HDFS2 producer and I've got a case when I need to dynamically specify path to a target file. Let's say my path to a destination file consists of two parts: The first one is a static path and the second one is dynamically defined path which depends on realtime processing context. I know that it is possible to setup a file name dynamically by specifying the FILE_NAME header value. This option almost works for me but at the same time it introduces another problem - setting up this header makes hdfs producer create a new file on each write. I propose to use FILE_PATH header to be able to dynamically specify path to the file. I can create a patch and tests for this case. What do you think about this?
Regards, Mikhail