Hi
HDFS Writer has great functionality for defining the destination folder for
indexing data:
{
"index": "bro",
"batchSize": 5,
"outputPathFunction": "FORMAT('uid-%s', uid)"
}
https://github.com/apache/metron/blob/master/metron-platform/metron-writer/README.md#hdfs-writer
Is it possible to make similar functionality for Elasticsearch Writer?
(something like "outputIndexFunction")
Typical usage scenarios for this:
- different indices of the same data type for different clients,
- separate indices for pilot projects,
- different rollover policies for indices, depending on the condition.
It would also be convenient to have such setting at the global level so that it
affects all active parsers.
