What might even be more interesting would be to have stellar evaluate conditions and set the index based on the evaluation:
pseudo: IF ( parser == BRO ) THEN match(FIELD =x) index = y or something On December 19, 2019 at 05:14:01, Vladimir Mikhailov ( [email protected]) wrote: Hi HDFS Writer has great functionality for defining the destination folder for indexing data: { "index": "bro", "batchSize": 5, "outputPathFunction": "FORMAT('uid-%s', uid)" } https://github.com/apache/metron/blob/master/metron-platform/metron-writer/README.md#hdfs-writer Is it possible to make similar functionality for Elasticsearch Writer? (something like "outputIndexFunction") Typical usage scenarios for this: - different indices of the same data type for different clients, - separate indices for pilot projects, - different rollover policies for indices, depending on the condition. It would also be convenient to have such setting at the global level so that it affects all active parsers.
