Jira for this is https://issues.apache.org/jira/browse/BEAM-3222
Also looking at https://github.com/json-path/JsonPath for providing the json fetching. On Fri, Nov 17, 2017 at 11:43 AM, NerdyNick <nerdyn...@gmail.com> wrote: > So I'm looking to expand the ElasticSearchIO writer to support dynamic > indexes. Wanted to kick off talking about design. I'll create a Jira for it > once I get going. > > Right now I'm thinking to expand the ConnectionConfiguration class to > accept either an index (Current), dot-path string to look into the JSON > with to pull the index value, or a user provided function that takes the > document and returns an index. The Write class will change to use a > MultiMap where keys are the index. The whole Multimap will be treated as > the source for the batch sizes. I opt for that since I see the index > usually being the same until time window endings. > > I'm aiming for the simplest change in design here. To not impact the work > Chet is doing on BEAM-3201. > > -- > Nick Verbeck - NerdyNick > ---------------------------------------------------- > NerdyNick.com > TrailsOffroad.com > NoKnownBoundaries.com > > > -- Nick Verbeck - NerdyNick ---------------------------------------------------- NerdyNick.com TrailsOffroad.com NoKnownBoundaries.com
