I personally like the idea of a typed schema per parser that we could translate to multiple targets. This would allow us a lot more modularity and extensibility in indexing down the road.
-Kyle On Fri, Feb 17, 2017 at 1:59 PM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > That sounds like a great idea Otto. Do you have any early design on that > we can look at. Also, rather than just elastic templates do you think we > should have some sort of typed schema we could translate to multiple > targets (solr, elastic, ur... other...) or are you thinking of packaging > specific scheme assets like template json with the parser? > > Simon > > > On 17 Feb 2017, at 18:42, Otto Fowler <ottobackwa...@gmail.com> wrote: > > > > > > Not to jump the gun, but I’m crafting a proposal about parsers and one > of the things I am going to propose relates to having the ES Template for a > given parser installed or packaged with the parser. We could load the > template from there, edit, save and deploy etc. We can extend that concept > more and more later (drafts, versioning etc ) > > > > > >> On February 17, 2017 at 13:22:45, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > >> > >> A little while ago the issue of managing Elastic templates for new > sensor configs came up, and we didn’t quite put it to bed. > >> > >> When creating new sensors, I almost invariably find the auto-generated > schemas for elastic pick some incorrect types. I also find I have to > recreate indexes every time to push in the proper dynamic templates for > things like geo enrichment fields. > >> > >> So, my questions are: > >> How should we address elastic template for new sensors? > >> Do we have circumstances where we would need to configure types, or can > we get away with inferring them? > >> Should we just add some additional dynamic templates to cover our > common fields like timestamp (the most common culprit I find for incorrect > typing)? > >> > >> I’d also like to think about ways we can generalise this. Does anyone > have any thoughts on what sort of additional index schemes we should want > to infer (solr seems an obvious one, any others?). > >> > >> Thoughts on a well typed, schemaed and easily indexed postcard please :) > >> > >> Simon >
