Can you elaborate on what you mean by "convert to internal?" From your description, it looks like the challenge is from our violations of DRY when it comes to constants referencing those keys, which would be eliminated by refactoring.
On Fri, Sep 7, 2018, 3:50 PM Ryan Merriman <merrim...@gmail.com> wrote: > I recently worked on a PR that involved changing the default behavior of > the ElasticsearchWriter to store data using field names with the default > Metron separator, dots. One of the unfortunate consequences of this is > that although dots are allowed in more recent versions of ES, it changes > how these fields are stored. Having a dot in a field name causes ES to > treat it as an object field type. We're not quite comfortable with this > because it could introduce unforeseen side effects that may not be > obvious. Here's the PR: https://github.com/apache/metron/pull/1181 > > As I worked through it I noticed there are a couple fields that include > separators where it's not actually necessary. They are not nested by > nature and are internal to Metron. The fact that they are internal means > they show up in constants and are hardcoded in several different places. > That made the work in the PR above much harder and tedious than it should > have been. There are 2 in particular that I had to deal with: source:type > and threat:triage:score in metaalerts. > > Is it worth considering converting these to internal Metron fields so that > they stay constant and this isn't a problem in the future? I could see > these fields following the same pattern as 'metron_alert'. However this > would cause pain when upgrading because existing data would need to be > updated with these new fields. > > Just an idea. Curious if other have an opinion on the subject. >
