Re: Grok Reader, Extract and the default patterns
It is also possible that Extract grok should be @depricated and noted as such in it’s documentation, with the GrokReader being refactored to have all the same levers to tune. But I would suggest that that would or should be done after the upgrade to 0.1.9 ( latest ), and after the follow on and current pr’s about empty capture returns and multiple pattern files. I would guess that the outstanding Jira’s would also be evaluated before depreciation. On May 8, 2018 at 21:37:41, Otto Fowler (ottobackwa...@gmail.com) wrote: I would add a test for putting in the defaults as a file when they are added by the processor as well. =or= I could expose a new property to turn it on. But really, by rule of least surprise, I would expect the core groks to always exist and to be able to overwrite if necessary. We do that in metron. There is something that feels wrong about possible duplication between the reader serialization service nar and standard processors ( GrokExpressionValidator for example ) but that is another kettle of fish. On May 8, 2018 at 21:33:59, Otto Fowler (ottobackwa...@gmail.com) wrote: I think so, the way that java-grok works ( refactored for 0.1.9 ) is that you register sets of patterns with the compiler, and the compiler produces the Grok object. When registering things with the same name, the last one in wins, so I think if an existing user had put a pattern file in the configuration with patterns named the same as the defaults, then they would just replace them in the map. On May 8, 2018 at 21:24:35, Joe Witt (joe.w...@gmail.com) wrote: You're probably not missing anything. ExtractGrok came before the record reader/writer enlightenment phase. If you think ExtractGrok is useful and want to improve it as you suggest I think you're good to go provided you do so in a way that doesn't break existing flows (change their behavior unless they opt in so to speak). Is that feasible for your idea? Thanks On Tue, May 8, 2018 at 9:11 PM, Otto Fowler wrote: > I’m working on upgrading java-grok to the new 0.1.9 release. While going > through the GrokReader the the ExtractGrok components I noticed that they > differ in a very important way grok wise. > The reader loads the default patterns ( which are a copy of the ubiquitous > default patterns in java-grok itself. The older ( I assume ) ExtractGrok > processor does not. > > I’m wondering if there is a reason for this going back to the creation of > the processor. It seems to me it would be better for the ExtractGrok > processor and the GrokReader to work similarly with regards > to the default patterns. At the moment, there is only one pattern file > allowed to be specified with ExtractGrok ( I think there is PR for multiple > pattern files ). Besides consistency between the two components, > it seems like it would be better for users if they didn’t have to waste or > merge pattern files to get the common patterns. > > I apologize in advance if I am missing something here. > > Would anyone object to setting the default patterns as we do in the > GrokReader? > > ottO
Re: Grok Reader, Extract and the default patterns
I would add a test for putting in the defaults as a file when they are added by the processor as well. =or= I could expose a new property to turn it on. But really, by rule of least surprise, I would expect the core groks to always exist and to be able to overwrite if necessary. We do that in metron. There is something that feels wrong about possible duplication between the reader serialization service nar and standard processors ( GrokExpressionValidator for example ) but that is another kettle of fish. On May 8, 2018 at 21:33:59, Otto Fowler (ottobackwa...@gmail.com) wrote: I think so, the way that java-grok works ( refactored for 0.1.9 ) is that you register sets of patterns with the compiler, and the compiler produces the Grok object. When registering things with the same name, the last one in wins, so I think if an existing user had put a pattern file in the configuration with patterns named the same as the defaults, then they would just replace them in the map. On May 8, 2018 at 21:24:35, Joe Witt (joe.w...@gmail.com) wrote: You're probably not missing anything. ExtractGrok came before the record reader/writer enlightenment phase. If you think ExtractGrok is useful and want to improve it as you suggest I think you're good to go provided you do so in a way that doesn't break existing flows (change their behavior unless they opt in so to speak). Is that feasible for your idea? Thanks On Tue, May 8, 2018 at 9:11 PM, Otto Fowler wrote: > I’m working on upgrading java-grok to the new 0.1.9 release. While going > through the GrokReader the the ExtractGrok components I noticed that they > differ in a very important way grok wise. > The reader loads the default patterns ( which are a copy of the ubiquitous > default patterns in java-grok itself. The older ( I assume ) ExtractGrok > processor does not. > > I’m wondering if there is a reason for this going back to the creation of > the processor. It seems to me it would be better for the ExtractGrok > processor and the GrokReader to work similarly with regards > to the default patterns. At the moment, there is only one pattern file > allowed to be specified with ExtractGrok ( I think there is PR for multiple > pattern files ). Besides consistency between the two components, > it seems like it would be better for users if they didn’t have to waste or > merge pattern files to get the common patterns. > > I apologize in advance if I am missing something here. > > Would anyone object to setting the default patterns as we do in the > GrokReader? > > ottO
Re: Grok Reader, Extract and the default patterns
I think so, the way that java-grok works ( refactored for 0.1.9 ) is that you register sets of patterns with the compiler, and the compiler produces the Grok object. When registering things with the same name, the last one in wins, so I think if an existing user had put a pattern file in the configuration with patterns named the same as the defaults, then they would just replace them in the map. On May 8, 2018 at 21:24:35, Joe Witt (joe.w...@gmail.com) wrote: You're probably not missing anything. ExtractGrok came before the record reader/writer enlightenment phase. If you think ExtractGrok is useful and want to improve it as you suggest I think you're good to go provided you do so in a way that doesn't break existing flows (change their behavior unless they opt in so to speak). Is that feasible for your idea? Thanks On Tue, May 8, 2018 at 9:11 PM, Otto Fowler wrote: > I’m working on upgrading java-grok to the new 0.1.9 release. While going > through the GrokReader the the ExtractGrok components I noticed that they > differ in a very important way grok wise. > The reader loads the default patterns ( which are a copy of the ubiquitous > default patterns in java-grok itself. The older ( I assume ) ExtractGrok > processor does not. > > I’m wondering if there is a reason for this going back to the creation of > the processor. It seems to me it would be better for the ExtractGrok > processor and the GrokReader to work similarly with regards > to the default patterns. At the moment, there is only one pattern file > allowed to be specified with ExtractGrok ( I think there is PR for multiple > pattern files ). Besides consistency between the two components, > it seems like it would be better for users if they didn’t have to waste or > merge pattern files to get the common patterns. > > I apologize in advance if I am missing something here. > > Would anyone object to setting the default patterns as we do in the > GrokReader? > > ottO
Re: Grok Reader, Extract and the default patterns
You're probably not missing anything. ExtractGrok came before the record reader/writer enlightenment phase. If you think ExtractGrok is useful and want to improve it as you suggest I think you're good to go provided you do so in a way that doesn't break existing flows (change their behavior unless they opt in so to speak). Is that feasible for your idea? Thanks On Tue, May 8, 2018 at 9:11 PM, Otto Fowler wrote: > I’m working on upgrading java-grok to the new 0.1.9 release. While going > through the GrokReader the the ExtractGrok components I noticed that they > differ in a very important way grok wise. > The reader loads the default patterns ( which are a copy of the ubiquitous > default patterns in java-grok itself. The older ( I assume ) ExtractGrok > processor does not. > > I’m wondering if there is a reason for this going back to the creation of > the processor. It seems to me it would be better for the ExtractGrok > processor and the GrokReader to work similarly with regards > to the default patterns. At the moment, there is only one pattern file > allowed to be specified with ExtractGrok ( I think there is PR for multiple > pattern files ). Besides consistency between the two components, > it seems like it would be better for users if they didn’t have to waste or > merge pattern files to get the common patterns. > > I apologize in advance if I am missing something here. > > Would anyone object to setting the default patterns as we do in the > GrokReader? > > ottO