batch jobs. With the DNS example, you might imagine a future
> where the enrichment data is streamed based on DHCP registrations, DNS
> update events, etc. In principle this could reduce the window of time where
> we might enrich a data source with out-of-date data.
> >
> >Charlie
> &g
gt;> SimpleHbaseEnrichmentWriter",
>>> "sensorTopic": "dns",
>>> "parserConfig": {
>>> "shew.table": " dns",
>>> "shew.cf": "dns",
>>&g
nt: 12 June 2018 20:33
To: dev@metron.apache.org
Subject: Re: Writing enrichment data directly from NiFi with PutHBaseJSON
I like the streaming enrichment solutions but it depends on how you are getting
the data in. If you get the data in a csv file just call the flat file loader
from a script
Having it in it’s own repo doesn’t tie it to Metron any less functional
wise, but allows
for a new release with Nifi only changes to be produced, or multiple
streams of releases
across nifi versions ( 1.7.x, 1.8.x ) to be produced.
On June 5, 2018 at 15:14:38, Casey Stella (ceste...@gmail.com)
Also, the bundle would be part of the metron project I expect, so the NiFi
release shouldn’t matter much, now NiFi can version only processors
independently.
Simon
> On 5 Jun 2018, at 20:14, Casey Stella wrote:
>
> I agree with Simon here, the benefit of providing NiFi tooling is to enable
I agree with Simon here, the benefit of providing NiFi tooling is to enable
NiFi to use our infrastructure (e.g. our parsers, MaaS, stellar
enrichments, etc). This would tie it to Metron pretty closely.
On Tue, Jun 5, 2018 at 3:12 PM Otto Fowler wrote:
> Nifi releases more often then Metron
Nifi releases more often then Metron does, that might be an issue.
On June 5, 2018 at 14:07:22, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
To be honest, I would expect this to be heavily linked to the Metron
releases, since it's going to use other metron classes and dependencies
Similar to Bro, we may need to release out of cycle.
On June 5, 2018 at 13:17:55, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
Do you mean in the sense of a separate module, or are you suggesting we go
as far as a sub-project?
On 5 June 2018 at 10:08, Otto Fowler wrote:
> If we
Do you mean in the sense of a separate module, or are you suggesting we go
as far as a sub-project?
On 5 June 2018 at 10:08, Otto Fowler wrote:
> If we do that, we should have it as a separate component maybe.
>
>
> On June 5, 2018 at 12:42:57, Simon Elliston Ball (
>
If we do that, we should have it as a separate component maybe.
On June 5, 2018 at 12:42:57, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
@otto, well, of course we would use the record api... it's great.
@casey, I have actually written a stellar processor, which applies stellar
to
@otto, well, of course we would use the record api... it's great.
@casey, I have actually written a stellar processor, which applies stellar
to all FlowFile attributes outputting the resulting stellar variable space
to either attributes or as json in the content.
Is it worth us creating an
PutMetronEnrichementRecords* ;)
On June 5, 2018 at 10:32:43, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
Do we, the community, think it would be a good idea to create a
PutMetronEnrichment NiFi processor for this use case? It seems a number of
people want to use NiFi to manage
I'd be in strong support of that, Simon. I think we should have some other
NiFi components in Metron to enable users to interact with our
infrastructure from NiFi (e.g. being able to transform via stellar, etc).
On Tue, Jun 5, 2018 at 10:32 AM Simon Elliston Ball <
si...@simonellistonball.com>
The problem, as you correctly diagnosed, is the key in HBase. We construct
the key very specifically in Metron, so it's unlikely to work out of the
box with the NiFi processor unfortunately. The key that we use is formed
here in the codebase:
14 matches
Mail list logo
