Thanks Simon.

 

--T.

 

From: Simon Elliston Ball <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Monday, February 3, 2020 at 2:02 PM
To: "[email protected]" <[email protected]>
Subject: Re: Using something other than colons in field names?

 

The colons were originally required to avoid poor performance in sub documents 
in elastic search 2.x. It’s really a legacy thing now, and the NOOP should be 
considered the best path going forward.

 

Simon

 

On Mon, 3 Feb 2020 at 12:41, Yerex, Tom <[email protected]> wrote:

Thank you Vladimir.

Before I go diving into making a lot of changes from the default, does anyone 
happen to recall why the colon was selected as the default? I'm curious if it 
works better for analysis in HDFS or Zeppelin?

Cheers,

Tom.

On 2020-02-02, 8:53 PM, "Vladimir Mikhailov" <[email protected]> 
wrote:

    Hi

    There is a parameter "fieldNameConverter" in the parser indexing 
configuration:

    fieldNameConverter  

    "Defines how field names are transformed before being written to the index. 
Only applicable to elasticsearch.       

    Defaults to DEDOT. Acceptable values are DEDOT that replaces all '.' with 
':' or NOOP that does not change the field names."

    
https://github.com/apache/metron/blob/master/metron-platform/metron-indexing/metron-indexing-common/README.md#sensor-indexing-configuration

    Usage example:

        "elasticsearch": {
                "batchSize": 100,
                "enabled": true,
                "index": "myindex",
                "fieldNameConverter": "NOOP"
        },

    On 2020/02/01 00:00:04, "Yerex, Tom" <[email protected]> wrote: 
    > Good afternoon,
    > 
    >  
    > 
    > Our Metron installation uses colons in the field names. For example, geo 
ip enriched data appears as “enrichments:geo:ip_dst_addr:country”. Under Kibana 
(and from what I read Banana), the colon cannot be properly escaped for use 
with Timelion.
    > 
    >  
    > 
    > My question: has anyone figured out a way to escape colons in their query 
or another work around in general? Is there a setting somewhere that can be 
used to change the default from a colon to a period or another character?
    > 
    >  
    > 
    > Thank you,
    > 
    >  
    > 
    > Tom.
    > 
    >  
    > 
    > 

-- 

--

simon elliston ball

@sireb

Attachment: smime.p7s
Description: S/MIME cryptographic signature

Reply via email to