[GitHub] metron issue #1269: METRON-1879 Allow Elasticsearch to Auto-Generate the Doc...

2018-12-05 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1269
  
Prior to this change we use the Metron-generated GUID as the documentID.  
They were always the same.  There were various places in the code where the 
name `guid`, `docID`, or `id` were used that implicitly meant the Metron GUID.

With this change, the Metron-generated guid will never be equal to the 
Elasticsearch-generated doc ID. You can view and use either the guid or the doc 
ID in the UI.  Both are available as separate fields in the index.

This PR has pulled on those strings and tried to make clear what is a 
Metron GUID and what is a document ID.  In the Alerts UI you can see and/or use 
either the GUID or the document ID.  It is up to the user which one they care 
to see.  Although most users will not care what the document ID is.

From my PR description:

> This change is backwards compatible. The Alerts UI should continue to 
work no matter if some of the underlying indices were written with the Metron 
GUID as the document ID and some are written with the auto-generated document 
ID.

There is no option to continue to use the GUID as the documentID.  I can't 
think of a use case worthy of the additional effort and testing needed to 
support that.

It is backwards compatible in that the Alerts UI will work when searching 
over both "legacy" indices where guid = docID and "new" indices where guid != 
docID.  All places in the code where a docID is needed, that docID is actually 
first retrieved from Elasticsearch, rather than making an assumption about what 
it is.






---


[GitHub] metron issue #1269: METRON-1879 Allow Elasticsearch to Auto-Generate the Doc...

2018-12-04 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1269
  
Refresh me on this - prior to this change, did we provide both a doc id and 
a guid that were identical? And now with this change you would expect to see 
guid != doc id (bc guid generated by Metron, doc id now generated by ES), 
except where otherwise configured for backwards compatibility?


---


[GitHub] metron issue #1269: METRON-1879 Allow Elasticsearch to Auto-Generate the Doc...

2018-11-19 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1269
  
This issue has been fixed.


---


[GitHub] metron issue #1269: METRON-1879 Allow Elasticsearch to Auto-Generate the Doc...

2018-11-19 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1269
  
I was doing some final testing and noticed that the 'guid' field for 
metaalerts is showing the document ID, rather than the guid.  I need to fix the 
presentation of metaalerts in the UI.

![screen shot 2018-11-19 at 10 56 19 
am](https://user-images.githubusercontent.com/2475409/48718694-e6fc4580-ebe9-11e8-9eeb-2650767b1f55.png)



---