Thanks for your help, Jochen. as I'm new to ElasticSearch and Graylog, deleting data without understanding is not safe, I guess. but... seems like there's no solution for it... I'll try it. :)
Thanks. On Thursday, November 3, 2016 at 8:22:26 PM UTC+9, Jochen Schalanda wrote: > > Hi Eugene, > > once you've identified the duplicate documents in Elasticsearch, you can > remove these using the Elasticsearch Delete API ( > https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-delete.html) > or the Delete By Query plugin ( > https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/delete-by-query-usage.html > ). > > Cheers, > Jochen > > On Thursday, 3 November 2016 11:54:49 UTC+1, Eugene Gwon wrote: >> >> Hello, Jochen. >> >> I've checked that link but unfortunately, I'm new to ElasticSearch. and >> It seems like that link only shows 'how to check duplicate data'. >> does it mean that I insert data first and check duplicate, and than >> delete duplicated data? >> >> I'm confusing :( >> I'm really sorry but would you please explain more detail? >> >> Thanks for your help, Jochen. >> >> On Thursday, November 3, 2016 at 7:08:29 PM UTC+9, Jochen Schalanda wrote: >>> >>> Hi Eugene, >>> >>> Graylog itself doesn't filter out "duplicate" data. >>> >>> You could check your Elasticsearch indices using the approach laid out >>> in https://qbox.io/blog/minimizing-document-duplication-in-elasticsearch >>> after ingesting the missing data. >>> >>> Cheers, >>> Jochen >>> >>> On Thursday, 3 November 2016 09:48:42 UTC+1, Eugene Gwon wrote: >>>> >>>> Hello all, >>>> I have a problem. please help :( >>>> I'm running 3 node of Elasticsearch. 1 master + data, and 2 data node. >>>> log shipper is fluentd. >>>> Unfortunately, today, I accidentally restarted graylog server. >>>> this caused lost of some data, which sent from another servers. >>>> >>>> so I must re-insert data, but WITHOUT duplication. >>>> Is it possible? >>>> >>>> or, can I just filtering duplicated data when search? >>>> >>>> Thanks for your help. >>>> >>>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/1972f102-93e7-4c18-8359-78da7fabc05e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
