Re: [graylog2] Re: Backfilling graylog with past data
On Fri, Jul 15, 2016 at 2:50 AM, Jeremy Farr wrote: > Jason have you noticed any issues when adding to indices that are not the > currently active one? > No. My indices don't last more than an hour and I have shoved in data that was days old - so it definitely all went into "old" indices. Waitaminute - that's not how it works. Mustn't it always go into the current index, even if the timestamps are no longer vaguely related? I can't say I've thought much about it - it simply worked -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/CAFChrg%2BYuLYWXUSX0BuqQWUf-Yo72AZrHiZF1TFviv-inu6kNg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Backfilling graylog with past data
Thank you Jochen. On Wednesday, July 13, 2016 at 2:14:45 AM UTC-5, Jochen Schalanda wrote: > > Hi Jeremy, > > you can use Logstash or Filebeat (or any other log shipper) to backfill > data into Graylog, too. Simply point it to the file (or source) you want to > use as an input and use a GELF output to send data into Graylog. Also make > sure that the timestamp field is valid, because otherwise Graylog would > use the ingestion time as timestamp (which is not what you want to have > when filling in historic logs). > > Cheers, > Jochen > > On Wednesday, 13 July 2016 04:10:04 UTC+2, Jeremy Farr wrote: >> >> How would I go about backfilling logs into graylog? Does it just handle >> it auto-magically? For instance, I'd like to analyze some transaction data >> that spans possibly the entire month. I can get the information at smaller >> intervals (i.e. Daily or weekly) but I would only be looking at it in >> monthly, quarterly or annual periods of time. I've seen people discussing >> using logstash to backfill elasticsearch but I couldn't find anything about >> back filling graylog specifically. Thanks in advance. > > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/27299714-53ae-4084-b564-18016c78721b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Backfilling graylog with past data
Jason have you noticed any issues when adding to indices that are not the currently active one? On Thursday, July 14, 2016 at 2:35:26 AM UTC-5, Jason Haar wrote: > > > On Wed, Jul 13, 2016 at 7:14 PM, Jochen Schalanda > wrote: > >> Simply point it to the file (or source) you want to use as an input and >> use a GELF output to send data into Graylog > > > I use that all the time - works great! Except I have a mental block and > keep "search" looking in the past 5 minutes and wonder why I don't see the > data I just pushed in (which typically had yesterday's date ;-) > > > -- > Cheers > > Jason Haar > Information Security Manager, Trimble Navigation Ltd. > Phone: +1 408 481 8171 > PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/3e31b812-5b0d-4ee5-b374-2cd067dc308c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Backfilling graylog with past data
On Wed, Jul 13, 2016 at 7:14 PM, Jochen Schalanda wrote: > Simply point it to the file (or source) you want to use as an input and > use a GELF output to send data into Graylog I use that all the time - works great! Except I have a mental block and keep "search" looking in the past 5 minutes and wonder why I don't see the data I just pushed in (which typically had yesterday's date ;-) -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/CAFChrgLNO%3DnHqf3_%3DB4jJG-O9dT9JC4c0BNwVKyF1%3DS56Wn55A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Backfilling graylog with past data
Hi Jeremy, you can use Logstash or Filebeat (or any other log shipper) to backfill data into Graylog, too. Simply point it to the file (or source) you want to use as an input and use a GELF output to send data into Graylog. Also make sure that the timestamp field is valid, because otherwise Graylog would use the ingestion time as timestamp (which is not what you want to have when filling in historic logs). Cheers, Jochen On Wednesday, 13 July 2016 04:10:04 UTC+2, Jeremy Farr wrote: > > How would I go about backfilling logs into graylog? Does it just handle > it auto-magically? For instance, I'd like to analyze some transaction data > that spans possibly the entire month. I can get the information at smaller > intervals (i.e. Daily or weekly) but I would only be looking at it in > monthly, quarterly or annual periods of time. I've seen people discussing > using logstash to backfill elasticsearch but I couldn't find anything about > back filling graylog specifically. Thanks in advance. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/1baca34e-1a98-4404-ac32-0083eab5008c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
