from:"Arkadiy Shinkarev"

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Arkadiy Shinkarev

Thank you for your reply!

Additionally - I just upgraded Elasticsearch cluster to 1.7.2 and run 
recalculate index range job - it fixed the problem.


On Monday, September 21, 2015 at 2:45:04 PM UTC+3, Jochen Schalanda wrote:
>
> Hi Arkadiy,
>
> thanks for posting these log messages. The underlying issue will be fixed 
> in Graylog 1.2.1 which we plan to release soon (see 
> https://github.com/Graylog2/graylog2-server/pull/1427).
>
>
> Cheers,
> Jochen
>
> On Monday, 21 September 2015 11:37:31 UTC+2, Arkadiy Shinkarev wrote:
>>
>> Now I see following errors while try to recalculate index ranges:
>> 2015-09-21T12:25:29.681+03:00 INFO  [RebuildIndexRangesJob] 
>> Re-calculating index ranges.
>> 2015-09-21T12:25:29.681+03:00 INFO  [SystemJobManager] Submitted 
>> SystemJob  
>> [org.graylog2.indexer.ranges.RebuildIndexRangesJob]
>> 2015-09-21T12:25:33.656+03:00 INFO  [RebuildIndexRangesJob] Could not 
>> calculate range of index [graylog2_30]. Skipping.
>> java.lang.IllegalArgumentException: Invalid format: "1.436259964997E12" 
>> is malformed at ".436259964997E12"
>> at 
>> org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
>> at 
>> org.graylog2.indexer.ranges.EsIndexRangeService.timestampStatsOfIndex(EsIndexRangeService.java:258)
>> at 
>> org.graylog2.indexer.ranges.EsIndexRangeService.calculateRange(EsIndexRangeService.java:216)
>> at 
>> org.graylog2.indexer.ranges.RebuildIndexRangesJob.execute(RebuildIndexRangesJob.java:96)
>> at 
>> org.graylog2.system.jobs.SystemJobManager$1.run(SystemJobManager.java:88)
>> at 
>> com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(InstrumentedScheduledExecutorService.java:235)
>> at 
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> 2015-09-21T12:25:36.550+03:00 INFO  [RebuildIndexRangesJob] Could not 
>> calculate range of index [graylog2_41]. Skipping.
>> java.lang.IllegalArgumentException: Invalid format: "1.441446878921E12" 
>> is malformed at ".441446878921E12"
>> at 
>> org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
>> at 
>> org.graylog2.indexer.ranges.EsIndexRangeService.timestampStatsOfIndex(EsIndexRangeService.java:258)
>> at 
>> org.graylog2.indexer.ranges.EsIndexRangeService.calculateRange(EsIndexRangeService.java:216)
>> at 
>> org.graylog2.indexer.ranges.RebuildIndexRangesJob.execute(RebuildIndexRangesJob.java:96)
>> at 
>> org.graylog2.system.jobs.SystemJobManager$1.run(SystemJobManager.java:88)
>> at 
>> com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(InstrumentedScheduledExecutorService.java:235)
>> at 
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>     at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> On Monday, September 21, 2015 at 11:59:24 AM UTC+3, Arkadiy Shinkarev 
>> wrote:
>>>
>>> Current index is graylog2_44
>>>
>>> On Monday, September 21, 2015 at 11:58:19 AM UTC+3, Arkadiy Shinkarev 
>>> wrote:
>>>>
>>>> Thanks for your reply!
>>>>
>>>> There is no error messages in logs at all.
>>>>
>>>> $  curl -XGET '
>>>> http://m1-gl-es01:9200/graylog2

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Arkadiy Shinkarev

Now I see following errors while try to recalculate index ranges:
2015-09-21T12:25:29.681+03:00 INFO  [RebuildIndexRangesJob] Re-calculating 
index ranges.
2015-09-21T12:25:29.681+03:00 INFO  [SystemJobManager] Submitted SystemJob 

[org.graylog2.indexer.ranges.RebuildIndexRangesJob]
2015-09-21T12:25:33.656+03:00 INFO  [RebuildIndexRangesJob] Could not 
calculate range of index [graylog2_30]. Skipping.
java.lang.IllegalArgumentException: Invalid format: "1.436259964997E12" is 
malformed at ".436259964997E12"
at 
org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
at 
org.graylog2.indexer.ranges.EsIndexRangeService.timestampStatsOfIndex(EsIndexRangeService.java:258)
at 
org.graylog2.indexer.ranges.EsIndexRangeService.calculateRange(EsIndexRangeService.java:216)
at 
org.graylog2.indexer.ranges.RebuildIndexRangesJob.execute(RebuildIndexRangesJob.java:96)
at 
org.graylog2.system.jobs.SystemJobManager$1.run(SystemJobManager.java:88)
at 
com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(InstrumentedScheduledExecutorService.java:235)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-09-21T12:25:36.550+03:00 INFO  [RebuildIndexRangesJob] Could not 
calculate range of index [graylog2_41]. Skipping.
java.lang.IllegalArgumentException: Invalid format: "1.441446878921E12" is 
malformed at ".441446878921E12"
at 
org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
at 
org.graylog2.indexer.ranges.EsIndexRangeService.timestampStatsOfIndex(EsIndexRangeService.java:258)
at 
org.graylog2.indexer.ranges.EsIndexRangeService.calculateRange(EsIndexRangeService.java:216)
at 
org.graylog2.indexer.ranges.RebuildIndexRangesJob.execute(RebuildIndexRangesJob.java:96)
at 
org.graylog2.system.jobs.SystemJobManager$1.run(SystemJobManager.java:88)
at 
com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(InstrumentedScheduledExecutorService.java:235)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

On Monday, September 21, 2015 at 11:59:24 AM UTC+3, Arkadiy Shinkarev wrote:
>
> Current index is graylog2_44
>
> On Monday, September 21, 2015 at 11:58:19 AM UTC+3, Arkadiy Shinkarev 
> wrote:
>>
>> Thanks for your reply!
>>
>> There is no error messages in logs at all.
>>
>> $  curl -XGET '
>> http://m1-gl-es01:9200/graylog2_*/index_range/_search?pretty=true'
>>
>>   "took" : 38,
>>   "timed_out" : false,
>>   "_shards" : {
>> "total" : 40,
>> "successful" : 40,
>> "failed" : 0
>>   },
>>   "hits" : {
>> "total" : 20,
>> "max_score" : 1.0,
>> "hits" : [ {
>>   "_index" : "graylog2_25",
>>   "_type" : "index_range",
>>   "_id" : "graylog2_25",
>>   "_score" : 1.0,
>>   
>> "_source":{"gl2_index_range_index_name":"graylog2_25","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-06-17T21:06:14.000Z","gl2_index_range_
>> calculated_at":"2015-09-12T23:01:29.000Z","gl2_index_range_took_ms":4191}
>> }, {
>>   "_index" : "graylog2_27",
>>   "_type" : "index_range",
>>   "_id" : "graylog2_27",
>>   "_score" :

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Arkadiy Shinkarev

Current index is graylog2_44

On Monday, September 21, 2015 at 11:58:19 AM UTC+3, Arkadiy Shinkarev wrote:
>
> Thanks for your reply!
>
> There is no error messages in logs at all.
>
> $  curl -XGET '
> http://m1-gl-es01:9200/graylog2_*/index_range/_search?pretty=true'
>
>   "took" : 38,
>   "timed_out" : false,
>   "_shards" : {
> "total" : 40,
> "successful" : 40,
> "failed" : 0
>   },
>   "hits" : {
> "total" : 20,
> "max_score" : 1.0,
> "hits" : [ {
>   "_index" : "graylog2_25",
>   "_type" : "index_range",
>   "_id" : "graylog2_25",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_25","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-06-17T21:06:14.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:01:29.000Z","gl2_index_range_took_ms":4191}
> }, {
>   "_index" : "graylog2_27",
>   "_type" : "index_range",
>   "_id" : "graylog2_27",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_27","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-06-26T22:43:14.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:01:09.000Z","gl2_index_range_took_ms":10686}
> }, {
>   "_index" : "graylog2_29",
>   "_type" : "index_range",
>   "_id" : "graylog2_29",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_29","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-07T09:06:05.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:00:45.000Z","gl2_index_range_took_ms":7077}
> }, {
>   "_index" : "graylog2_30",
>   "_type" : "index_range",
>   "_id" : "graylog2_30",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_30","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-11T21:37:45.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:00:35.000Z","gl2_index_range_took_ms":14713}
> }, {
>   "_index" : "graylog2_32",
>   "_type" : "index_range",
>   "_id" : "graylog2_32",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_32","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-22T05:26:06.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:01:33.000Z","gl2_index_range_took_ms":3401}
> }, {
>   "_index" : "graylog2_34",
>   "_type" : "index_range",
>   "_id" : "graylog2_34",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_34","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-01T16:14:17.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:00:58.000Z","gl2_index_range_took_ms":2560}
> }, {
>   "_index" : "graylog2_36",
>   "_type" : "index_range",
>   "_id" : "graylog2_36",
>   "_score" : 1.0,
>   
> "_source":{"gl2_index_range_index_name":"graylog2_36","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-13T10:29:23.000Z","gl2_index_range_
> calculated_at":"2015-09-12T23:02:12.000Z","gl2_index_range_took_ms":4309}
> }, {
>   "_index" : "graylog2_38",
>   "_type" : "index_range",
>   "_id" : "graylog2_38",
>   "_score" : 1.0,
>   
> "_source":{&qu

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Arkadiy Shinkarev

Thanks for your reply!

There is no error messages in logs at all.

$  curl -XGET 
'http://m1-gl-es01:9200/graylog2_*/index_range/_search?pretty=true'

  "took" : 38,
  "timed_out" : false,
  "_shards" : {
"total" : 40,
"successful" : 40,
"failed" : 0
  },
  "hits" : {
"total" : 20,
"max_score" : 1.0,
"hits" : [ {
  "_index" : "graylog2_25",
  "_type" : "index_range",
  "_id" : "graylog2_25",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_25","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-06-17T21:06:14.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:01:29.000Z","gl2_index_range_took_ms":4191}
}, {
  "_index" : "graylog2_27",
  "_type" : "index_range",
  "_id" : "graylog2_27",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_27","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-06-26T22:43:14.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:01:09.000Z","gl2_index_range_took_ms":10686}
}, {
  "_index" : "graylog2_29",
  "_type" : "index_range",
  "_id" : "graylog2_29",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_29","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-07T09:06:05.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:00:45.000Z","gl2_index_range_took_ms":7077}
}, {
  "_index" : "graylog2_30",
  "_type" : "index_range",
  "_id" : "graylog2_30",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_30","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-11T21:37:45.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:00:35.000Z","gl2_index_range_took_ms":14713}
}, {
  "_index" : "graylog2_32",
  "_type" : "index_range",
  "_id" : "graylog2_32",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_32","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-07-22T05:26:06.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:01:33.000Z","gl2_index_range_took_ms":3401}
}, {
  "_index" : "graylog2_34",
  "_type" : "index_range",
  "_id" : "graylog2_34",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_34","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-01T16:14:17.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:00:58.000Z","gl2_index_range_took_ms":2560}
}, {
  "_index" : "graylog2_36",
  "_type" : "index_range",
  "_id" : "graylog2_36",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_36","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-13T10:29:23.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:02:12.000Z","gl2_index_range_took_ms":4309}
}, {
  "_index" : "graylog2_38",
  "_type" : "index_range",
  "_id" : "graylog2_38",
  "_score" : 1.0,
  
"_source":{"gl2_index_range_index_name":"graylog2_38","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-26T06:34:49.000Z","gl2_index_range_
calculated_at":"2015-09-12T23:02:01.000Z","gl2_index_range_took_ms":7399}
}, {
  "_index" : "graylog2_41",
  "_type" : "index_range",
  "_id" : "graylog2_41",
  "_score"

[graylog2] Search issues after update to 1.2

2015-09-21 Thread Arkadiy Shinkarev

Hi!

I have search issues after update Graylog to 1.2.
I able to search data only in current index.
Actually, previous indexes contain all data (I see them when perform query 
to elastic).

I tryed to recalculate index range, but there is no luck.

Any suggestions?

ES: 1.3.4
Graylog (server, web): 1.2

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/bf0b7ea0-313e-492a-93a8-aa818dce9a60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Raw UDP buffer size

2015-05-07 Thread Arkadiy Shinkarev

Ok, will be waiting for fix.
Thank you!

On Thursday, May 7, 2015 at 6:50:59 PM UTC+3, Jochen Schalanda wrote:
>
> Hi Arkadiy,
>
> seems like you're right. I've just checked the relevant code for UDP 
> inputs in Graylog and the size of UDP packets is indeed limited to 8192 
> bytes. Unfortunately there's currently no configuration option to change 
> that but we'll address the issue in the next Graylog release.
>
> Until then I unfortunately can only recommend either using TCP or using an 
> intermediate log shipper like nxlog or logstash to get those messages into 
> Graylog. :(
>
>
> Cheers,
> Jochen
>
> On Thursday, 7 May 2015 17:24:58 UTC+2, Arkadiy Shinkarev wrote:
>>
>> Jochen, thanks for you reply!
>>
>> As I mentioned in my first post, I can see with tcpdump that packet 
>> lenght, that comes to Graylog node is more than 8192 bytes:
>> $ sudo tcpdump -n -i tunl0 port 12500 and udp
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on tunl0, link-type RAW (Raw IP), capture size 65535 bytes
>> 18:22:19.062304 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 54
>> 18:22:19.079891 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81
>> 18:22:19.113119 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 92
>> 18:22:19.117398 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 93
>> 18:22:19.121636 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81
>> 18:22:19.123707 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 94
>> 18:22:22.092734 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108
>> 18:22:22.093300 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 70
>> 18:22:22.238882 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 71
>> 18:22:24.067068 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 87
>> 18:22:26.148394 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 85
>> 18:22:27.477703 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 
>> 13642
>> 18:22:31.158020 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 91
>> 18:22:35.945376 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 69
>> 18:22:35.945489 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 91
>> 18:22:37.279499 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108
>>
>> Also, I have another node with Graylog2 0.20, messages from same sources 
>> comes to logstash UDP input without problems.
>>
>> So, I don't thinks that it is network related problems.
>>
>> On Thursday, May 7, 2015 at 4:04:26 PM UTC+3, Jochen Schalanda wrote:
>>>
>>> Hi Arkadiy,
>>>
>>> not all network devices support UDP packets bigger than 8KiB (8192 
>>> bytes) and this seems to be the case on your network somewhere. To 
>>> circumvent this restriction, Graylog or more specifically the GELF format 
>>> supports chunking which means splitting a large message into multiple UDP 
>>> packets of a certain size (see https://www.graylog.org/resources/gelf 
>>> for details on the GELF format). So in order to solve your problem you 
>>> could collect the logs on the same machine which produces those raw log 
>>> lines with a log shipper like nxlog or logstash and send them to Graylog 
>>> via GELF. Alternatively you'll have to switch from UDP to TCP which doesn't 
>>> suffer this kind of restriction.
>>>
>>> Cheers,
>>> Jochen
>>>
>>> On Thursday, 7 May 2015 13:56:25 UTC+2, Arkadiy Shinkarev wrote:
>>>>
>>>> Hi!
>>>>
>>>> I'm trying to send messages to raw udp input in Graylog 1.0.2.
>>>> The message size is 1k-20k, but Graylog only shown first 8k of message.
>>>>
>>>> I have configured "recv_buffer_size: 10485760" for input, also 
>>>> set net.core.rmem_max = 26214400 in sysctl.conf.
>>>>
>>>> When I run tcpdump, I see that message len is ok (>8k).
>>>> When I run strace -e trace=network i see the following:
>>>> [pid 10539] recvfrom(365, 
>>>> "\24\21\0\0\370\370\352\1778q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>>>  
>>>> 8192, 0, NULL, NULL) = 4372
>>>> [pid 10539] recvfrom(365, 
>>>> "\24\21\0\0\371\370\352\1779q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>>>  
>>>> 8192, 0, NULL, NULL) = 4372
>>>> [pid 10537] <... recvfrom resumed> 
>>>> "\24\21\0

[graylog2] Re: Raw UDP buffer size

2015-05-07 Thread Arkadiy Shinkarev

Jochen, thanks for you reply!

As I mentioned in my first post, I can see with tcpdump that packet lenght, 
that comes to Graylog node is more than 8192 bytes:
$ sudo tcpdump -n -i tunl0 port 12500 and udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tunl0, link-type RAW (Raw IP), capture size 65535 bytes
18:22:19.062304 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 54
18:22:19.079891 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81
18:22:19.113119 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 92
18:22:19.117398 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 93
18:22:19.121636 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81
18:22:19.123707 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 94
18:22:22.092734 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108
18:22:22.093300 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 70
18:22:22.238882 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 71
18:22:24.067068 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 87
18:22:26.148394 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 85
18:22:27.477703 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 13642
18:22:31.158020 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 91
18:22:35.945376 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 69
18:22:35.945489 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 91
18:22:37.279499 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108

Also, I have another node with Graylog2 0.20, messages from same sources 
comes to logstash UDP input without problems.

So, I don't thinks that it is network related problems.

On Thursday, May 7, 2015 at 4:04:26 PM UTC+3, Jochen Schalanda wrote:
>
> Hi Arkadiy,
>
> not all network devices support UDP packets bigger than 8KiB (8192 bytes) 
> and this seems to be the case on your network somewhere. To circumvent this 
> restriction, Graylog or more specifically the GELF format supports chunking 
> which means splitting a large message into multiple UDP packets of a 
> certain size (see https://www.graylog.org/resources/gelf for details on 
> the GELF format). So in order to solve your problem you could collect the 
> logs on the same machine which produces those raw log lines with a log 
> shipper like nxlog or logstash and send them to Graylog via GELF. 
> Alternatively you'll have to switch from UDP to TCP which doesn't suffer 
> this kind of restriction.
>
> Cheers,
> Jochen
>
> On Thursday, 7 May 2015 13:56:25 UTC+2, Arkadiy Shinkarev wrote:
>>
>> Hi!
>>
>> I'm trying to send messages to raw udp input in Graylog 1.0.2.
>> The message size is 1k-20k, but Graylog only shown first 8k of message.
>>
>> I have configured "recv_buffer_size: 10485760" for input, also 
>> set net.core.rmem_max = 26214400 in sysctl.conf.
>>
>> When I run tcpdump, I see that message len is ok (>8k).
>> When I run strace -e trace=network i see the following:
>> [pid 10539] recvfrom(365, 
>> "\24\21\0\0\370\370\352\1778q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10539] recvfrom(365, 
>> "\24\21\0\0\371\370\352\1779q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10537] <... recvfrom resumed> 
>> "\24\21\0\0O\372\352\177\235q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10537] recvfrom(365, 
>> "\24\21\0\0X\372\352\177\236q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10541] recvfrom(365, 
>> "\24\21\0\0\200\372\352\177\240q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10541] recvfrom(365, 
>> "\24\21\0\0\201\372\352\177\241q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10534] recvfrom(365, 
>> "\24\21\0\0\302\372\352\177\330q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10534] recvfrom(365, 
>> "\24\21\0\0\303\372\352\177\331q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>>  
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10540] <... recvfrom resumed> 
>> "\24\21\0\0$\373\352\177-r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
>> 8192, 0, NULL, NULL) = 4372
>> [pid 10540] <... recvfrom resumed> 
>> "\24\21\0\0&\373\352\177/r\1\0\1\0\0\0\10\0\

[graylog2] Raw UDP buffer size

2015-05-07 Thread Arkadiy Shinkarev

Hi!

I'm trying to send messages to raw udp input in Graylog 1.0.2.
The message size is 1k-20k, but Graylog only shown first 8k of message.

I have configured "recv_buffer_size: 10485760" for input, also 
set net.core.rmem_max = 26214400 in sysctl.conf.

When I run tcpdump, I see that message len is ok (>8k).
When I run strace -e trace=network i see the following:
[pid 10539] recvfrom(365, 
"\24\21\0\0\370\370\352\1778q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10539] recvfrom(365, 
"\24\21\0\0\371\370\352\1779q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10537] <... recvfrom resumed> 
"\24\21\0\0O\372\352\177\235q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10537] recvfrom(365, 
"\24\21\0\0X\372\352\177\236q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10541] recvfrom(365, 
"\24\21\0\0\200\372\352\177\240q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10541] recvfrom(365, 
"\24\21\0\0\201\372\352\177\241q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10534] recvfrom(365, 
"\24\21\0\0\302\372\352\177\330q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10534] recvfrom(365, 
"\24\21\0\0\303\372\352\177\331q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10540] <... recvfrom resumed> 
"\24\21\0\0$\373\352\177-r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10540] <... recvfrom resumed> 
"\24\21\0\0&\373\352\177/r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10535] recvfrom(362, 
"\24\21\0\0b\373\352\177jr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10535] recvfrom(362, 
"\24\21\0\0c\373\352\177kr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10537] recvfrom(362, 
"\24\21\0\0\315\375\352\177_s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10537] recvfrom(362, 
"\24\21\0\0\316\375\352\177`s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 
8192, 0, NULL, NULL) = 4372
[pid 10538] recvfrom(362, 
"\24\21\0\0\v\376\352\177\224s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10538] recvfrom(362, 
"\24\21\0\0\f\376\352\177\225s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10534] <... recvfrom resumed> 
"\24\21\0\0\234\376\352\177\30t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372
[pid 10534] <... recvfrom resumed> 
"\24\21\0\0\237\376\352\177\33t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
 
8192, 0, NULL, NULL) = 4372

I'm also test GELF UDP input, messages comes from logstash (to logstash 
messages comes from UDP input with 32k buffer size), there is no problem - 
message looks good, Graylog shows full message.

Where is the problem?

Some additional information:
OS: CentOS release 6.5 (Final)
Kernel: 2.6.32-431.29.2.el6.centos.plus.x86_64
Graylog: 1.0.2

2 graylog-server nodes behind load balancer (LVS) + 2 nodes elasticsearch 
cluster.

Thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Timezone in Graylog2

2014-11-07 Thread Arkadiy Shinkarev

Hey guys, any things?
Russian Graylog2 users, do you have same problem?


On Wednesday, November 5, 2014 6:53:53 PM UTC+3, Arkadiy Shinkarev wrote:
>
> Correct: 
>
> On Wednesday, November 5, 2014 6:53:01 PM UTC+3, Arkadiy Shinkarev wrote:
>>
>> I see wrong timestamps in web interface:
>>
>>
>> <https://lh3.googleusercontent.com/-NK86lshBpVA/VFpHDxzh09I/COM/mI3AoLsjnk8/s1600/tz2.PNG>
>>
>>
>> <https://lh6.googleusercontent.com/-UkNXcjCmSxg/VFpHBW9K8eI/COE/__ioSFg5jT8/s1600/tz1.PNG>
>>
>>
>> Also, when i preform search I see wrong data (+1 hour), Graylog2 thinks 
>> that my timezone is UTC+4, but actually it is UTC+3.
>>
>> Timezone for GL user set to Europe/Moscow, tzdata-java installed on all 
>> servers (2x graylog2-server + 1x graylog2-web).
>>
>>
>>
>> On Wednesday, November 5, 2014 5:19:58 PM UTC+3, Edmundo Alvarez wrote:
>>>
>>> Could you please explain a little bit more about the problem you are 
>>> experiencing? I mean, are you just seeing wrong timestamps in the web 
>>> interface or is there something wrong when you perform searches as well? 
>>>
>>> Edmundo 
>>>
>>> -- 
>>> Developer 
>>>
>>> Tel.: +49 (0)40 609 452 077 
>>> Mobile: +49 (0)171 27 22 181 
>>> Mobile (US): +1 (713) 321 8126 
>>> Fax.: +49 (0)40 609 452 078 
>>>
>>> TORCH GmbH 
>>> Steckelhörn 11 
>>> 20457 Hamburg 
>>> Germany 
>>> https://www.torch.sh/ 
>>>
>>> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
>>> Geschäftsführer: Lennart Koopmann (CEO) 
>>>
>>> > On 05 Nov 2014, at 14:13, Shinkarev Arkadiy  wrote: 
>>> > 
>>> > Hi Edmundo, 
>>> > I'm already on 0.91.3, but it doesn't help. 
>>> > 
>>> > -- 
>>> > Sincerely, 
>>> > Arkadiy  Shinkarev 
>>> > e-mail: kew...@gmail.com 
>>> > Cell.:   +7 (926) 147-51-87 
>>> > 
>>> > 2014-11-05 16:10 GMT+03:00 Edmundo Alvarez : 
>>> > Hello, 
>>> > 
>>> > We have released Graylog2 0.90.3 and 0.91.3 that should fix some 
>>> issues with timezones and DST in the web interface. Could you please try 
>>> with one of those versions and let us know if that helped? 
>>> > 
>>> > Regards, 
>>> > 
>>> > Edmundo 
>>> > 
>>> > -- 
>>> > Developer 
>>> > 
>>> > Tel.: +49 (0)40 609 452 077 
>>> > Mobile: +49 (0)171 27 22 181 
>>> > Mobile (US): +1 (713) 321 8126 
>>> > Fax.: +49 (0)40 609 452 078 
>>> > 
>>> > TORCH GmbH 
>>> > Steckelhörn 11 
>>> > 20457 Hamburg 
>>> > Germany 
>>> > https://www.torch.sh/ 
>>> > 
>>> > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
>>> > Geschäftsführer: Lennart Koopmann (CEO) 
>>> > 
>>> > > On 05 Nov 2014, at 13:14, Arkadiy Shinkarev  
>>> wrote: 
>>> > > 
>>> > > Hi! 
>>> > > The same issue for web&server applications :( 
>>> > > CentOS 6.5 
>>> > > 
>>> > > # ls -la /etc/alternatives/jre/lib/zi 
>>> > > lrwxrwxrwx 1 root root 17 Nov  5 14:42 /etc/alternatives/jre/lib/zi 
>>> -> /usr/share/javazi 
>>> > > 
>>> > > # yum info tzdata-java 
>>> > > Loaded plugins: fastestmirror, security 
>>> > > Loading mirror speeds from cached hostfile 
>>> > > Installed Packages 
>>> > > Name: tzdata-java 
>>> > > Arch: noarch 
>>> > > Version : 2014i 
>>> > > Release : 1.el6 
>>> > > Size: 358 k 
>>> > > Repo: installed 
>>> > > From repo   : tcs-centos-6-updates-x86_64 
>>> > > Summary : Timezone data for Java 
>>> > > URL : https://www.iana.org/time-zones 
>>> > > License : Public Domain 
>>> > > Description : This package contains timezone information for use by 
>>> Java runtimes. 
>>> > > 
>>> > > 
>>> > > 
>>> > > On Thursday, October 30, 2014 2:33:02 PM UTC+3, Petr Sukharev wrote: 
>>> > > Hello! 
>>> > > I have some trouble with my graylog instance and incorrect time

Re: [graylog2] Timezone in Graylog2

2014-11-05 Thread Arkadiy Shinkarev

Correct: 

On Wednesday, November 5, 2014 6:53:01 PM UTC+3, Arkadiy Shinkarev wrote:
>
> I see wrong timestamps in web interface:
>
>
> <https://lh3.googleusercontent.com/-NK86lshBpVA/VFpHDxzh09I/COM/mI3AoLsjnk8/s1600/tz2.PNG>
>
>
> <https://lh6.googleusercontent.com/-UkNXcjCmSxg/VFpHBW9K8eI/COE/__ioSFg5jT8/s1600/tz1.PNG>
>
>
> Also, when i preform search I see wrong data (+1 hour), Graylog2 thinks 
> that my timezone is UTC+4, but actually it is UTC+3.
>
> Timezone for GL user set to Europe/Moscow, tzdata-java installed on all 
> servers (2x graylog2-server + 1x graylog2-web).
>
>
>
> On Wednesday, November 5, 2014 5:19:58 PM UTC+3, Edmundo Alvarez wrote:
>>
>> Could you please explain a little bit more about the problem you are 
>> experiencing? I mean, are you just seeing wrong timestamps in the web 
>> interface or is there something wrong when you perform searches as well? 
>>
>> Edmundo 
>>
>> -- 
>> Developer 
>>
>> Tel.: +49 (0)40 609 452 077 
>> Mobile: +49 (0)171 27 22 181 
>> Mobile (US): +1 (713) 321 8126 
>> Fax.: +49 (0)40 609 452 078 
>>
>> TORCH GmbH 
>> Steckelhörn 11 
>> 20457 Hamburg 
>> Germany 
>> https://www.torch.sh/ 
>>
>> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
>> Geschäftsführer: Lennart Koopmann (CEO) 
>>
>> > On 05 Nov 2014, at 14:13, Shinkarev Arkadiy  wrote: 
>> > 
>> > Hi Edmundo, 
>> > I'm already on 0.91.3, but it doesn't help. 
>> > 
>> > -- 
>> > Sincerely, 
>> > Arkadiy  Shinkarev 
>> > e-mail: kew...@gmail.com 
>> > Cell.:   +7 (926) 147-51-87 
>> > 
>> > 2014-11-05 16:10 GMT+03:00 Edmundo Alvarez : 
>> > Hello, 
>> > 
>> > We have released Graylog2 0.90.3 and 0.91.3 that should fix some issues 
>> with timezones and DST in the web interface. Could you please try with one 
>> of those versions and let us know if that helped? 
>> > 
>> > Regards, 
>> > 
>> > Edmundo 
>> > 
>> > -- 
>> > Developer 
>> > 
>> > Tel.: +49 (0)40 609 452 077 
>> > Mobile: +49 (0)171 27 22 181 
>> > Mobile (US): +1 (713) 321 8126 
>> > Fax.: +49 (0)40 609 452 078 
>> > 
>> > TORCH GmbH 
>> > Steckelhörn 11 
>> > 20457 Hamburg 
>> > Germany 
>> > https://www.torch.sh/ 
>> > 
>> > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
>> > Geschäftsführer: Lennart Koopmann (CEO) 
>> > 
>> > > On 05 Nov 2014, at 13:14, Arkadiy Shinkarev  
>> wrote: 
>> > > 
>> > > Hi! 
>> > > The same issue for web&server applications :( 
>> > > CentOS 6.5 
>> > > 
>> > > # ls -la /etc/alternatives/jre/lib/zi 
>> > > lrwxrwxrwx 1 root root 17 Nov  5 14:42 /etc/alternatives/jre/lib/zi 
>> -> /usr/share/javazi 
>> > > 
>> > > # yum info tzdata-java 
>> > > Loaded plugins: fastestmirror, security 
>> > > Loading mirror speeds from cached hostfile 
>> > > Installed Packages 
>> > > Name: tzdata-java 
>> > > Arch: noarch 
>> > > Version : 2014i 
>> > > Release : 1.el6 
>> > > Size: 358 k 
>> > > Repo: installed 
>> > > From repo   : tcs-centos-6-updates-x86_64 
>> > > Summary : Timezone data for Java 
>> > > URL : https://www.iana.org/time-zones 
>> > > License : Public Domain 
>> > > Description : This package contains timezone information for use by 
>> Java runtimes. 
>> > > 
>> > > 
>> > > 
>> > > On Thursday, October 30, 2014 2:33:02 PM UTC+3, Petr Sukharev wrote: 
>> > > Hello! 
>> > > I have some trouble with my graylog instance and incorrect time in 
>> Timestamp field 
>> > > This happened after updating timezone (i am from Russia and we had 
>> time-change-issue here at 26 Oct) 
>> > > Current system time is Thu Oct 30 14:28:03 MSK 2014 for example. 
>> > > Version is 0.91.1. I install tzupdater for java (version 2014i ) and 
>> no luck here. 
>> > > Here is date from Graylog 
>> > > 
>> > > User admin: 
>> > > 2014-10-30 15:28:26.641 +04:00 
>> > > Web browser: 
>> > > 2014-10-30 14:28:26.902 +03:00 
>> > > Default

Re: [graylog2] Timezone in Graylog2

2014-11-05 Thread Arkadiy Shinkarev

I see wrong timestamps in web interface:

<https://lh3.googleusercontent.com/-NK86lshBpVA/VFpHDxzh09I/COM/mI3AoLsjnk8/s1600/tz2.PNG>

<https://lh6.googleusercontent.com/-UkNXcjCmSxg/VFpHBW9K8eI/COE/__ioSFg5jT8/s1600/tz1.PNG>


Also, when i preform search I see wrong data (+1 hour), Graylog2 thinks 
that my timezone is UTS+3, but actually it is UTC+3.

Timezone for GL user set to Europe/Moscow, tzdata-java installed on all 
servers (2x graylog2-server + 1x graylog2-web).



On Wednesday, November 5, 2014 5:19:58 PM UTC+3, Edmundo Alvarez wrote:
>
> Could you please explain a little bit more about the problem you are 
> experiencing? I mean, are you just seeing wrong timestamps in the web 
> interface or is there something wrong when you perform searches as well? 
>
> Edmundo 
>
> -- 
> Developer 
>
> Tel.: +49 (0)40 609 452 077 
> Mobile: +49 (0)171 27 22 181 
> Mobile (US): +1 (713) 321 8126 
> Fax.: +49 (0)40 609 452 078 
>
> TORCH GmbH 
> Steckelhörn 11 
> 20457 Hamburg 
> Germany 
> https://www.torch.sh/ 
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
> Geschäftsführer: Lennart Koopmann (CEO) 
>
> > On 05 Nov 2014, at 14:13, Shinkarev Arkadiy  > wrote: 
> > 
> > Hi Edmundo, 
> > I'm already on 0.91.3, but it doesn't help. 
> > 
> > -- 
> > Sincerely, 
> > Arkadiy  Shinkarev 
> > e-mail: kew...@gmail.com  
> > Cell.:   +7 (926) 147-51-87 
> > 
> > 2014-11-05 16:10 GMT+03:00 Edmundo Alvarez : 
> > Hello, 
> > 
> > We have released Graylog2 0.90.3 and 0.91.3 that should fix some issues 
> with timezones and DST in the web interface. Could you please try with one 
> of those versions and let us know if that helped? 
> > 
> > Regards, 
> > 
> > Edmundo 
> > 
> > -- 
> > Developer 
> > 
> > Tel.: +49 (0)40 609 452 077 
> > Mobile: +49 (0)171 27 22 181 
> > Mobile (US): +1 (713) 321 8126 
> > Fax.: +49 (0)40 609 452 078 
> > 
> > TORCH GmbH 
> > Steckelhörn 11 
> > 20457 Hamburg 
> > Germany 
> > https://www.torch.sh/ 
> > 
> > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 
> > Geschäftsführer: Lennart Koopmann (CEO) 
> > 
> > > On 05 Nov 2014, at 13:14, Arkadiy Shinkarev  > wrote: 
> > > 
> > > Hi! 
> > > The same issue for web&server applications :( 
> > > CentOS 6.5 
> > > 
> > > # ls -la /etc/alternatives/jre/lib/zi 
> > > lrwxrwxrwx 1 root root 17 Nov  5 14:42 /etc/alternatives/jre/lib/zi -> 
> /usr/share/javazi 
> > > 
> > > # yum info tzdata-java 
> > > Loaded plugins: fastestmirror, security 
> > > Loading mirror speeds from cached hostfile 
> > > Installed Packages 
> > > Name: tzdata-java 
> > > Arch: noarch 
> > > Version : 2014i 
> > > Release : 1.el6 
> > > Size: 358 k 
> > > Repo: installed 
> > > From repo   : tcs-centos-6-updates-x86_64 
> > > Summary : Timezone data for Java 
> > > URL : https://www.iana.org/time-zones 
> > > License : Public Domain 
> > > Description : This package contains timezone information for use by 
> Java runtimes. 
> > > 
> > > 
> > > 
> > > On Thursday, October 30, 2014 2:33:02 PM UTC+3, Petr Sukharev wrote: 
> > > Hello! 
> > > I have some trouble with my graylog instance and incorrect time in 
> Timestamp field 
> > > This happened after updating timezone (i am from Russia and we had 
> time-change-issue here at 26 Oct) 
> > > Current system time is Thu Oct 30 14:28:03 MSK 2014 for example. 
> > > Version is 0.91.1. I install tzupdater for java (version 2014i ) and 
> no luck here. 
> > > Here is date from Graylog 
> > > 
> > > User admin: 
> > > 2014-10-30 15:28:26.641 +04:00 
> > > Web browser: 
> > > 2014-10-30 14:28:26.902 +03:00 
> > > Default JDK/JRE: 
> > > 2014-10-30 15:28:26.641 +04:00 
> > > 
> > > 
> > > Graylog2 web interface: 
> > > 2014-10-30 15:28:26.641 +04:00 
> > > Graylog2 master server: 
> > > 2014-10-30 15:28:26.642 +04:00 
> > > 
> > > 
> > > 
> > > -- 
> > > You received this message because you are subscribed to the Google 
> Groups "graylog2" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> an email to graylog2+u...@googlegroups.com . 
> > > For

[graylog2] Re: Timezone in Graylog2

2014-11-05 Thread Arkadiy Shinkarev

Hi!
The same issue for web&server applications :(
CentOS 6.5

# ls -la /etc/alternatives/jre/lib/zi
lrwxrwxrwx 1 root root 17 Nov  5 14:42 /etc/alternatives/jre/lib/zi -> 
/usr/share/javazi

# yum info tzdata-java
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
Installed Packages
Name: tzdata-java
Arch: noarch
Version : 2014i
Release : 1.el6
Size: 358 k
Repo: installed
>From repo   : tcs-centos-6-updates-x86_64
Summary : Timezone data for Java
URL : https://www.iana.org/time-zones
License : Public Domain
Description : This package contains timezone information for use by Java 
runtimes.



On Thursday, October 30, 2014 2:33:02 PM UTC+3, Petr Sukharev wrote:
>
> Hello!
> I have some trouble with my graylog instance and incorrect time 
> in Timestamp field
> This happened after updating timezone (i am from Russia and we had 
> time-change-issue here at 26 Oct)
> Current system time is Thu Oct 30 14:28:03 MSK 2014 for example.
> Version is 0.91.1. I install tzupdater for java (version 2014i ) and no 
> luck here.
> Here is date from Graylog
>
> User admin:2014-10-30 15:28:26.641 +04:00Web browser:2014-10-30 
> 14:28:26.902 +03:00Default JDK/JRE:2014-10-30 15:28:26.641 +04:00
>
>
> Graylog2 web interface:2014-10-30 15:28:26.641 +04:00Graylog2 master 
> server:2014-10-30 15:28:26.642 +04:00
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: UTF-8, syslog input

2014-07-30 Thread Arkadiy Shinkarev

Hi! It's a pretty easy solution, there is a some details.
1. You need a logstash - download it from http://logstash.net, or find a 
package for your OS
2. Next, create include in logstash's conf.d directory (in my case, 
/etc/logstash/conf.d/some.conf), you can use something like this:
$ cat /etc/logstash/conf.d/sitecore.conf
input {
udp {
charset => "CP1252" # this is your input logs encoding
port => 20514 # logstash will listen this UDP port
buffer_size => 32768 # udp buffer size, e. g. i receive 
long XMLs
}
}

# next block - some grok magic, read man @logstash.net
filter {
grok {
match => [ "message", "%{LOGLEVEL:severity} 
\[(?.*)\] %{DOTNEWLINE:message}" ]
overwrite => [ "message" ]
patterns_dir => "/etc/logstash/patterns"
}
}

# and finally, send it to graylog2 gelf input (you may also specify port, 
etc)
output {
gelf {
host => localhost
}

}

I use this config for IIS application logs that sends by log4net, maybe you 
need to customise it.

3. Run GELF input in logstash
4. Run logstash

On Wednesday, July 30, 2014 5:17:46 PM UTC+4, ellyas ellyas wrote:
>
> Please explain in detail what is the solution? I have similiar problem. I 
> pick up logs from win7x64rus. Russian symbols in string looks like that
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: Having Issues configuring and send data to my newly deployed graylog2

2014-05-30 Thread Arkadiy Shinkarev

You don't need GELF TCP input, 'cause logstash sends GELF over UDP.
_discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] {:level=>:debug, 
:file=>"filewatch/watch.rb", :line=>"117"}

The message "glob is: []" in most cases means that logstash process does 
not have permissions to read file.


On Thursday, May 29, 2014 5:48:02 PM UTC+4, Joseph DJOMEDA wrote:
>
> Hi Thanks for the feedback and your effort.
>
> Did you explicitly created a tcp gelf input?
>
>
> On Thursday, May 29, 2014 12:26:26 PM UTC, Dmitri Stoljarov wrote:
>>
>> Joseph,
>>
>> You cannot telnet to UDP port (your netstat shows only udp port).
>>
>> Here's my output:
>>
>> $ netstat -ano | grep 12201 |grep -v ESTAB
>> tcp0  0 0.0.0.0:12201   
>> 0.0.0.0:*   LISTEN  off (0.00/0/0)
>> udp0  0 0.0.0.0:12201   
>> 0.0.0.0:*   off (0.00/0/0)
>>
>>
>>
>> On Thursday, May 29, 2014 2:28:47 PM UTC+3, Joseph DJOMEDA wrote:
>>>
>>> Hello Arkadiy ,
>>>
>>> thanks for your effort my answers *inline*
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: [ANNOUNCE] Graylog2 v0.20.2 has been released

2014-05-26 Thread Arkadiy Shinkarev

Great news!
Waiting for RPM :)

On Friday, May 23, 2014 11:01:21 PM UTC+4, lennart wrote:
>
> Hey everybody, 
>
> the final release of Graylog2 v0.20.2 has arrived: 
> http://blog.graylog2.org/graylog2-v0-20-2-has-been-released/ 
>
> A big thank you to the TORCH team. We put a lot of effort into this 
> release and will follow up with a v0.21.0 that brings ElasticSearch 
> v1.x support. 
>
> Thank you very much, 
> Lennart 
>

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Having Issues configuring and send data to my newly deployed graylog2

2014-05-15 Thread Arkadiy Shinkarev

You have an error in grok pattern, try this one:
%{DATESTAMP:datestamp} %{LOGLEVEL:loglevel} \[%{GREEDYDATA:thread}\] 
\[%{GREEDYDATA:classinfo}\] %{GREEDYDATA:loginfo}

Later, you can use Grok Debugger - http://grokdebug.herokuapp.com/


On Saturday, April 19, 2014 4:19:57 PM UTC+4, Joseph DJOMEDA wrote:
>
> Hello Good People,
>
> I am coming from splunk background with even little experience on it. But 
> I am having issue getting basic stuff done. I have graylog2 server and web 
> interface running fine let's say on IP :112. I have a java application 
> running on a server IP : 27. the log of the app is of the type shown below. 
> I know it needs some cleanups but I am more concerned about sending 
> something to graylog2:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2014-04-01 21:54:17,398 INFO [Thread-2] 
> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - 
> HHH06: Autocommit mode: true2014-04-01 21:54:17,399 WARN [Thread-2] 
> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - 
> HHH000148: No JDBC Driver class was specified by property 
> hibernate.connection.driver_class2014-04-01 21:54:17,425 INFO [Thread-2] 
> [com.mchange.v2.log.MLog] - MLog clients using log4j logging.2014-04-01 
> 21:54:17,545 INFO [Thread-2] [com.mchange.v2.c3p0.C3P0Registry] - 
> Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; 
> trace: 10]2014-04-01 21:54:17,930 INFO [Thread-2] 
> [com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource] - Initializing c3p0 
> pool... com.mchange.v2.c3p0.PoolBackedDataSource@d678e16f [ 
> connectionPoolDataSource -> 
> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@7bb4a24 [ 
> acquireIncrement -> 2, acquireRetryAttempts -> 30, acquireRetryDelay -> 
> 1000, autoCommitOnClose -> false, automaticTestTable -> null, 
> breakAfterAcquireFailure -> false, checkoutTimeout -> 0, 
> connectionCustomizerClassName -> null, connectionTesterClassName -> 
> com.mchange.v2.c3p0.impl.DefaultConnectionTester, 
> debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> 
> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 
> nm1r17918k7ta81op67fy|71b3cc1f, idleConnectionTestPeriod -> 300, 
> initialPoolSize -> 3, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 
> 0, maxIdleTime -> 6, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 
> 40, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 2, 
> nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@2ba1b5de [ 
> description -> null, driverClass -> null, factoryClassLocation -> null, 
> identityToken -> nm1r17918k7ta81op67fy|67f095ba, jdbcUrl -> 
> jdbc:mysql://localhost:3306/do_my_app, properties -> {user=**, 
> password=**, autocommit=true, driverClassName=com.mysql.jdbc.Driver, 
> release_mode=auto} ], preferredTestQuery -> null, propertyCycle -> 0, 
> testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, 
> unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> 
> false; userOverrides: {} ], dataSourceName -> null, factoryClassLocation -> 
> null, identityToken -> nm1r17918k7ta81op67fy|51af67f9, numHelperThreads -> 
> 3 ]2014-04-01 21:54:18,453 INFO [Thread-2] [org.hibernate.dialect.Dialect] 
> - HHH000400: Using dialect: org.hibernate.dialect.MySQLDialect *
>
> I got logstash on the server IP: 27 and have its configuration shown below:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *#logstash for IP 27 server# logstash.conf fileinput {  file {type => 
> my_apppath => ["/opt/tomcatinstances/my_app/logs/catalina.out"]  
> }} filter { grok {   match => {"message" => %{DATESTAMP_EVENTLOG:datestamp} 
> %{LOGLEVEL:loglevel} \[%{WORD:thread}\] \[%{GREEDYDATA:classinfo}\] 
> %{WORD:loginfo}"} } } output { gelf { host => "xxx.xxx.xxx.112" facility => 
> "%{@type}" custom_fields => ["environment", "production"] }}*
>
> when I run bin/logstash --debug -f logstasb.conf I have bunch of the 
> following
>
>
>
>
>
>
>
> *=>"my_server_name", 
> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", 
> "tags"=>["_grokparsefailure"]}, "tags"]}>, @data={"message"=>"
> voucher_type vouchertyp0_", "@version"=>"1", 
> "@timestamp"=>"2014-04-19T10:55:17.333Z", "type"=>"my_app", 
> "host"=>"my_server_name", 
> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", 
> "tags"=>["_grokparsefailure"]}, @cancelled=false>, :level=>:debug, 
> :file=>"logstash/filters/grok.rb", :line=>"310"}["Sending GELF event", 
> {"short_message"=>"from", "full_message"=>"from", 
> "host"=>"my_server_name", "facility"=>"%{@type}", "_type"=>"my_app", 
> "_path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", 
> "_tags"=>"_grokparsefailure", "_environment"=>"production", "level"=>6}] 
> {:level=>:debug, :file=>"logstash/outputs/gelf.rb", :line=>"203"}output 
> received {:event=>{"message"=>"voucher_type vouchertyp0_", 
> "@version"=>"1", "@timestamp"=>"2014-04-19T10:55:17.333Z", 
> "type"=>"m

[graylog2] Re: UTF-8, syslog input

2014-05-15 Thread Arkadiy Shinkarev

Ok, I solved it.
logs encoding was CP1252, so I imlement Logstash and now everything works 
fine :)

On Monday, May 12, 2014 12:30:00 PM UTC+4, Arkadiy Shinkarev wrote:
>
> Hello all!
>
> We have several log4net instances pointed to syslog and send logs via UDP.
> I forward all from syslog to Graylog2:
> $template GRAYLOGRFC5424,"<%pri%>%protocol-version% 
> %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
> *.* @127.0.0.1:514;GRAYLOGRFC5424
>
> When data comes to Graylog2 i see this:
>
>
> <https://lh4.googleusercontent.com/-QiDlAA8NIKI/U3CEEW0iEII/BOU/KoUYMWGbhqg/s1600/%D0%A1%D0%BD%D0%B8%D0%BC%D0%BE%D0%BA.PNG>
>
> Russian text in UTF-8 also broken, but in /var/log/messages all logs looks 
> ok.
> Where is the problem?
>
> rsyslog 8.2.1
> elasticsearch 0.90.13
> Graylog2 0.20.1
>
> Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] UTF-8, syslog input

2014-05-12 Thread Arkadiy Shinkarev

Hello all!

We have several log4net instances pointed to syslog and send logs via UDP.
I forward all from syslog to Graylog2:
$template GRAYLOGRFC5424,"<%pri%>%protocol-version% 
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
*.* @127.0.0.1:514;GRAYLOGRFC5424

When data comes to Graylog2 i see this:



Russian text in UTF-8 also broken, but in /var/log/messages all logs looks 
ok.
Where is the problem?

rsyslog 8.2.1
elasticsearch 0.90.13
Graylog2 0.20.1

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Search issues after update to 1.2

[graylog2] Re: Search issues after update to 1.2

[graylog2] Re: Search issues after update to 1.2

[graylog2] Re: Search issues after update to 1.2

[graylog2] Search issues after update to 1.2

[graylog2] Re: Raw UDP buffer size

[graylog2] Re: Raw UDP buffer size

[graylog2] Raw UDP buffer size

Re: [graylog2] Timezone in Graylog2

Re: [graylog2] Timezone in Graylog2

Re: [graylog2] Timezone in Graylog2

[graylog2] Re: Timezone in Graylog2

[graylog2] Re: UTF-8, syslog input

Re: [graylog2] Re: Having Issues configuring and send data to my newly deployed graylog2

[graylog2] Re: [ANNOUNCE] Graylog2 v0.20.2 has been released

[graylog2] Re: Having Issues configuring and send data to my newly deployed graylog2

[graylog2] Re: UTF-8, syslog input

[graylog2] UTF-8, syslog input

18 matches

Site Navigation

Mail list logo

Footer information