Re: [Kibana] group by request?
I'm not sure whether I understand your issue in full depth but you can use nested aggregations to have hierarchical grouping in Kibana 4. Maybe this solves your issue? Am Montag, 22. Dezember 2014 09:58:57 UTC+1 schrieb stephanos: > > Thanks for the answer! > I think wasn't clear enough: all our log messages already have a > requestID. So if there *was* a grouping feature we'd apply it to that > field. > > I'm just wondering, how do you troubleshoot a issue of a user? When we see > a problem we look at all requests of that user in the GAE log viewer. Then > you quickly see requests that have non-200 status codes. Then we drill into > a request and see all logs of *that* request chronologically. While in > Kibana I can also look at all logs from a user ordered by time, but it's > not always completely clear which request log messages belong to. It's more > like one big stream. > > My point is, you should really try out the Google App Engine log viewer - > then you would know what you are missing! :) > > Stephan > > > On Monday, December 22, 2014 7:38:26 AM UTC+1, Magnus Bäck wrote: >> >> On Tuesday, December 16, 2014 at 10:03 CET, >> stephanos wrote: >> >> > we are using Google App Engine to host our SaaS app. Google offers a >> > nice log browser but it is way too slw. So one of my colleagues >> > suggested we pipe our logs to logstash and make them accessible via >> > Kibana. So far so good, we managed to set everything up. >> > But when Kibana was shown to the other team members they weren't >> > really excited. It was much faster, yes. It allowed to make better >> > queries, yes. BUT it broke the pattern they knew from the Google App >> > Engine log browser: >> > /some-request >> > log message 1 >> > log message 2 >> > /another-request >> > log message 3 >> > /yet-another-request >> > log message 4 >> > While Kibana works like this: >> > log message 1/some-request >> > log message 2/some-request >> > log message 3/another-request >> > log message 4/yet-another-request >> > So basically App Engine groups log messages by request. To get my >> > team on board, can we make Kibana do the same? >> >> Not out of the box, no. Kibana doesn't have any such contextual >> understanding of messages and currently can't be configured as >> such either. >> >> -- >> Magnus Bäck| Software Engineer, Development Tools >> magnu...@sonymobile.com | Sony Mobile Communications >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0513eb37-5742-46c8-b7c6-fd56f609d0e4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Kibana] group by request?
Hi, Maybe graylog2 can interst you as a solution to store your data in ES, and therefore you have better searching of your data. Especially data coming form a webserver as I understand. You then can stil search or display data with Kibana. A. Op maandag 22 december 2014 09:58:57 UTC+1 schreef stephanos: > > Thanks for the answer! > I think wasn't clear enough: all our log messages already have a > requestID. So if there *was* a grouping feature we'd apply it to that > field. > > I'm just wondering, how do you troubleshoot a issue of a user? When we see > a problem we look at all requests of that user in the GAE log viewer. Then > you quickly see requests that have non-200 status codes. Then we drill into > a request and see all logs of *that* request chronologically. While in > Kibana I can also look at all logs from a user ordered by time, but it's > not always completely clear which request log messages belong to. It's more > like one big stream. > > My point is, you should really try out the Google App Engine log viewer - > then you would know what you are missing! :) > > Stephan > > > On Monday, December 22, 2014 7:38:26 AM UTC+1, Magnus Bäck wrote: >> >> On Tuesday, December 16, 2014 at 10:03 CET, >> stephanos wrote: >> >> > we are using Google App Engine to host our SaaS app. Google offers a >> > nice log browser but it is way too slw. So one of my colleagues >> > suggested we pipe our logs to logstash and make them accessible via >> > Kibana. So far so good, we managed to set everything up. >> > But when Kibana was shown to the other team members they weren't >> > really excited. It was much faster, yes. It allowed to make better >> > queries, yes. BUT it broke the pattern they knew from the Google App >> > Engine log browser: >> > /some-request >> > log message 1 >> > log message 2 >> > /another-request >> > log message 3 >> > /yet-another-request >> > log message 4 >> > While Kibana works like this: >> > log message 1/some-request >> > log message 2/some-request >> > log message 3/another-request >> > log message 4/yet-another-request >> > So basically App Engine groups log messages by request. To get my >> > team on board, can we make Kibana do the same? >> >> Not out of the box, no. Kibana doesn't have any such contextual >> understanding of messages and currently can't be configured as >> such either. >> >> -- >> Magnus Bäck| Software Engineer, Development Tools >> magnu...@sonymobile.com | Sony Mobile Communications >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cfe61a16-63ad-46a3-9747-2d4a73815627%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Kibana] group by request?
Thanks for the answer! I think wasn't clear enough: all our log messages already have a requestID. So if there *was* a grouping feature we'd apply it to that field. I'm just wondering, how do you troubleshoot a issue of a user? When we see a problem we look at all requests of that user in the GAE log viewer. Then you quickly see requests that have non-200 status codes. Then we drill into a request and see all logs of *that* request chronologically. While in Kibana I can also look at all logs from a user ordered by time, but it's not always completely clear which request log messages belong to. It's more like one big stream. My point is, you should really try out the Google App Engine log viewer - then you would know what you are missing! :) Stephan On Monday, December 22, 2014 7:38:26 AM UTC+1, Magnus Bäck wrote: > > On Tuesday, December 16, 2014 at 10:03 CET, > stephanos > wrote: > > > we are using Google App Engine to host our SaaS app. Google offers a > > nice log browser but it is way too slw. So one of my colleagues > > suggested we pipe our logs to logstash and make them accessible via > > Kibana. So far so good, we managed to set everything up. > > But when Kibana was shown to the other team members they weren't > > really excited. It was much faster, yes. It allowed to make better > > queries, yes. BUT it broke the pattern they knew from the Google App > > Engine log browser: > > /some-request > > log message 1 > > log message 2 > > /another-request > > log message 3 > > /yet-another-request > > log message 4 > > While Kibana works like this: > > log message 1/some-request > > log message 2/some-request > > log message 3/another-request > > log message 4/yet-another-request > > So basically App Engine groups log messages by request. To get my > > team on board, can we make Kibana do the same? > > Not out of the box, no. Kibana doesn't have any such contextual > understanding of messages and currently can't be configured as > such either. > > -- > Magnus Bäck| Software Engineer, Development Tools > magnu...@sonymobile.com | Sony Mobile Communications > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8cbd90e7-5e12-4cd7-90d6-35f49dc44e1d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Kibana] group by request?
On Tuesday, December 16, 2014 at 10:03 CET, stephanos wrote: > we are using Google App Engine to host our SaaS app. Google offers a > nice log browser but it is way too slw. So one of my colleagues > suggested we pipe our logs to logstash and make them accessible via > Kibana. So far so good, we managed to set everything up. > But when Kibana was shown to the other team members they weren't > really excited. It was much faster, yes. It allowed to make better > queries, yes. BUT it broke the pattern they knew from the Google App > Engine log browser: > /some-request > log message 1 > log message 2 > /another-request > log message 3 > /yet-another-request > log message 4 > While Kibana works like this: > log message 1/some-request > log message 2/some-request > log message 3/another-request > log message 4/yet-another-request > So basically App Engine groups log messages by request. To get my > team on board, can we make Kibana do the same? Not out of the box, no. Kibana doesn't have any such contextual understanding of messages and currently can't be configured as such either. -- Magnus Bäck| Software Engineer, Development Tools magnus.b...@sonymobile.com | Sony Mobile Communications -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20141222063817.GB11963%40seldlx20533.corpusers.net. For more options, visit https://groups.google.com/d/optout.
[Kibana] group by request?
Hey there, we are using Google App Engine to host our SaaS app. Google offers a nice log browser but it is way too slw. So one of my colleagues suggested we pipe our logs to logstash and make them accessible via Kibana. So far so good, we managed to set everything up. But when Kibana was shown to the other team members they weren't really excited. It was much faster, yes. It allowed to make better queries, yes. BUT it broke the pattern they knew from the Google App Engine log browser: /some-request log message 1 log message 2 /another-request log message 3 /yet-another-request log message 4 While Kibana works like this: log message 1/some-request log message 2/some-request log message 3/another-request log message 4/yet-another-request So basically App Engine groups log messages by request. To get my team on board, can we make Kibana do the same? Stephan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aacdaf38-c614-4dbc-b4d8-a81b832dbc31%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
