Re: Marvel not collecting system metrics
Hi, As I am using Docker Elasticsearch is not running during the build process. I have posted my Dockerfile below. https://github.com/damm/dockerfiles/blob/master/elasticsearch/Dockerfile#L7 FWIW my /_node/stats is correct and fully populated. Additionally I have looked over https://groups.google.com/forum/#!msg/elasticsearch/UPBMKvxwwM8/VBaqh0B11tQJand tried applying that as a custom template and deleting my indices and starting over; did not help. Additionally the elasticsearch.yml in that repo is the current one in use. Thanks! On Fri, Feb 14, 2014 at 12:17 AM, joergpra...@gmail.com joergpra...@gmail.com wrote: How did you install Marvel? You need to add Marvel plugin on every node and each node must be restarted. Also take care of the setting marvel.agent.exporter.es.hosts if you use other host/port than localhost:9200 Jörg -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/dhpxaOuoZWI/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEWrMoZPm8bHfkQbq_-uJojAFLQgaexu0UXXjz5M%2B0biQ%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALDJu4XJRFyWWk7mhHcKt2i%3D9-kXf8Ji54LQTLU4ebmAjTZNRA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Marvel not collecting system metrics
Hi Boaz,
fs: {
data: [
{
path: /data/elasticsearch/shared/docker/nodes/0
}
],
Does not appear to be. This field is populated when I run it without
Docker; so is it expecting any particular file to exist like /etc/fstab or
/etc/mtab?
Thanks!
On Fri, Feb 14, 2014 at 7:35 AM, Boaz Leskes b.les...@gmail.com wrote:
Hi scott
next to Binh's question can you check you have the following field in your
nodes stats( GET _node/stats): fs.total.available_in_bytes ?
On Friday, February 14, 2014 1:06:06 PM UTC+1, Binh Ly wrote:
May I ask which OS you are running ES on?
--
You received this message because you are subscribed to a topic in the
Google Groups elasticsearch group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/dhpxaOuoZWI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/466b1bd5-173d-46df-a8e2-39d52f2b8146%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALDJu4U%3D__oqEDGVQQcUGK%2Ba9vvKMu8deJEEtrRRyDO7zW0BCw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Marvel not collecting system metrics
Tony, We appear to be getting the information back in the cgroup currently; so what's provided is fairly good. It would be better obviously to grab them from the cgroup and push it in; but that would be external to the container (unless you mounted the cgroups in the container) So for my case it works; some spots I do get the 'host' data and not the container data but it's good enough for the moment. Scott On Fri, Feb 14, 2014 at 7:41 AM, Tony Su tonysu...@gmail.com wrote: On Friday, February 14, 2014 4:06:06 AM UTC-8, Binh Ly wrote: May I ask which OS you are running ES on? IIRC Docker is a management tool for LXC. So, it does pique the thoughts... Where should marvel be pulling stats in an LXC deployment? It's not fully isolated like other virtualization technologies and access to various Host processes are blocked. I suppose it might be possible to point to statistics derived by Docker, but that probably requires special configuration. Some thoughts, Tony -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/dhpxaOuoZWI/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b409c6f0-1ea3-44f1-bffc-423edc4bb792%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALDJu4XXVEJSdb4zs9sF6%3DgM92UK15K7A%2BUJ%2B8ugBg6Kn_yB_Q%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Marvel not collecting system metrics
Hi,
I've been deploying Elasticsearch with Docker and it's been working great.
So I added the Marvel plugin and I get a ton of these in my Elasticsearch
error log and no system metrics in Marvel. :(
[2014-02-14 02:55:02,753][DEBUG][action.search.type ] [Kamal]
[.marvel-2014.02.14][0], node[_Ofz6YYlTc2hA9ASu3qlRg], [P], s[STARTED]:
Failed to execute [org.elasticsearch.action.search.SearchRequest@53b9f5]
org.elasticsearch.search.SearchParseException: [.marvel-2014.02.14][0]:
query[ConstantScore(BooleanFilter(+no_cache(@timestamp:[1392342894189 TO
1392346502752]) +cache(_type:node_stats) +cache(@timestamp:[139234590
TO 139234656])))],from[-1],size[0]: Parse Failure [Failed to parse
source
[{size:0,query:{filtered:{query:{match_all:{}},filter:{bool:{must:[{range:{@timestamp:{from:1392342894189,to:now}}},{term:{_type:node_stats}},{range:{@timestamp:{from:now-10m/m,to:now/m}}}],facets:{timestamp:{terms_stats:{key_field:node.ip_port.raw,value_field:@timestamp,order:term,size:2000}},os.cpu.usage:{terms_stats:{key_field:node.ip_port.raw,value_field:os.cpu.usage,order:term,size:2000}},os.load_average.1m:{terms_stats:{key_field:node.ip_port.raw,value_field:os.load_average.1m,order:term,size:2000}},jvm.mem.heap_used_percent:{terms_stats:{key_field:node.ip_port.raw,value_field:jvm.mem.heap_used_percent,order:term,size:2000}},fs.total.available_in_bytes:{terms_stats:{key_field:node.ip_port.raw,value_field:fs.total.available_in_bytes,order:term,size:2000}},fs.total.disk_io_op:{terms_stats:{key_field:node.ip_port.raw,value_field:fs.total.disk_io_op,order:term,size:2000]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:586)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:489)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:474)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:467)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:239)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at
org.elasticsearch.action.search.type.TransportSearchCountAction$AsyncAction.sendExecuteFirstPhase(TransportSearchCountAction.java:70)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.search.facet.FacetPhaseExecutionException:
Facet [fs.total.available_in_bytes]: failed to find mapping for
fs.total.available_in_bytes
at
org.elasticsearch.search.facet.termsstats.TermsStatsFacetParser.parse(TermsStatsFacetParser.java:132)
at
org.elasticsearch.search.facet.FacetParseElement.parse(FacetParseElement.java:93)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:574)
... 12 more
Making Marvel only semi-useful; I don't have anything disabled in my
elasticsearch.yml it's pretty stock; any help?
I tried asking in IRC but #elasticsearch isn't very helpful with questions
in regards to Marvel.
Thanks!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/326afeb5-1282-4ccd-979e-689556f84403%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Marvel not collecting system metrics
I forgot to mention, the Dockerfiles I use to build my Elasticsearch are at
github.com/damm/dockerfiles; feel free to review the java/ image for how I
build java and elasticsearch/ for ES.
Is it possible there's some external tools I need installed?
Thanks!
On Thu, Feb 13, 2014 at 6:57 PM, Scott Likens smlik...@gmail.com wrote:
Hi,
I've been deploying Elasticsearch with Docker and it's been working great.
So I added the Marvel plugin and I get a ton of these in my Elasticsearch
error log and no system metrics in Marvel. :(
[2014-02-14 02:55:02,753][DEBUG][action.search.type ] [Kamal]
[.marvel-2014.02.14][0], node[_Ofz6YYlTc2hA9ASu3qlRg], [P], s[STARTED]:
Failed to execute [org.elasticsearch.action.search.SearchRequest@53b9f5]
org.elasticsearch.search.SearchParseException: [.marvel-2014.02.14][0]:
query[ConstantScore(BooleanFilter(+no_cache(@timestamp:[1392342894189 TO
1392346502752]) +cache(_type:node_stats) +cache(@timestamp:[139234590
TO 139234656])))],from[-1],size[0]: Parse Failure [Failed to parse
source
[{size:0,query:{filtered:{query:{match_all:{}},filter:{bool:{must:[{range:{@timestamp:{from:1392342894189,to:now}}},{term:{_type:node_stats}},{range:{@timestamp:{from:now-10m/m,to:now/m}}}],facets:{timestamp:{terms_stats:{key_field:node.ip_port.raw,value_field:@timestamp,order:term,size:2000}},os.cpu.usage:{terms_stats:{key_field:node.ip_port.raw,value_field:os.cpu.usage,order:term,size:2000}},os.load_average.1m:{terms_stats:{key_field:node.ip_port.raw,value_field:os.load_average.1m,order:term,size:2000}},jvm.mem.heap_used_percent:{terms_stats:{key_field:node.ip_port.raw,value_field:jvm.mem.heap_used_percent,order:term,size:2000}},fs.total.available_in_bytes:{terms_stats:{key_field:node.ip_port.raw,value_field:fs.total.available_in_bytes,order:term,size:2000}},fs.total.disk_io_op:{terms_stats:{key_field:node.ip_port.raw,value_field:fs.total.disk_io_op,order:term,size:2000]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:586)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:489)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:474)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:467)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:239)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at
org.elasticsearch.action.search.type.TransportSearchCountAction$AsyncAction.sendExecuteFirstPhase(TransportSearchCountAction.java:70)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.search.facet.FacetPhaseExecutionException:
Facet [fs.total.available_in_bytes]: failed to find mapping for
fs.total.available_in_bytes
at
org.elasticsearch.search.facet.termsstats.TermsStatsFacetParser.parse(TermsStatsFacetParser.java:132)
at
org.elasticsearch.search.facet.FacetParseElement.parse(FacetParseElement.java:93)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:574)
... 12 more
Making Marvel only semi-useful; I don't have anything disabled in my
elasticsearch.yml it's pretty stock; any help?
I tried asking in IRC but #elasticsearch isn't very helpful with questions
in regards to Marvel.
Thanks!
--
You received this message because you are subscribed to a topic in the
Google Groups elasticsearch group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/dhpxaOuoZWI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/326afeb5-1282-4ccd-979e-689556f84403%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALDJu4VnrbgMfGmcz_hKsnAo5re0Jun%2BttF101OBJ7vWVe6Btg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
