Checking for tampering of indices
In Splunk, it is possible to detect tampering of logs. Splunk will take an event at ingestion time and create a hash value based on the event and your certificates/keys. You can then write searches that will re-hash the event to be compared to the original to indicate if anything has changed. We need something like that. How is that possible with elasticsearch? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3b724745-88ac-4484-9d21-284ec28697a9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
How to fix IndexMissingException
I recently ran into an issue where my cluster is reporting an
IndexMissingException. I tried deleting the faulty index, but I keep
getting the same error returned. How do I fix this problem?
$ curl -XDELETE 'http://localhost:9200/logstash-2014.09.04.11'
{error:IndexMissingException[[logstash-2014.09.04.11]
missing],status:404}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8b6cd6fb-14b9-4775-9750-7352c4c1369e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Unassigned Shards Problem
I removed all the extra allocation stuff. When I did that, the shards were all propogated. Health is green again. On Thursday, May 22, 2014 6:56:24 PM UTC-4, Brian Wilkins wrote: Went back and read the page again. So I made one master, workhorse, and balancer with rackid of rack_two for testing. One master shows rackid of rack_one. All nodes were restarted. The shards are still unassigned. Also,the indices in ElasticHQ are empty. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b6e26a5c-7709-40e2-ae06-7c94e830ae2c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Unassigned Shards Problem
I have five nodes : Two Master Nodes, One Balancer Node, One Workhorse
Node, and One Coordinator Node.
I am shipping events from logstash, redis, to elasticsearch.
At the moment, my cluster is RED. The shards are created but no index is
created. I used to get an index like logstash.2014-05-22, but not anymore.
I deleted all my data, Cluster health goes GREEN.
However, as soon as data is sent from logstash - redis - elasticsearch,
my cluster health goes RED. I end up with unassigned shards. In my
/var/log/elasticsearch/logstash.log on my master, I see this in the log:
[2014-05-22 12:03:20,599][INFO ][cluster.metadata ] [Bora]
[logstash-2014.05.22] creating index, cause [auto(bulk api)], shards
[5]/[1], mappings [_default_]
On my master, this is the configuration:
cluster:
name: logstash
routing:
allocation:
awareness:
attributes: rack
node:
data: true
master: true
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
{
cluster_name : logstash,
status : red,
timed_out : false,
number_of_nodes : 5,
number_of_data_nodes : 3,
active_primary_shards : 0,
active_shards : 0,
relocating_shards : 0,
initializing_shards : 0,
unassigned_shards : 10
}
Is there an incorrect setting? I also installed ElasticHQ. It tells me the
same information.
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/03c5974b-ae50-4f1c-9ba3-4ef94b564323%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Unassigned Shards Problem
Thanks for your reply. I set the node.rack to rack_one on all the nodes as
a test. In ElasticHQ, on the right it shows no indices. It is empty. In my
master, I see that the nodes are identifying with rack_one (all of them).
Any other clues?
Thanks
Brian
On Thursday, May 22, 2014 5:10:25 PM UTC-4, Mark Walkom wrote:
It does create an index, it says so in the log - [logstash-2014.05.22]
creating index - it's jut not assigning things.
You've set routing.allocation.awareness.attribute, but have you set the
node value, ie node.rack?
See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com javascript:
web: www.campaignmonitor.com
On 23 May 2014 02:22, Brian Wilkins bwil...@gmail.com javascript:wrote:
I have five nodes : Two Master Nodes, One Balancer Node, One Workhorse
Node, and One Coordinator Node.
I am shipping events from logstash, redis, to elasticsearch.
At the moment, my cluster is RED. The shards are created but no index is
created. I used to get an index like logstash.2014-05-22, but not anymore.
I deleted all my data, Cluster health goes GREEN.
However, as soon as data is sent from logstash - redis - elasticsearch,
my cluster health goes RED. I end up with unassigned shards. In my
/var/log/elasticsearch/logstash.log on my master, I see this in the log:
[2014-05-22 12:03:20,599][INFO ][cluster.metadata ] [Bora]
[logstash-2014.05.22] creating index, cause [auto(bulk api)], shards
[5]/[1], mappings [_default_]
On my master, this is the configuration:
cluster:
name: logstash
routing:
allocation:
awareness:
attributes: rack
node:
data: true
master: true
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
{
cluster_name : logstash,
status : red,
timed_out : false,
number_of_nodes : 5,
number_of_data_nodes : 3,
active_primary_shards : 0,
active_shards : 0,
relocating_shards : 0,
initializing_shards : 0,
unassigned_shards : 10
}
Is there an incorrect setting? I also installed ElasticHQ. It tells me
the same information.
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/03c5974b-ae50-4f1c-9ba3-4ef94b564323%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/03c5974b-ae50-4f1c-9ba3-4ef94b564323%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5bd4c3d5-3be5-44ef-a8a6-5dba6876130c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Unassigned Shards Problem
Went back and read the page again. So I made one master, workhorse, and balancer with rackid of rack_two for testing. One master shows rackid of rack_one. All nodes were restarted. The shards are still unassigned. Also,the indices in ElasticHQ are empty. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4ac7b65e-3bc7-4f51-85e5-65ca3719880d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Kibana 3 unable to connect to elasticsearch
I am on RHEL 6. I can send messages from my Logstash shipper to Redis to
Elasticsearch. I installed logstash via RPM on all my servers and I
installed elasticsearch 1.0.3 via RPM. When I issue the command via curl to
check my node status, I get two different versions. In Kibana 3, it tells
me that Your version of Elasticsearch is too old. Kibana requires
Elasticsearch 0.90.9 or above. As you can see, I am running a version
greater than 0.90.9. Kibana also tells me that Could not reach
http://[elastic-search-hostname]:9200/_nodes. If you are using a proxy,
ensure it is configured correctly. I am not using a proxy.
Any idea what is wrong?
$ curl http://[elastic-search-hostname]:9200/_nodes/process?pretty
{
cluster_name : logstash,
nodes : {
43V635FUQ46pksH9yZ8SuA : {
name : logstash-[hostname]-23165-2010,
transport_address : inet[/[logstash-central-server:9300],
host : [hostname],
ip : [logstash-central-server],
version : 1.1.1,
build : f1585f0,
attributes : {
client : true,
data : false
},
process : {
refresh_interval : 1000,
id : 23165,
max_file_descriptors : 16384,
mlockall : false
}
},
tV3Qwo-YRu-m_GUDZq5flQ : {
name : [hostname],
transport_address : inet[/[elastic-search-host:9300],
host : [hostname],
ip : [elastic-search-ip],
version : 1.0.3,
build : 61bfb72,
http_address : inet[/[elastic-search-host]:9200],
process : {
refresh_interval : 1000,
id : 3334,
max_file_descriptors : 65535,
mlockall : false
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53fdd8a0-5f61-41c7-8551-441ff6e225ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Kibana 3 unable to connect to elasticsearch
Thanks for the tips. It turns it that it was a DNS issue. But I am
going to install one of those tools.
On Fri, May 9, 2014 at 5:22 PM, Mark Walkom ma...@campaignmonitor.com wrote:
It'll help if you can gist/pastebin your config for kibana.
Also install a monitoring plugin like Marvel or ElasticHQ to give yourself
better insight into your cluster.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 9 May 2014 23:08, Brian Wilkins bwilk...@gmail.com wrote:
I am on RHEL 6. I can send messages from my Logstash shipper to Redis to
Elasticsearch. I installed logstash via RPM on all my servers and I
installed elasticsearch 1.0.3 via RPM. When I issue the command via curl to
check my node status, I get two different versions. In Kibana 3, it tells me
that Your version of Elasticsearch is too old. Kibana requires
Elasticsearch 0.90.9 or above. As you can see, I am running a version
greater than 0.90.9. Kibana also tells me that Could not reach
http://[elastic-search-hostname]:9200/_nodes. If you are using a proxy,
ensure it is configured correctly. I am not using a proxy.
Any idea what is wrong?
$ curl http://[elastic-search-hostname]:9200/_nodes/process?pretty
{
cluster_name : logstash,
nodes : {
43V635FUQ46pksH9yZ8SuA : {
name : logstash-[hostname]-23165-2010,
transport_address : inet[/[logstash-central-server:9300],
host : [hostname],
ip : [logstash-central-server],
version : 1.1.1,
build : f1585f0,
attributes : {
client : true,
data : false
},
process : {
refresh_interval : 1000,
id : 23165,
max_file_descriptors : 16384,
mlockall : false
}
},
tV3Qwo-YRu-m_GUDZq5flQ : {
name : [hostname],
transport_address : inet[/[elastic-search-host:9300],
host : [hostname],
ip : [elastic-search-ip],
version : 1.0.3,
build : 61bfb72,
http_address : inet[/[elastic-search-host]:9200],
process : {
refresh_interval : 1000,
id : 3334,
max_file_descriptors : 65535,
mlockall : false
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53fdd8a0-5f61-41c7-8551-441ff6e225ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups elasticsearch group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/ShUxMVg-PHk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624Z5c0%2BEiVF8viwo6SNNRwdd6%3DDEex-nT%2BKjtOauvEK%3DZQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAO_Eq30c91wCs%2BxsVOG%2BUckwNvKLXKvzPhUes2VQ8g%2Bca0zbeA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
