My first attempt at Kibana 3.1.0 was a little bumpy due to browser issues.
After some reading, I performed the minimal download, unpack, and point
the browser at index.html. If both Kibana and ES are on the same machine it
will just work.
Not quite! Contrary to all of the don't set this to
Drew,
This may not help you, but it's based on my own experience.
Using the Java API (I also don't use the REST API except for exploration
and problem reporting), I just use the JSON string as the source for every
document.
For one of my applications (more generic), I have my own generic
I believe that the host names must be comma-separated and no quotes or
other punctuation should be present. The config within the environment
variable or java -D (system properties) is YAML-like, not JSON. No blanks
between the names, for example:
export ES_HOSTS=host1,host2,host3
When
Are versions of the ELK stack components compatible with each other as seen
by the Elasticsearch Downloads link? Or do I need to specifically regress
back to the older 1.1.1 ES version?
From the following Downloads site
http://www.elasticsearch.org/overview/elkdownloads/ I downloaded the most
For all my previous ES versions (up to and including 1.1.1), my Java code
passed the
org.elasticsearch.action.search.SearchOperationThreading.THREAD_PER_SHARD value
to the SearchRequestBuilder.setOperationThreading API.
Now with ES 1.2, the Javac build fails as the SearchOperationThreading
Up through ES 1.1.1 the following Java snippet was able to take an array of
one or more index specifications (e.g. test*, ix*, sgen) and create a list
of index names that match (e.g. test1, test2, test3, ix1, ix2, sgen):
/* Create array of 1 or more index specifications */
String[]
Thanks, Ivan!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
Finally figured out the answer to my own question! Digging through the
deprecated REST API documentation and stumbling across
https://github.com/elasticsearch/elasticsearch/pull/5094/files I found
enough to be able to convert my code to use the new recovery-based API and
no longer use the
Prateek ,
I've collected this from various sources and put it all together. Works
fine for me, though I haven't yet dived into ELK:
-
You may verify the current soft limit by logging into the user that runs
the Elastisearch JVM and issuing the following command:
$ ulimit -Sn
Finally,
You aren't doing anything wrong. Elasticsearch is just telling you that you
have attempted to change the mapping to one that is not compatible with the
current mapping.
I believe that the best advice would be to export / snapshot the data, then
delete and re-add the index with the corrected
The _id (metadata _id, that is) is a string value.
The value of the _id string is whatever you set it to (or else a long
string automatically generated by Elasticsearch when the document is
created). You have complete freedom to set it to any string value you wish.
The _id metadata is the
Stefan,
If the TransportClient or NodeClient is set up as needed, then you can
easily wait for the cluster to be ready to use and let Elasticsearch do the
per-server discovery itself. At a high level (assuming this is part of some
class that includes private Client client;). Error reporting is
Jing,
I don't have much experience with ES in a production cluster environment;
all my experience has been with the Java API for mapping, bulk load, and
query logic, and with huge databases and things like that. But my 3-node
test ES cluster has gathered some dust over the past few months as
Robbie,
Are you repeating the scan until there are no more hits returned? I've
never bothered to check the repeatability of the individual chunks and
only notice that the overall total count is as expected. Also note the
following from the guide:
The scroll request also returns a *new
My 2 cents: I vastly prefer the Java API. Much better performance than
REST, much better control over the many features, and while there is a lot
of code to write (yes, this is a benefit to me, as I love to write code)
there is also a lot more gained insight and learning. The Java API is very
Axel,
Well, I use the Java API and not the REST API. And while there is a lot
more code around this, here's the core of my bulk load process. I use the
BulkRequestBuilder (a new one for each set of bulk updates!) and not the
BulkProcessor because (a) this code was already written and works and
Deepak,
The output you see is likely calling the XContentBuilder.*to*String method.
The output you want (the JSON string) is obtained by calling the
XContentBuilder.string method instead.
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch
Matt,
I don't know if this helps, but we are seeing similar issues with Flume
using log4j2 (not log4j v1 as used by ES). For tomcat-hosted servlets,
flume failover works fine. But for non-tomcat applications (such as looping
batch-mode applications and Netty-based servers with static main
Preeti,
I just updated my Java update command to allow the source JSON to be
specified (it typically created the source from one or more name=value
pairs). It reads the JSON from a file so that there is no Bash nor non-ES
Java interpretation of the \u sequences.
I updated the same record
When you query each type separately, ES is automatically choosing the
correct analyzer for your query term.
But when you query both types, ES can only choose one analyzer, and if the
two types are mapped with analyzers that are different enough, that one
analyzer will work in the type to which
Preeti,
I believe that your problem is in calling the *setScript* method. My Java
code uses the *IndexRequestBuilder.setSource* method. Try that instead. (I
have no idea what *setScript* does, or is for. The Javadocs need a lot more
care and feeding.)
Vai the HTTP interface, I verified that
Perhaps, index the Id field but do not analyze it? Then it will be indexed
and queried intact as-is.
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
Preeti,
Well, I tried to look at the addScript method code but that trail was
getting a bit too long.
However, I do have a JSON parser (wrapped inside a Bash script) that uses
the stream parser in the version of Jackson supplied with ES, and I can get
it to parse your JSON (stored in the
This usually means that there is no local server bound to that port. If
you're performing an integration test, it could be that you aren't giving
ES time to completely initialize and bind to the port and be ready to
accept connections. Or that you aren't configuring your local node to be a
When I use the Java TransportClient and the BulkRequest builder, my
throughput is like a scalded cat racing a bolt of greased lightning, with
the cat way ahead!
the Java API does not say how you are using it. Since I cannot see your
code, I cannot comment on where your mistake is located.
But
Yes, that is sufficient to clear out the documents. But... take the advice
given by Jörg to heart.
Elasticsearch is already optimized to take a bulk request and optimally
process it as fast as it can be done. There should not be more than one of
them at a time; no gain will be seen, and (as
Arun,
*Getting error when i applied TTL while creating Index. Able to search
data. I set TTL for 4m , after 4m still i see data available. How to make
TTL work and delete records after specified time. *
TTL processing is not something that can be depended upon to delete a
document
What I did with the high-performance query engine I built in 2001-2010 was
to OR the queries for individual fields, creating a query-time version of
the _all field. It was blindingly fast.
What I now do with Elasticsearch is to disable the _all field because of
the issues you've found (and
The HTTP interface is bound to port 9200. The TransportClient's server side
is bound to port 9300.
That is the reason.
Brian
Q)Actually why i am putting port 9200?
Ans: my elasticsearch url index is with port 9200,so i am trying to put
9200 in the place of 9300,but you initiated me to
Here is my code for the INDEX action. Similar code (not shown) exists for
the CREATE action. This is for ES 1.0.0 GA, though it hasn't changed since
its 0.90.X versions:
IndexResponse response = null;
try
{
IndexRequestBuilder irb = client.prepareIndex(index, type, id);
Are you using the BulkRequestBuilder? If so, create a new one for each bulk
operation (and let the de-referenced old one be garbage collected);
otherwise you'll be filling it up and times will drop as seen. At least,
that's what I do, and it runs like the blazes for the entire 97M document
Ivan,
Yes, ES stores all strings in UTF-8 encoding.
Referring to your 3 POST commands, the first two succeeded because in the
first one, you presented the data in the UTF-8 encoding and it was
accepted. In the second one, you presented the same name but in using the
\u notation which is
Jörg,
Just to clarify: The links below point to OpenJDK, not to the Oracle JDK? I
only ask because the version and build numbers seem to track those in
Oracle's JDK.
For what it's worth, I am currently running ES 1.0.0 GA with the latest
Oracle JDK 7u51 on Mac Mavericks and on Linux, and I
What version of ES are you using? I seem to recall reading about ambiguity
surrounding the use of type as a field name in one of the version's
release notes, but I cannot find it quickly now.
However, ES seems to overuse the word type, referring to both the
document type (roughly analogous to
But be careful: the docs for the Java 7
Boolean.parseBooleanhttp://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html#parseBoolean(java.lang.String)say
that:
The boolean returned represents the value true if the string argument is
not null and is equal, ignoring case, to the stringtrue.
Binh,
It's even more interesting. The behavior you describe matches the
configuration and REST parsing rules. But the document _source is JSON and
a boolean value must be either *true* or *false*. So my recommendation is
to use the configuration and REST interface as a forgiving practice to
The link to the corresponding documentation for version 0.90.XX is here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/0.90/search-request-highlighting.html#highlighting-settings
I was thinking that perhaps 0.90 did not include them, but they are also
listed in that version
I always start Elasticsearch from within my own wrapper script, es.sh.
Inside this wrapper script is the following incantation:
NODE_OPT=-D*es.node.name*=$(uname -n | cut -d'.' -f1)
This is verified to work on Linux, Mac OS X, and Solaris (at least).
I then pass $NODE_OPT as a command-line
And note that if you GET it and save the version number, and then pass the
version number into the DELETE, you can be sure it will be deleted only if
nobody else updated it in the meantime.
This all works so much better in Java than in scripts + curl.
Brian
On Tuesday, February 25, 2014
Based on posts to this newsgroup early on in my usage of ES (over a year
now!), I used to put the following in my elasticsearch.yml file. Any field
that was not explicitly assigned an analyzer and that was deemed by ES to
be a string would pick up English snowball analyzer with no stop words
I am not sure what the complaints are all about.
Over the past 20 years, my best practices are to treat the installed
configurations as a template that is subject to change upon reinstallation.
Then, I always create my own configuration and point the server to it, and
never point a server to
Tony,
(1) Your first question seems to be about the curl command. If so, then it
follows shell script rules. In most Unix shell scripts, double-quotes
preserve spaces and such, but allow for variable substitution. Sngle-quotes
also preserve spaces but do not allow variable substitution and
Tony,
JSON and ES both support uppercase. It's just that FOO and foo are two
completely different field names. Hence, case-sensitive.
In some contexts, ES does not support uppercase. For example, names of
indices. But field names can be in lowercase, uppercase, or mixed case. I
believe that
H. I see that the
src/main/java/org/elasticsearch/common/unit/ByteSizeUnit.java file defines
the units by uppercase enum name (e.g. `MB`), while the comments inside the
config/elasticsearch.yml file suggest lowercase (e.g. `mb`).
Perhaps there's an issue? Or am I reading something
, Ivan Brusic iv...@brusic.comjavascript:
wrote:
Aircraft use nautical miles? You learn something new every day!
--
Ivan
On Mon, Feb 10, 2014 at 3:21 PM, InquiringMind
brian@gmail.comjavascript:
wrote:
Would it be reasonable to create an issue to request nautical miles
(nm
Working on a pull request... I've created a fork off of master and cloned
it to my laptop. (First time using git and GitHub in this way...)
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving
Jörg,
Thanks for the tip! I just updated my client code and it works great! Will
be a help during future problem determination!
Brian
*On Monday, February 10, 2014 1:07:24 PM UTC-5, Jörg Prante wrote:*
*To find out Java versions, do the following.*
*On server, execute curl
Would it be reasonable to create an issue to request nautical miles (nm
as the abbreviation) for the DistanceUnit enumeration?
This would make it much more natural to adapt Elasticsearch for aircraft
planning / charting applications. Everything in that world is in nautical
miles and knots
*2) Though *not* recommended - kill -9 should not result in data loss. If
so it's a bug and should be reported.*
It *should* not, but it *may*. A kill -9 ends a process without allowing it
to flush any unwritten buffers to disk, close any open files, or even
finish writing what it started.
Not sure if this is your problem, but OpenJDK 6 is very bad. OpenJDK 7 has
been known to work well. Oracle Java 7 is best. And there's no correlation
between version numbers across Oracle Java and OpenJDK; 7 is just a
coincidence.
This is based on what I've read on this newsgroup, on blogs,
Since it's JSON, the order of the names does not matter. But to make it
nicer for humans to read, I have found that the LinkedHashMap and
LinkedHashSet classes to be nice drop-in replacements for their non-Linked
counterparts, have very little overhead, and have the nice property that
Nik,
I added an external post-query sorting facility that works very well.
Especially for a scan query that cannot be sorted by ES. For bounded but
accurate sorts, I implement a class that uses a TreeSort. When each search
hit is added, I build the sort keys and then add the (implements
David,
Using the Java API:
try
{
/* Create using the 3 strings: index, type, and id */
IndexRequestBuilder irb = client.prepareIndex(index, Name, id);
*/* Set the version number (a long integer value) */*
* irb.setVersion(versionNumber);*
*/* Set external versioning */*
*
ImmutableSettings.Builder settingsBuilder =
ImmutableSettings.settingsBuilder();
settingsBuilder.put(cluster.name, clusterName);
TransportClient client = new TransportClient(settingsBuilder.build());
for (String host : hostNames)
{
InetSocketTransportAddress server_address = new
IMHO, you don't need maven. You just need the Java trick to include all of
the ES dependent jar files by setting the classpath to a colon-separated
list of directory names, but append /* to the end of each directory name.
If you do this, you don't need to list all of the jars in the directory:
Honza,
They are totally different. Very simple examples of ES mappings might
appear similar to JSON schema. But ES mappings also allow you to define
analyzers and tokenizers and character filters and stop words and stemming
and time to live and such that are needed for search.
Brian
--
You
Jason,
Perhaps this might describe what you're looking for. It's based on ES
version 0.90.X:
https://groups.google.com/d/msg/elasticsearch/p4i3GvGDoUs/NFZat2BNj-wJ
I don't know what's in store for ES 1.X that might implement this.
I hope this helps you!
Brian
--
You received this message
Looking at the following to prepare for RC1:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/_multi_fields.html
I find the following. But shouldn't the semicolon after the first type
string be a comma?
Brian
you can now write:
title: {
type: string;
fields: {
Not a lot of code at all. Just some code that will be a lot of fun to
write. I did it with a lot more code, before I got into Jackson data
binding. But I wanted a generic Document class that adapted to a schema;
you won't need all that.
First, get Jackson and add the following jars to your
Also, see
https://groups.google.com/d/topic/elasticsearch/1152jRW_rQg/discussion for
how easy it is to issue an index request in Java. Hardly any code at all.
(Keep in mind the method name mismatch between the actual code and the
on-line docs, but those issues are clearly shown in this post).
One thought occurred to me. Perhaps:
1. Build the token count into your ingredients field. Here's how:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#token_count
2a. Pre-analyze your query arguments and remove the duplicates. For
example,
Jörg
I avoided multicast and preferred unicast based on many discussions in the
newsgroups and other sites. In particular, the ElasticSearch Preflight
Checklisthttp://asquera.de/opensource/2012/11/25/elasticsearch-pre-flight-checklist/.
Within this checklist, the sections entitled DISCOVERY
Wow! And, many, many thanks to the detailed Breaking Changes list. That
really helps with planning ahead!
Brian
On Wednesday, January 15, 2014 12:35:49 PM UTC-5, Alexander Reelsen wrote:
Hey
The release candidate 1 of the 1.0.0 series is now available, featuring
many scale and scalability
Thank you very much, Jörg. Your explanation is clear and concise and
greatly improves my understanding!
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
3) [ apples and oranges] //length =3, even with filter query (why not 1)?
...not an issue in my particular use case since all ingredients are
1-word, but I would like to understand how to address case #3 above.
From my experience with querying, ES slurps all values of an array into one
The data directory pathname is stored within the elasticsearch.yml
configuration file.
The path name to the configuration file may be specified passing the
following option to the startup script, where *path-to-config* is the name
of the directory into which you store your customized
I guess the usual question need to be answered: Are the Java versions
exactly the same on all systems?
And just to be sure, are the Elasticsearch versions exactly the same on all
systems?
*Does anyone know the issue or any resources that walk through how to
troubleshoot these issues?
Your client converts IP addresses to base64 strings before it indexes the
documents, so it must also use the same conversion to a base64 string when
querying the document.
Ranges are a little more complicated. For example. if you simply encode 9
and 111 as strings, then you cannot perform
Quick typing correction:
On Tuesday, January 14, 2014 4:00:58 PM UTC-5, InquiringMind wrote:
Your client converts IP addresses to base64 strings before it indexes the
documents, so it must also use the same conversion to a base64 string when
querying the document.
Ranges are a little more
*Wouldn't the value increase as you add more nodes?*
Indeed, it most certainly will.
And with unicast discovery, each node will need to be told about the new
node. Which is the perfect time to tell it about the newly calculated
minimum number of masters.
Brian
--
You received this
*But today elasticsearch doesn't do this automatically?*
Short answer: No.
Long answer: Nooo. *:-)*
For unicast discovery, you need to tell each node the full list of nodes in
the cluster. As far as I can tell, that list may include the local node so
*Never mind, I just had to import more jars from /lib.*
You can import all jars from /*some_base_path*/lib (for example) by adding
a /* to the end of the path, and then add that to the -cp / -classpathoption's
value, separating multiple paths with semicolons. That single
* (and *not*
Matthew,
I don't know if this is simple (though it was easy enough for me in Java),
or even if it's exactly what you had in mind. But it sounds as if you are
asking for a hierarchical combination to include the top URLs by uid. Is
that correct?
If so, perhaps
Nik,
No, there is not.
There's a work-around in which the number of terms in a field can be stored
in another field during indexing time. And then you can analyze your query
string to count the number of terms, and then use that count to match
against the documents that have the same count.
Chris,
I updated one of my tests to reproduce your issue. My text field is a
multi-field where *text.na* is the text field without any analysis at all.
This Lucene query does not find anything at all:
{
bool : {
must : {
query_string : {
query : *text.na:Immortal-Li**
*Never, never, never* kill -9 and expect any application to properly and
cleanly shut down. Never.
The -9 signal cannot be caught by the process to which it is directed. The
process is ended in the middle for whatever it is doing.
Issue a normal kill, and then ES (via the JVM) will have a
If it helps, here are my index settings and mappings. Note that I chose the
name text.na as the non-analyzed form, not text.raw. Perhaps I should
follow convention? But for now, a rose by any other name is still not
analyzed:
{
settings : {
index : {
number_of_shards : 1,
A quick guess: The first one works because the first document for that type
is indexed and therefore the type is created when the operation returns.
But the second one doesn't work because there is a refresh interval between
the completion of a bulk load operation and the actual document being
Ville,
Perhaps: Don't include the private fields in _all. Then a query against
_all would be restricted to the (perhaps hundreds) of public fields.
A query that includes the private fields would need to list _all and then
the private fields. But since you have only 2 or 3 private fields, there
Ville,
By default, the _all field includes all of the indexed field. Then, for
your private fields, explicitly exclude them from the _all field by adding
the following to their properties:
include_in_all : false
See the ES guide for more details, Specifically,
Adolfo,
Still could not test how sockets relate to shards and why I automatically
get 10 established sockets when opening a client:
node = builder.client(clientOnly).data(!clientOnly).local(local).node();
client = node.client();
on default ES configuration, and many many more sockets
This is an interesting problem. Typically, my view of stop words is dim. I
would prefer that the client side avoids searching on them if that is
desired, rather than the engine ignores them. Then, phrase matching can
work properly. And queries such as The Wall can look for just Wall(ignoring
I updated my elasticsearch.yml file to disable automatic index creation
(works great) and to disable indexing of unknown fields (not so great):
*action.auto_create_index: false*
*index.mapper.dynamic: false*
I fully shutdown ElasticSearch 0.90.8, then removed the default mappings
that were in
Perhaps I answered my own question. Doing a bit more research, I found that
the following can be added to the mappings object in front of all of the
individual mapped types when the index is created:
_default_ : {
dynamic : strict,
properties : { }
},
Of course, this doesn't
Jörg,
Thanks again for your insights and patience!
*OK, didn't know you use Java 6. The InetAddress change was in 6u45, so you
are affected when you use 6u65 with 6u18.*
That's very interesting. It threw me since I have always been able to query
and update remote single-node clusters with
One more breaking change (this compiled cleanly on 9.90.3):
compile:
[echo] debug support = true
[javac] Compiling 79 source files to /Users/brian/proj/build/classes
[javac]
/Users/brian/proj/src/java/com/acme/database/FieldMapping.java:240:
incompatible types
[javac] found :
Ivan,
*Online Javadoc? Is there such a unicorn? :)*
Hahaha! Yes, there are some older versions (19.X, and also 20.1 Beta) but
nothing later. I don't use them for the actual descriptive writeup (which
is usually quite barren), but find them very useful to check class and
method names during
Thanks so much, Jörg!
Or, as my favorite customer from my previous company would say, Kiitos!
(The only Finnish I know :-)
Brian
On Thursday, December 19, 2013 4:52:54 PM UTC-5, Jörg Prante wrote:
FYI here is ES 1.0.0.Beta2 Javadoc as of 8 Nov 2013
--
You received this message because
Jörg,
*Beside the cluster node JVMs you also have to take care of the client JVM.
Are you accessing the cluster also with Solaris x86 and Java 6u18?*
Oooh. Don't know why this didn't occur to me. Short answer: No.
Java on the MacBook (where the client / driver runs):
$ java -version
java
89 matches
Mail list logo
