Template queries with params in Java API. Bug?
Hi,
I am trying to figure out issues with these kinds of queries when using the
Java API.
curl http://10.10.9.13:9200/global6/user/_search/template '{
"template": {
"query": {
"term": {
"{{field}}" : "{{value}}"
}
},
"aggs" : {
"{{field}}" :
{ "terms" : { "field": "{{field}}"} }
}
},
"params": {
"field" : "twuser.twuser.name",
"value" : "alexander"
}
}'
Unfortunately, the params don’t seem to work in template queries when used
via the Java API (without params the template works). I am building the
query like this (thanks Colin G):
getSearchRequestBuilder().setTemplateSource(Resources.toString(url,
Charset.forName("UTF-8"))).setTemplateParams(templateParams);
I think the problem lies here:
org.elasticsearch.action.search.SearchRequest
if (out.getVersion().onOrAfter(Version.V_1_1_0)) {
out.writeBytesReference(templateSource);
out.writeOptionalString(templateName);
booleanexistTemplateParams = templateParams != null;
out.writeBoolean(existTemplateParams);
if (existTemplateParams) {
out.writeGenericValue(templateParams);
}
}
}
As far as I can see the params don’t get added as a json map, although I’m
not entirely familiar with the format of transport messages.
Logs from the slow log on the server show an empty space where the
placeholder should be (e.g. "{{value}}" in the template above looks like
“”).
Is this a bug? If so happy to produce a patch. If not, any workaround would
be welcome - I’m currently just doing the templating myself.
Kind regards,
Luke
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/187eed4b-0ac4-42e0-9dfe-c2027b667b13%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Aggregations don't seem to work with template queries.
Hi Colin, Thanks Colin. This does seem like it's the problem. I am building the query with the java query DSL but it seems to just stick the whole thing into the query section. TemplateQueryBuilder qb = new TemplateQueryBuilder(myQueryString, myParams) SearchResponse resp = client.prepareSearch(index).setQuery(qb ).execute().actionGet(); Any idea how to get the Java API to point to the template endpoint? Luke On Friday, 28 November 2014 11:48:10 UTC, Colin Goodheart-Smithe wrote: > > Hi Luke, > > The page you linked to is for templating just the query section of a > search request. To template the entire search request there is a dedicated > endpoint described in the following link: > > > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-template.html > > Let me know if this helps or not, > > Colin > > On Friday, 28 November 2014 11:37:26 UTC, Luke Wilson-Mawer wrote: >> >> If I send a perfectly well formed aggregation query as a template query, >> I don't get the aggregation results. >> >> >> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-template-query.html >> >> Is this by design, or is there a workaround? I am using the Java query >> DSL and version 1.2.3. >> >> Kind regards, >> >> Luke >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d13daac4-2092-4c87-b3d8-ef4c4a918337%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Aggregations don't seem to work with template queries.
If I send a perfectly well formed aggregation query as a template query, I don't get the aggregation results. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-template-query.html Is this by design, or is there a workaround? I am using the Java query DSL and version 1.2.3. Kind regards, Luke -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/960b74e9-4ef0-4e34-8b2d-e7c5dbf3930f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Upgrade to 1.2.1 routing fix tool gives stack trace.
I am getting a stack trace when I run the routing fix tool against my index.
Is this a known issue?
Kind regards,
Luke
java -jar elasticsearch-fix-routing-1.0.jar 10.10.9.14 9300 global count
Jun 10, 2014 1:03:16 PM org.elasticsearch.plugins
INFO: [Choice] loaded [], sites []
Index: global, number of shards: 12
Shard 1/12:
0 misplaced document(s)
Shard 2/12:
817595 misplaced document(s)
Shard 3/12:
Exception in thread "main"
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
execute phase [query], all shards failed; shardFailures
{[hqD_Ax-vQ5uZvVOUdW-Xaw][global][2]: RemoteTransportException[[Beautiful
Dreamer][inet[/10.10.9.13:9300]][search/phase/query]]; nested:
QueryPhaseExecutionException[[global][2]:
query[filtered(ConstantScore(ScriptFilter(bad-routes)))->cache(org.elasticsearch.index.search.nested.NonNestedDocsFilter@5233b6a2)],from[0],size[10]:
Query Failed [Failed to execute main query]]; nested: NullPointerException;
}
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.onFailure(TransportSearchTypeAction.java:179)
at
org.elasticsearch.search.action.SearchServiceTransportAction$6.handleException(SearchServiceTransportAction.java:224)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:181)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:171)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6797181-b205-4b20-83ef-944cd008c68a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Get by _id doesn't work but search does.
Great, thanks Adrien. I will eagerly await the tool.
Kind regards,
Luke
On Thu, Jun 5, 2014 at 1:34 PM, Adrien Grand wrote:
> Hi,
>
> This is very likely because of
> https://github.com/elasticsearch/elasticsearch/pull/6393
>
> See http://www.elasticsearch.org/blog/elasticsearch-1-2-1-released/ for
> more information, we are currently working on a tool that would help
> relocate documents to the right shard.
>
>
> On Thu, Jun 5, 2014 at 1:54 PM, Luke Wilson-Mawer <
> lukewilsonma...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm seeing weird behaviours with ids on elasticsearch 1.2.0 (recently
>> upgraded from 1.0.1).
>>
>> A search retrieves my document, showing the correct value for _id:
>>
>> [terminal] curl 'myServer:9200/global/_search?q=someField:something
>>
>> {"took":79,"timed_out":false,"_shards":{"total":12,"successful":12,"failed":0},"hits":{"total":1,"max_score":17.715034,"hits":[{"_index":"global","_type":"user","_id":"7a113e4f-44de-3b2b-a3f1-fb881da1b00a",...
>>
>> But a direct lookup on id doesn't:
>> [terminal] curl
>> 'myServer:9200/global/user/7a113e4f-44de-3b2b-a3f1-fb881da1b00a'
>>
>> {"_index":"global","_type":"user","_id":"7a113e4f-44de-3b2b-a3f1-fb881da1b00a","found":false}
>>
>> Any ideas? I'm hoping I won't have to reindex...
>>
>> Kind regards,
>>
>> Luke
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/e0c68f79-12fc-42e9-9a26-102c3b75f42e%40googlegroups.com
>> <https://groups.google.com/d/msgid/elasticsearch/e0c68f79-12fc-42e9-9a26-102c3b75f42e%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Adrien Grand
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/B2dlshf_dQI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6JRYTe01iXb4Lwdurxgmfp6QTXfmEBbA%2Bmci8cQMGZ-A%40mail.gmail.com
> <https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6JRYTe01iXb4Lwdurxgmfp6QTXfmEBbA%2Bmci8cQMGZ-A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKmH%2BjqgLZ6-qBHPvKbWLAbBiP_jqjt8QJ7kPWhSATKch0d4Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Get by _id doesn't work but search does.
Hi,
I'm seeing weird behaviours with ids on elasticsearch 1.2.0 (recently
upgraded from 1.0.1).
A search retrieves my document, showing the correct value for _id:
[terminal] curl 'myServer:9200/global/_search?q=someField:something
{"took":79,"timed_out":false,"_shards":{"total":12,"successful":12,"failed":0},"hits":{"total":1,"max_score":17.715034,"hits":[{"_index":"global","_type":"user","_id":"7a113e4f-44de-3b2b-a3f1-fb881da1b00a",...
But a direct lookup on id doesn't:
[terminal] curl
'myServer:9200/global/user/7a113e4f-44de-3b2b-a3f1-fb881da1b00a'
{"_index":"global","_type":"user","_id":"7a113e4f-44de-3b2b-a3f1-fb881da1b00a","found":false}
Any ideas? I'm hoping I won't have to reindex...
Kind regards,
Luke
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e0c68f79-12fc-42e9-9a26-102c3b75f42e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Multi GET API with facets
Hi, Can you use the Multi Get API with aggregation/facets? For example, if I store documents called Shapes and I have 500 shape_ids and I'd like to find out the average number of sides within that list of shape_ids. Kind regards, Luke -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eaa531c0-f3bf-45fd-b41e-b9599b0b013d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Slow query performance
Hi, I'm trying to improve query performance. It takes an average of about 3 seconds for simple queries which don't even touch a nested document, and it's sometimes longer. curl "http://searchbox:9200/global/user/_search?n=0&sort=influence:asc&q=user.name:Bill%20Smith"; Even without the sort it takes seconds. Here are the details of the cluster: 1.4TB index size. 210m documents that aren't nested (About 10kb each) 500m documents in total. (nested documents are small: 2-5 fields). About 128 segments per node. 3 nodes, m2.4xlarge (-Xmx set to 40g, machine memory is 60g) 3 shards. Index is on amazon EBS volumes. Replication 0 (have tried replication 2 with only little improvement) I don't see any noticeable spikes in CPU/memory etc. Any ideas how this could be improved? Kind regards, Luke -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/204f6112-18ca-46fb-83b9-730579c8fedd%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Adding arrays of objects using java update API and scripting.
How do I use updateRequestBuilder.addScriptParam to add List/Arrays.
I want to replace "ctx._source.communities" with the list of maps called
"new_communities" but it always ends up as an empty array.
List> communityMapList = new
ArrayList>();
for(CommunityRelationship community :
communitiesByTwitterUser.get(userId)){
Map communityMap = new
HashMap();
communityMap.put("communityId", community.getCommunityId());
communityMap.put("communityName",
community.getCommunityName());
communityMapList.add(communityMap);
}
UpdateRequestBuilder updateRequestBuilder = client
.prepareUpdate(SearchFields.INDEX, SearchFields.USER_TYPE, userId)
.addScriptParam("new_communities", communityMapList)
.setConsistencyLevel(WriteConsistencyLevel.ALL)
.setRetryOnConflict(10)
.setScript("ctx._source.communities = new_communities");
I've figured out how to pass in a single community append but I'd really
like to pass in a list of communities.
This seems a simple thing to do using json (Just write out the string
[{"communityId":32,"communityName":"lukes_community"},[{"communityId":33,"communityName":"bills_community"}]
but I can't see how to pass this into the java api.
Any ideas?
Luke
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e48362ad-e7d2-48df-ae3c-02c0ad867b3c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
