bq: Only thing which we can achieve is , documents will be routed
based on the hash values of the field values.
Then you have created your collection with compositeID routing or have
some other misconfiguration. You _must_
create your collection with "router.name=implicit".
Rather than _tell_ us
Hi Ketan,
I believe you need multiple shard looking the count 800M. How much will be
the index size? Assume it comes out to 400G and assume your VM/machines
has 64GB and practically you want to fit your index into memory for each
shard... With that I would create 10shards on 10 machines (40 GB
Hi Ketan,
I also tried various ways to route documents to different shards based on
some routing key value. eg: status: active,inactive and terminated should
go to 3 different shards. I tried creating implicit as well as composite id
routers. I could not route the documents to the shard I want.
Hi,
Can someone please let me for my below mentions use-case.
Regards,
Ketan
-Original Message-
From: Ketan Thanki
Sent: Wednesday, November 15, 2017 3:42 PM
To: 'solr-user@lucene.apache.org'
Subject: RE: How to routing document for send to particular shard range
Thanks Erik,
I have
Thanks Erik,
I have re-mentions it as some required details are missing in my mail. Using
CloudSolrClient my test case as below.
I have used routing key as projectId/2!documentId
1: Detail of Insert Document in SolrIndex
Document Size: 919551
Document Batch Size for insert: 5000 documents in
These numbers aren't very useful. inserting how much data? Querying
how much data? What kinds of queries? Are you indexing in batches or
one document at a time? Are you using SolrJ and CloudSolrClient?
94 seconds to do _what_? Execute 1,000 queries? Fetch all the
documents from the shard? Execute
Thanks Amrit ,
Actually we have huge amount of data so that's why thinking to index data into
particular shard accept it's looks difficult but need to achieve the
performance using document routing for huge data.
With configuration of 4 shard and 4 replica is it better to distribute the
Surely someone else can chim in;
but when you say: "so regarding to it we need to index the particular
> client data into particular shard so if its manageable than we will
> improve the performance as we need"
You can / should create different collections for different client data, so
that
Thanks Amrit,
My requirement to achieve best performance while using document routing
facility in solr so regarding to it we need to index the particular client data
into particular shard so if its manageable than we will improve the
performance as we need.
Please do needful.
Regards,
Thanks Amrit,
I getting it know so can you please told me anyhow can I achieve using
composite routing ? as mentions my requirement below.
Because will need to send particular client data to particular shard.
Regards,
-Original Message-
From: Amrit Sarkar
Ketan,
here I have also created new field 'core' which value is any shard where I
> need to send documents and on retrieval use '_route_' parameter with
> mentioning the particular shard. But issue facing still my
> clusterstate.json showing the "router":{"name":"compositeId"} is it means
> my
Hi Erik,
My requirement to index the documents of particular organization to specific
shard. Also I have made changes in core.properties as menions below.
Model Collection:
name=model
shard=shard1
collection=model
router.name=implicit
router.field=core
shards=shard1,shard2
Workset Collection:
You cannot just make configuration changes, whether you use implicit
or compositeId is defined when you _create_ the collection and cannot
be changed later.
You need to create a new collection and specify router.name=implicit
when you create it. Then you can route documents as you desire.
I
Thanks Amrit,
For suggesting me the approach.
I have got some understanding regarding to it and i need to implement implicit
routing for specific shard based. I have try by make changes on
core.properties. but it can't work So can you please let me for the
configuration changes needed. Is it
Ketan,
If you know defined indexing architecture; isn't it better to use
"implicit" router by writing logic on your own end.
If the document is of "Org1", send the document with extra param*
"_route_:shard1"* and likewise.
Snippet from official doc:
15 matches
Mail list logo
