Hi All,
I am trying to understand on how we can have commit issued to solr while
indexing documents. Around 200K to 300K document/per hour with an avg size of
10 KB size each will be getting into SOLR . JAVA code fetches the document from
MQ and streamlines it to SOLR. The problem is the client
Hi Sangeetha,
What is sure is that it is not going to work - with 200-300K doc/hour,
there will be >50 commits/second, meaning there are <20ms time for
doc+commit.
You can do is let Solr handle commits and maybe use real time get to
verify doc is in Solr or do some periodic sanity checks.
Are y
Hi Sangeetha,
Well I don't think you need to commit after every document add.
You can rely on Solr's transaction log feature . If you are using SolrCloud
it's mandatory to have a transaction log . So every documents get written
to the tlog . Now say a node crashes even if documents were not commi
failure the
document can be resent from MQ.
Thanks
Sangeetha
--
View this message in context:
http://lucene.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261575.html
Sent from the Solr - User mailing list archive at Nabble.com.
context:
http://lucene.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261576.html
Sent from the Solr - User mailing list archive at Nabble.com.
om MQ.
>
> Thanks
> Sangeetha
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261575.html
> Sent from the Solr - User mailing list archive at Nabble.com.
xt:
> http://lucene.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261575.html
> Sent from the Solr - User mailing list archive at Nabble.com.
this message in context:
http://lucene.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261587.html
Sent from the Solr - User mailing list archive at Nabble.com.
.472066.n3.nabble.com/Commit-after-every-document-alternate-approach-tp4260946p4261575.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On 3/3/2016 11:36 PM, sangs8788 wrote:
> When a commit fails, the document doesnt get cleared out from MQ and there is
> a task which runs in a background to republish the files to SOLR. If we do a
> batch commit we will not know we will end up redoing the same batch commit
> again. We currenlty ha
10 matches
Mail list logo