[
https://issues.apache.org/jira/browse/SOLR-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14703277#comment-14703277
]
Scott Blum edited comment on SOLR-6760 at 8/19/15 4:25 PM:
-----------------------------------------------------------
Fixed. To recap IRC discussion, the old test code assumed that calling offer()
would result in the offered element immediately being available from poll().
This is contrary to the design decision in the new DQ.
1) Fixed the test code assumptions and generally cleaned up the test code to be
more clear.
2) Documented offer() semantics.
On the decision to decouple offer() and poll():
The rationale for the design decision is that being the offering VM is not
"special". In the general case different VMs often offer while a single VM is
doing the polling, the two are decoupled in reality. So there's no real need
to guarantee that offer() followed by poll() in a single VM always returns the
element immediately
was (Author: dragonsinth):
Fixed. To recap IRC discussion, the old test code assumed that calling offer()
would result in the offered element immediately being available from poll().
This is contrary to the design decision in the new DQ.
1) Fixed the test code assumptions and generally cleaned up the test code to be
more clear.
2) Documented offer() semantics.
> New optimized DistributedQueue implementation for overseer
> ----------------------------------------------------------
>
> Key: SOLR-6760
> URL: https://issues.apache.org/jira/browse/SOLR-6760
> Project: Solr
> Issue Type: Bug
> Reporter: Noble Paul
> Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-6760.patch, SOLR-6760.patch, SOLR-6760.patch,
> deadlock.patch
>
>
> Currently the DQ works as follows
> * read all items in the directory
> * sort them all
> * take the head and return it and discard everything else
> * rinse and repeat
> This works well when we have only a handful of items in the Queue. If the
> items in the queue is much larger (in tens of thousands) , this is
> counterproductive
> As the overseer queue is a multiple producers + single consumer queue, We can
> read them all in bulk and before processing each item , just do a
> zk.exists(itemname) and if all is well we don't need to do the fetch all +
> sort thing again
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
