[
https://issues.apache.org/jira/browse/SOLR-10003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-10003:
----------------------------------
Description:
Certain parallel operations, for example *Pregel*, require all workers to
complete a step in the process before starting the next step. This is sometimes
called a *superstep*.
This ticket will introduce a Streaming Expression that coordinates the
superstep. The basic structure of this syntax is:
{code}
superstep(parallel(daemon(...)))
{code}
Each call to superstep.read() calls a parallel(daemon()) which sends a batch
job to the workers. The superstep will only return the tuple from read() when
each parallel daemon has completed the step.
Under the covers the superstep will poll the workers and check the state of the
daemons on the workers. New daemon states will be added so that processes can
vote to end the entire task. When all workers vote to end the task superstep
will return the EOF tuple and the job will end.
was:
Certain parallel operations, for example *Pregel*, require all workers to
complete a step in the process before starting the next step. This is sometimes
called a *superstep*.
This ticket will introduce a Streaming Expression thats coordinates the
superstep. The basic structure of this syntax is:
{code}
superstep(parallel(daemon(...)))
{code}
Each call to superstep.read() calls a parallel(daemon()) which sends a batch
job to the workers. The superstep will only return the tuple from read() when
each parallel daemon has completed the step.
Under the covers the superstep will poll the workers and check the state of the
daemons on the workers. New daemon states will be added so that processes can
vote to end the entire task. When all workers vote to end the task superstep
will return the EOF tuple and the job will end.
> Add superstep Streaming Expression
> ----------------------------------
>
> Key: SOLR-10003
> URL: https://issues.apache.org/jira/browse/SOLR-10003
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Joel Bernstein
>
> Certain parallel operations, for example *Pregel*, require all workers to
> complete a step in the process before starting the next step. This is
> sometimes called a *superstep*.
> This ticket will introduce a Streaming Expression that coordinates the
> superstep. The basic structure of this syntax is:
> {code}
> superstep(parallel(daemon(...)))
> {code}
> Each call to superstep.read() calls a parallel(daemon()) which sends a batch
> job to the workers. The superstep will only return the tuple from read() when
> each parallel daemon has completed the step.
> Under the covers the superstep will poll the workers and check the state of
> the daemons on the workers. New daemon states will be added so that processes
> can vote to end the entire task. When all workers vote to end the task
> superstep will return the EOF tuple and the job will end.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
