[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261709#comment-16261709
]
Aroop commented on SOLR-11600:
--
Thanks a lot [~joel.bernstein] ! I will try and use this as the guide!
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261446#comment-16261446
]
Joel Bernstein commented on SOLR-11600:
---
There is an approach in Java that you can use to build the expressions so you
don't have to build the strings directly.
The approach is to use the StreamExpression classes to construct the expression:
StreamExpression
StreamExpressionParameter
StreamExpressionNamedParameter
StreamExpressionValue
If you take a look at the toExpression method of many of the Streams you'll see
how TupleStream turns itself to a StreamExpression.
You can think of Streaming Expressions as having three representations:
1) String Expression: used as the query language and serialization format.
2) StreamExpression: intermediate format that can become a string a TupleStream
3) TupleStream: these are the compiled stream objects.
At Alfresco we have code that uses the StreamExpression classes to build
expressions and compile them to TupleStreams. This has worked quite well for us.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261252#comment-16261252
]
Aroop commented on SOLR-11600:
--
Thanks Joel.
Its good to know what the roadmap is here.
Personally though I am a fan of SolrStreaming Java classes approach, it does
set Solr apart from other data stores.
Although, I can imagine this seeming complicated to many users in java api, I
guess I have been programming using it a lot recently so I might be a bit
biased as I feel comfortable so far :)
But then we are moving in the sql direction or rather a DSL direction
completely?
A compile-time correctness-verification feature, then becomes vital for
programers' confidence.
Maybe when there is enough time and a larger audience (greater than just me :)
), you can revisit this ;)
Amrit - I am not 100% sure I understand what the next action items are, based
on your note, but I guess you will keep us posted.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260797#comment-16260797
]
Amrit Sarkar commented on SOLR-11600:
-
Thank you [~joel.bernstein] for the explanation;
> Each expression has it's own set of rules for the parameters that it accepts
> so we can get very specific with how type safety is handled
I completely understand this by the following example
{code}
replace( fieldA, add( fieldB, if( eq(fieldC,0), 0, 1)))
{code}
This nested evaluation and operation is not possible to create with current
Java constructors available, as the constructors of evaluators and operations
have most just one type of constructor with {{StreamExpression}}
(StreamExpressionParameter interface) parameter which the evaluators or
operators doesn't implement (they implement Expressible interface).
{code}
public AddEvaluator(StreamExpression expression, StreamFactory factory)
throws IOException{
super(expression, factory);
if(containedEvaluators.size() < 1){
throw new IOException(String.format(Locale.ROOT,"Invalid expression %s -
expecting at least one value but found
%d",expression,containedEvaluators.size()));
}
}
{code}
To accomodate the above request, strongly types java objects for all, we need
to create rule-based constructors for all the evaluators and operators, so that
those can be used in {{SelectStream}}.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260755#comment-16260755
]
Joel Bernstein commented on SOLR-11600:
---
Some of early Streaming API class have constructors that can be used directly.
The current direction though is to move away from people using the Streaming
API java classes directly. This is mainly because it's just too difficult to
support the java classes directly because it doubles the documentation, tests
etc...
It's also quite tricky to actually use the Java API directly and Streaming
Expressions are much easier to work with.
One of the things to consider is looking at where the Streaming Expressions can
become more type safe. Each expression has it's own set of rules for the
parameters that it accepts so we can get very specific with how type safety is
handled. We could also add a command to the /stream handler to just compile the
expression and not run, to see what errors occur at compile time.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260412#comment-16260412
]
Aroop commented on SOLR-11600:
--
ok Amrit. But I think we really should take a look at providing a strongly
typed option to make a Select stream with custom evaluators.
It is pretty odd that we can create a complex RollupStream in a strongly typed
way, but the moment we want to wrap it within a SelectStream, with some
standard yet custom Evaluators, one needs to go back to constructing a string
.. I was able to go make it as nice as possible, but this is sub-optimal in my
opinion :)
Once I have some time and I get compliance clearance from my organization, I
will consider contributing to this if possible.
But if possible lets do a thought experiment on this, may be we can discuss
this offline and propose a solution. I am still not sure why its not easy to
do. Maybe something is getting lost in translation.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259537#comment-16259537
]
Amrit Sarkar commented on SOLR-11600:
-
Meanwhile I had a second look on the description of yours again; you are
aspiring proper Java constructors. Well it is bit challenging considering it
{{StreamOperation}} is an interface and not exactly class which we can pass
incoming raw string value. I will see what can be done.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258474#comment-16258474
]
Amrit Sarkar commented on SOLR-11600:
-
Examples are listed under
https://lucene.apache.org/solr/guide/6_6/streaming-expressions.html#StreamingExpressions-StreamingRequestsandResponses
and http://joelsolr.blogspot.in/2015/04/the-streaming-api-solrjio-basics.html.
I have cook one example against {{master}} branch, which strictly required
httpClient::4.5.3
{code}
package stream.example;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.io.SolrClientCache;
import org.apache.solr.client.solrj.io.Tuple;
import org.apache.solr.client.solrj.io.eval.DivideEvaluator;
import org.apache.solr.client.solrj.io.stream.CloudSolrStream;
import org.apache.solr.client.solrj.io.stream.SelectStream;
import org.apache.solr.client.solrj.io.stream.StreamContext;
import org.apache.solr.client.solrj.io.stream.TupleStream;
import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.lang.invoke.MethodHandles;
import java.util.ArrayList;
import java.util.List;
public class QuerySolr {
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
static StreamFactory streamFactory = new StreamFactory()
.withCollectionZkHost("collection1","localhost:9983")
.withFunctionName("select", SelectStream.class)
.withFunctionName("search", CloudSolrStream.class)
.withFunctionName("div", DivideEvaluator.class);
public static void main(String args[]) throws IOException,
SolrServerException {
SelectStream stream = (SelectStream)streamFactory
.constructStream("select(\n" +
" search(collection1, fl=\"id,A_i,B_i\", q=\"*:*\",
sort=\"id asc\"),\n" +
" id as UNIQUE_KEY,\n" +
" div(A_i,B_i) as divRes\n" +
")");
attachStreamFactory(stream);
List tuples = getTuples(stream);
for (Tuple tuple : tuples) {
log.info("tuple: " + tuple.getMap());
System.out.println("tuple: " + tuple.getMap());
}
System.exit(0);
}
private static void attachStreamFactory(TupleStream tupleStream) {
StreamContext context = new StreamContext();
context.setSolrClientCache(new SolrClientCache());
context.setStreamFactory(streamFactory);
tupleStream.setStreamContext(context);
}
private static List getTuples(TupleStream tupleStream) throws
IOException {
tupleStream.open();
List tuples = new ArrayList();
for(;;) {
Tuple t = tupleStream.read();
if(t.EOF) {
break;
} else {
tuples.add(t);
}
}
tupleStream.close();
return tuples;
}
}
{code}
I need {{System.exit(0);}} to terminate the program, so pretty sure some
httpclient is not getting closed properly or such.
*_Also, the patch above is absolutely not required to make this work_*, we can
move forward with above examples and streams can be constructed without adding
constructors to each stream source, decorators or evaluators. The only
condition is we have to pass our own {{streamFactory}}.
Hope it helps.
P.S. Please disregard the PATCH, it serves no purpose.
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide
[jira] [Commented] (SOLR-11600) Add Constructor to SelectStream which takes StreamEvaluators as argument. Current schema forces one to enter a stream expression string only
[
https://issues.apache.org/jira/browse/SOLR-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258396#comment-16258396
]
Aroop commented on SOLR-11600:
--
Hi Amrit
Can you provide a sample test for this patch ?
I am trying to move forward with this but seems like it might take a while
before we can get this into a release?
> Add Constructor to SelectStream which takes StreamEvaluators as argument.
> Current schema forces one to enter a stream expression string only
> -
>
> Key: SOLR-11600
> URL: https://issues.apache.org/jira/browse/SOLR-11600
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ, streaming expressions
>Affects Versions: 6.6.1, 7.1
>Reporter: Aroop
>Priority: Trivial
> Labels: easyfix
> Attachments: SOLR-11600.patch
>
>
> The use case is to be able able to supply stream evaluators over a rollup
> stream in the following manner, but with instead with Strongly typed objects
> and not steaming-expression strings.
> {code:bash}
> curl --data-urlencode 'expr=select(
> id,
> div(sum(cat1_i),sum(cat2_i)) as metric1,
> coalesce(div(sum(cat1_i),if(eq(sum(cat2_i),0),null,sum(cat2_i))),0) as
> metric2,
> rollup(
> search(col1, q=*:*, fl="id,cat1_i,cat2_i,cat_s", qt="/export", sort="cat_s
> asc"),
> over="cat_s",sum(cat1_i),sum(cat2_i)
> ))' http://localhost:8983/solr/col1/stream
> {code}
> the current code base does not allow one to provide selectedEvaluators in a
> constructor, so one cannot prepare their select stream via java code:
> {code:java}
> public class SelectStream extends TupleStream implements Expressible {
> private static final long serialVersionUID = 1L;
> private TupleStream stream;
> private StreamContext streamContext;
> private Map selectedFields;
> private Map selectedEvaluators;
> private List operations;
> public SelectStream(TupleStream stream, List selectedFields)
> throws IOException {
> this.stream = stream;
> this.selectedFields = new HashMap();
> Iterator var3 = selectedFields.iterator();
> while(var3.hasNext()) {
> String selectedField = (String)var3.next();
> this.selectedFields.put(selectedField, selectedField);
> }
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> public SelectStream(TupleStream stream, Map
> selectedFields) throws IOException {
> this.stream = stream;
> this.selectedFields = selectedFields;
> this.operations = new ArrayList();
> this.selectedEvaluators = new HashMap();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
