If you can include the stack trace and version of Solr we can see what's causing the exception.
Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jul 13, 2017 at 4:33 PM, Joe Obernberger < joseph.obernber...@gmail.com> wrote: > Thanks for this. I'm now trying to use stream for classify, but am > getting an ArrayIndexOutOfBounds error on the stream.open(). I'm setting > the streamFactory up, and including .withFunctionName("classify", > ClassifyStream.class) - but is that class in orga.apache.solr.handler? > > ----- > StringBuilder expression = new StringBuilder(); > solrCollection = getCollectionFromProfileBean(pBean); > expression.append("classify(model(models,id=\"").append(pBea > n.getModelID()).append("\",cacheMillis=5000),"); > expression.append("search(").append(solrCollection).append(" > ,q=\"DocumentId:").append(docID).append("\","); > expression.append("fl=\"ClusterText,id\",sort=\"id > asc\"),field=\"ClusterText\")"); > logger.info("Have classify expression:\n" + > expression.toString() + "\n"); > params.set("expr", expression.toString()); > params.set("qt", "/stream"); > params.set("explain", "true"); > params.set("q", "*:*"); > params.set("fl", "id"); > params.set("sort", "id asc"); > > context = new StreamContext(); > context.setSolrClientCache(StaticInfo.getSingleton(props). > getClientCache()); > context.workerID = 0; > context.numWorkers = 1; > context.setModelCache(StaticInfo.getSingleton(props).getModelCache()); > > streamFactory.withCollectionZkHost(solrCollection, > props.getProperty("hbase.zookeeper.solr.quorum")) > .withFunctionName("search", CloudSolrStream.class) > .withFunctionName("facet", FacetStream.class) > .withFunctionName("update", UpdateStream.class) > .withFunctionName("jdbc", JDBCStream.class) > .withFunctionName("topic", TopicStream.class) > .withFunctionName("commit", CommitStream.class) > // decorator streams > .withFunctionName("merge", MergeStream.class) > .withFunctionName("unique", UniqueStream.class) > .withFunctionName("top", RankStream.class) > .withFunctionName("reduce", ReducerStream.class) > .withFunctionName("parallel", ParallelStream.class) > .withFunctionName("rollup", RollupStream.class) > .withFunctionName("stats", StatsStream.class) > .withFunctionName("innerJoin", InnerJoinStream.class) > .withFunctionName("leftOuterJoin", > LeftOuterJoinStream.class) > .withFunctionName("hashJoin", HashJoinStream.class) > .withFunctionName("outerHashJoin", > OuterHashJoinStream.class) > .withFunctionName("intersect", IntersectStream.class) > .withFunctionName("complement", > ComplementStream.class) > .withFunctionName(SORT, SortStream.class) > .withFunctionName("train", TextLogitStream.class) > .withFunctionName("features", > FeaturesSelectionStream.class) > .withFunctionName("daemon", DaemonStream.class) > .withFunctionName("shortestPath", > ShortestPathStream.class) > .withFunctionName("gatherNodes", > GatherNodesStream.class) > .withFunctionName("nodes", GatherNodesStream.class) > .withFunctionName("select", SelectStream.class) > .withFunctionName("shortestPath", > ShortestPathStream.class) > .withFunctionName("gatherNodes", > GatherNodesStream.class) > .withFunctionName("nodes", GatherNodesStream.class) > .withFunctionName("scoreNodes", > ScoreNodesStream.class) > .withFunctionName("model", ModelStream.class) > .withFunctionName("classify", ClassifyStream.class) > .withFunctionName("fetch", FetchStream.class) > .withFunctionName("executor", ExecutorStream.class) > .withFunctionName("null", NullStream.class) > .withFunctionName("priority", PriorityStream.class) > .withFunctionName("significantTerms", > SignificantTermsStream.class) > .withFunctionName("cartesianProduct", > CartesianProductStream.class) > .withFunctionName("shuffle", ShuffleStream.class) > .withFunctionName("calc", CalculatorStream.class) > .withFunctionName("eval", EvalStream.class) > .withFunctionName("echo", EchoStream.class) > .withFunctionName("cell", CellStream.class) > .withFunctionName("list", ListStream.class) > .withFunctionName("let", LetStream.class) > .withFunctionName("get", GetStream.class) > .withFunctionName("timeseries", > TimeSeriesStream.class) > .withFunctionName("tuple", TupStream.class) > // metrics > .withFunctionName("min", MinMetric.class) > .withFunctionName("max", MaxMetric.class) > .withFunctionName("avg", MeanMetric.class) > .withFunctionName("sum", SumMetric.class) > .withFunctionName("count", CountMetric.class) > // tuple manipulation operations > .withFunctionName("replace", ReplaceOperation.class) > .withFunctionName("concat", ConcatOperation.class) > // stream reduction operations > .withFunctionName("group", GroupOperation.class) > .withFunctionName("distinct", DistinctOperation.class) > .withFunctionName("having", HavingStream.class) > // Stream Evaluators > .withFunctionName("val", RawValueEvaluator.class) > // Boolean Stream Evaluators > .withFunctionName("and", AndEvaluator.class) > .withFunctionName("eor", ExclusiveOrEvaluator.class) > .withFunctionName("eq", EqualsEvaluator.class) > .withFunctionName("gt", GreaterThanEvaluator.class) > .withFunctionName("gteq", > GreaterThanEqualToEvaluator.class) > .withFunctionName("lt", LessThanEvaluator.class) > .withFunctionName("lteq", > LessThanEqualToEvaluator.class) > .withFunctionName("not", NotEvaluator.class) > .withFunctionName("or", OrEvaluator.class) > // Date Time Evaluators > .withFunctionName(TemporalEvaluatorYear.FUNCTION_NAME, > TemporalEvaluatorYear.class) > .withFunctionName(TemporalEvaluatorMonth.FUNCTION_NAME, > TemporalEvaluatorMonth.class) > .withFunctionName(TemporalEvaluatorDay.FUNCTION_NAME, > TemporalEvaluatorDay.class) > .withFunctionName(TemporalEvaluatorDayOfYear.FUNCTION_NAME, > TemporalEvaluatorDayOfYear.class) > .withFunctionName(TemporalEvaluatorHour.FUNCTION_NAME, > TemporalEvaluatorHour.class) > .withFunctionName(TemporalEvaluatorMinute.FUNCTION_NAME, > TemporalEvaluatorMinute.class) > .withFunctionName(TemporalEvaluatorSecond.FUNCTION_NAME, > TemporalEvaluatorSecond.class) > .withFunctionName(TemporalEvaluatorEpoch.FUNCTION_NAME, > TemporalEvaluatorEpoch.class) > .withFunctionName(TemporalEvaluatorWeek.FUNCTION_NAME, > TemporalEvaluatorWeek.class) > .withFunctionName(TemporalEvaluatorQuarter.FUNCTION_NAME, > TemporalEvaluatorQuarter.class) > .withFunctionName(TemporalEvaluatorDayOfQuarter.FUNCTION_NAME, > TemporalEvaluatorDayOfQuarter.class) > // Number Stream Evaluators > .withFunctionName("abs", AbsoluteValueEvaluator.class) > .withFunctionName("add", AddEvaluator.class) > .withFunctionName("div", DivideEvaluator.class) > .withFunctionName("mult", MultiplyEvaluator.class) > .withFunctionName("sub", SubtractEvaluator.class) > .withFunctionName("log", NaturalLogEvaluator.class) > .withFunctionName("pow", PowerEvaluator.class) > .withFunctionName("mod", ModuloEvaluator.class) > .withFunctionName("ceil", CeilingEvaluator.class) > .withFunctionName("floor", FloorEvaluator.class) > .withFunctionName("sin", SineEvaluator.class) > .withFunctionName("asin", ArcSineEvaluator.class) > .withFunctionName("sinh", > HyperbolicSineEvaluator.class) > .withFunctionName("cos", CosineEvaluator.class) > .withFunctionName("acos", ArcCosineEvaluator.class) > .withFunctionName("cosh", > HyperbolicCosineEvaluator.class) > .withFunctionName("tan", TangentEvaluator.class) > .withFunctionName("atan", ArcTangentEvaluator.class) > .withFunctionName("tanh", > HyperbolicTangentEvaluator.class) > .withFunctionName("round", RoundEvaluator.class) > .withFunctionName("sqrt", SquareRootEvaluator.class) > .withFunctionName("cbrt", CubedRootEvaluator.class) > .withFunctionName("coalesce", CoalesceEvaluator.class) > .withFunctionName("uuid", UuidEvaluator.class) > .withFunctionName("corr", CorrelationEvaluator.class) > // Conditional Stream Evaluators > .withFunctionName("if", IfThenElseEvaluator.class) > .withFunctionName("convert", > ConversionEvaluator.class); > context.setStreamFactory(streamFactory); > try { > stream = new CloudSolrStream(props.getPrope > rty("hbase.zookeeper.solr.quorum"), solrCollection, params); > stream.setStreamContext(context); > > stream.open(); > > > On 7/13/2017 2:43 PM, Susheel Kumar wrote: > >> This the working code snippet I have, if that helps >> >> public static void main(String []args) throws IOException >> { >> String clause; >> TupleStream stream; >> List<Tuple> tuples; >> StreamContext streamContext = new StreamContext(); >> SolrClientCache solrClientCache = new SolrClientCache(); >> streamContext.setSolrClientCache(solrClientCache); >> >> StreamFactory factory = new StreamFactory() >> .withCollectionZkHost("gettingstarted", >> "server1:2182, server2:2182,server3:2182/solr66") >> // .withCollectionZkHost("gettingstarted", "localhost:2181") >> .withFunctionName("search", CloudSolrStream.class) >> .withFunctionName("select", SelectStream.class) >> .withFunctionName("add", AddEvaluator.class) >> .withFunctionName("if", IfThenElseEvaluator.class) >> .withFunctionName("gt", GreaterThanEvaluator.class) >> .withFunctionName("let", LetStream.class) >> .withFunctionName("get", GetStream.class) >> .withFunctionName("echo", EchoStream.class) >> .withFunctionName("merge", MergeStream.class) >> .withFunctionName("sort", SortStream.class) >> .withFunctionName("tuple", TupStream.class) >> .withFunctionName("rollup",RollupStream.class) >> .withFunctionName("hashJoin", HashJoinStream.class) >> .withFunctionName("complement", ComplementStream.class) >> .withFunctionName("fetch", FetchStream.class) >> .withFunctionName("having",HavingStream.class) >> .withFunctionName("eq", EqualsEvaluator.class) >> .withFunctionName("count", CountMetric.class) >> .withFunctionName("facet", FacetStream.class) >> .withFunctionName("sum", SumMetric.class) >> .withFunctionName("unique", UniqueStream.class) >> .withFunctionName("uniq", UniqueMetric.class) >> .withFunctionName("innerJoin", InnerJoinStream.class) >> .withFunctionName("intersect", IntersectStream.class) >> >> ; >> try { >> clause = getClause2(); >> // clause = getFacet(); >> stream = factory.constructStream(clause); >> stream.setStreamContext(streamContext); >> tuples = getTuples(stream); >> >> for(Tuple tuple : tuples ) >> { >> System.out.println(tuple.getString("id")); >> System.out.println(tuple.getString("sr_sv_business_email_s")); >> System.out.println(tuple.getString("sum(price_i)")); >> System.out.println(tuple.getString("count(price_i)")); >> System.out.println(tuple.getString("unique(price_i)")); >> System.out.println(tuple.getString("email")); >> } >> >> System.out.println("Total tuples retunred "+tuples.size()); >> } finally { >> solrClientCache.close(); >> } >> >> On Thu, Jul 13, 2017 at 2:18 PM, Joel Bernstein <joels...@gmail.com> >> wrote: >> >> It's most likely that you're not setting the StreamContext. New versions >>> of >>> Solr expect the StreamContext to be set before the stream is opened. The >>> SolrClientCache also needs to present in the StreamContext. You can take >>> a >>> look at how the StreamHandler does this for an example: >>> https://github.com/apache/lucene-solr/blob/master/solr/ >>> core/src/java/org/apache/solr/handler/StreamHandler.java#L339 >>> >>> Joel Bernstein >>> http://joelsolr.blogspot.com/ >>> >>> On Thu, Jul 13, 2017 at 2:06 PM, Joe Obernberger < >>> joseph.obernber...@gmail.com> wrote: >>> >>> Hi All - trying to call ClouderSolrStream.open(), but I'm getting this >>>> error: >>>> >>>> java.io.IOException: java.lang.NullPointerException >>>> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.const >>>> ructStreams(CloudSolrStream.java:408) >>>> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open( >>>> CloudSolrStream.java:299) >>>> >>>> I'm passing in a valid zkHost, collection name, and parameters. In fact, >>>> if I take the stream expression and past it into the GUI, it works OK. >>>> >>> I'm >>> >>>> stumped by what could be null here. >>>> >>>> My code looks like the following, and I'm getting the error on >>>> stream.open(). >>>> >>>> StringBuilder expression = new StringBuilder(); >>>> expression.append("update(models, batchSize=\"50\","); >>>> expression.append("train(").append(solrCollectionName).append(","); >>>> expression.append("features(").append(solrCollectionName).append(","); >>>> expression.append("q=\"*:*\",featureSet=\"FSet_").append(sol >>>> rCollectionName).append("\","); >>>> expression.append("field=\"Text\",outcome=\"out_i\",positive >>>> Label=1,numTerms=").append(numTerms).append("),"); >>>> expression.append("q=\"*:*\",name=\"").append(docID).append( >>>> "\",field=\"Text\",outcome=\"out_i\",maxIterations=\""). >>>> append(maxIterations).append("\"))"); >>>> logger.info("Have update expression:\n"+expression.toSt >>>> ring()+"\n"); >>>> params.set("expr", expression.toString()); >>>> params.set("qt", "/stream"); >>>> params.set("explain", "true"); >>>> params.set("q", "*:*"); >>>> params.set("fl", "id"); >>>> params.set("sort", "id asc"); >>>> try { >>>> System.out.println("Open: "+props.getProperty("hbase.zoo >>>> keeper.solr.quorum")+"\nCollection: "+solrCollectionName+" \nWith >>>> >>> params: >>> >>>> "+params); >>>> stream = new CloudSolrStream(props.getPrope >>>> rty("hbase.zookeeper.solr.quorum"), solrCollectionName, params); >>>> stream.open(); >>>> while (true) { >>>> Tuple tuple = stream.read(); >>>> logger.info("Tuple Read: "+tuple.fields.toString()); >>>> if (tuple.EOF) { >>>> break; >>>> } >>>> } >>>> } catch (IOException ex) { >>>> logger.error("Solr stream error: " + ex); >>>> ex.printStackTrace(); >>>> } finally { >>>> if (stream != null) { >>>> try { >>>> stream.close(); >>>> } catch (IOException ex) { >>>> logger.error("Could not close stream: "+ex); >>>> } >>>> } >>>> } >>>> >>>> I'm stuck! Thanks! >>>> >>>> -Joe >>>> >>>> >>>> >> --- >> This email has been checked for viruses by AVG. >> http://www.avg.com >> >> >
