Ok, I hadn't anticipated some of the scenarios that you've been trying out. Particularly reading streams into variables and performing joins etc...
The main idea with variables was to use them with the new statistical evaluators. So you perform retrievals (search, random, nodes, knn etc...) set the results to variables and then perform statistical analysis. The problem with joining variables is that is doesn't scale very well because all the records are read into memory. Also the parallel stream won't work over variables. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jun 22, 2017 at 3:50 PM, Susheel Kumar <susheel2...@gmail.com> wrote: > Hi Joel, > > I am able to reproduce this in a simple way. Looks like Let Stream is > having some issues. Below complement function works fine if I execute > outside let and returns an EOF:true tuple but if a tuple with EOF:true > assigned to let variable, it gets changed to EXCEPTION "Index 0, Size 0" > etc. > > So let stream not able to handle the stream/results which has only EOF > tuple and breaks the whole let expression block > > > ===Complement inside let > let( > a=echo(Hello), > b=complement(sort(select(tuple(id=1,email="A"),id,email),by="id asc,email > asc"), > sort(select(tuple(id=1,email="A"),id,email),by="id asc,email asc"), > on="id,email"), > c=get(b), > get(a) > ) > > Result > === > { > "result-set": { > "docs": [ > { > "EXCEPTION": "Index: 0, Size: 0", > "EOF": true, > "RESPONSE_TIME": 1 > } > ] > } > } > > ===Complement outside let > > complement(sort(select(tuple(id=1,email="A"),id,email),by="id asc,email > asc"), > sort(select(tuple(id=1,email="A"),id,email),by="id asc,email asc"), > on="id,email") > > Result > === > { "result-set": { "docs": [ { "EOF": true, "RESPONSE_TIME": 0 } ] } } > > > > > > > > > On Thu, Jun 22, 2017 at 11:55 AM, Susheel Kumar <susheel2...@gmail.com> > wrote: > > > Sorry for typo > > > > Facing a weird behavior when using hashJoin / innerJoin etc. The below > > expression display tuples from variable a shown below > > > > > > let(a=fetch(SMS,having(rollup(over=email, > > count(email), > > select(search(SMS, > > q=*:*, > > fl="id,dv_sv_business_email", > > sort="dv_sv_business_email asc"), > > id, > > dv_sv_business_email as email)), > > eq(count(email),1)), > > fl="id,dv_sv_business_email as email", > > on="email=dv_sv_business_email"), > > b=fetch(SMS,having(rollup(over=email, > > count(email), > > select(search(SMS, > > q=*:*, > > fl="id,dv_sv_personal_email", > > sort="dv_sv_personal_email asc"), > > id, > > dv_sv_personal_email as email)), > > eq(count(email),1)), > > fl="id,dv_sv_personal_email as email", > > on="email=dv_sv_personal_email"), > > c=innerJoin(sort(get(a),by="email asc"),sort(get(b),by="email > > asc"),on="email"), > > #d=select(get(c),id,email), > > get(a) > > ) > > > > var a result > > == > > { > > "result-set": { > > "docs": [ > > { > > "count(email)": 1, > > "id": "1", > > "email": "A" > > }, > > { > > "count(email)": 1, > > "id": "2", > > "email": "C" > > }, > > { > > "EOF": true, > > "RESPONSE_TIME": 18 > > } > > ] > > } > > } > > > > And after uncomment var d above, even though we are displaying a, we get > > results shown below. I understand that join in my test data didn't find > any > > match but then it should not skew up the results of var a. When data > > matches during join then its fine but otherwise I am running into this > > issue and whole next expressions doesn't get evaluated due to this... > > > > > > after uncomment var d > > === > > { > > "result-set": { > > "docs": [ > > { > > "EXCEPTION": "Index: 0, Size: 0", > > "EOF": true, > > "RESPONSE_TIME": 44 > > } > > ] > > } > > } > > > > On Thu, Jun 22, 2017 at 11:51 AM, Susheel Kumar <susheel2...@gmail.com> > > wrote: > > > >> Hello Joel, > >> > >> Facing a weird behavior when using hashJoin / innerJoin etc. The below > >> expression display tuples from variable a and the moment I use get on > >> innerJoin / hashJoin expr on variable c > >> > >> > >> let(a=fetch(SMS,having(rollup(over=email, > >> count(email), > >> select(search(SMS, > >> q=*:*, > >> fl="id,dv_sv_business_email", > >> sort="dv_sv_business_email asc"), > >> id, > >> dv_sv_business_email as email)), > >> eq(count(email),1)), > >> fl="id,dv_sv_business_email as email", > >> on="email=dv_sv_business_email"), > >> b=fetch(SMS,having(rollup(over=email, > >> count(email), > >> select(search(SMS, > >> q=*:*, > >> fl="id,dv_sv_personal_email", > >> sort="dv_sv_personal_email asc"), > >> id, > >> dv_sv_personal_email as email)), > >> eq(count(email),1)), > >> fl="id,dv_sv_personal_email as email", > >> on="email=dv_sv_personal_email"), > >> c=innerJoin(sort(get(a),by="email asc"),sort(get(b),by="email > >> asc"),on="email"), > >> #d=select(get(c),id,email), > >> get(a) > >> ) > >> > >> var a result > >> == > >> { > >> "result-set": { > >> "docs": [ > >> { > >> "count(email)": 1, > >> "id": "1", > >> "email": "A" > >> }, > >> { > >> "count(email)": 1, > >> "id": "2", > >> "email": "C" > >> }, > >> { > >> "EOF": true, > >> "RESPONSE_TIME": 18 > >> } > >> ] > >> } > >> } > >> > >> after uncomment var d above, even though we are displaying a, we get > >> results like below. I understand that join in my test data didn't find > any > >> match but then it should not skew up the results of var a. When data > >> matches during join then its fine but otherwise I am running into this > >> issue and whole next expressions doesn't get evaluated due to this... > >> > >> > >> { > >> "result-set": { > >> "docs": [ > >> { > >> "EXCEPTION": "Index: 0, Size: 0", > >> "EOF": true, > >> "RESPONSE_TIME": 44 > >> } > >> ] > >> } > >> } > >> > > > > >
