Hi, Thanks again Joel for your reply. I have noted your suggestions.
I observed one more thing while using solrj to fetch the data using /stream with export and direct /export. The solr QTime is almost same, however elapsed time(total time) to fetch response in streaming with export is better than direct /export (Streaming export taking 30% less than /export). Is this also expected ? Regards Kamal Kishore On Tue, Oct 16, 2018 at 3:21 AM Joel Bernstein <joels...@gmail.com> wrote: > Yes this is correct. But keep in mind Streaming Expression has a wide range > of features that have nothing at all to do with the export handler. In > general with Streaming Expressions you want to find the functions that get > the job done using the least amount of work. The /export handler is often > not the best choice. You'll want to read through the various streaming > expressions to see if they might be more efficient for your use case. > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Mon, Oct 15, 2018 at 12:05 PM Kamal Kishore Aggarwal < > kkroyal....@gmail.com> wrote: > > > Hi, > > > > After I performed the test on my data, I found out that direct /export > and > > streaming expression with export, both are giving almost same response > > time. This was also pointed out by *Jan Høydahl* in his reply. > > > > Also, the documentation says export feature uses stream sorting technique > > and streaming expression also uses steam technique. So, are they > internally > > works in same fashion. Please confirm. > > > > Regards > > Kamal Kishore > > > > > > > > On Tue, Oct 2, 2018 at 5:51 PM Kamal Kishore Aggarwal < > > kkroyal....@gmail.com> > > wrote: > > > > > Hi, > > > > > > Thanks Jan & Joel. > > > > > > Though I will evaluate the performance over my data, but based on your > > > experience, which one of the two is better in performance ?. Please > > suggest > > > > > > Yeah, I know export does not get the data from all shards, but we can > > > write code to aggregate the data from all shards. But only if export is > > > better over stream. > > > > > > Thanks > > > Kamal Kishore > > > > > > > > > On Thu, Sep 27, 2018 at 11:04 PM Joel Bernstein <joels...@gmail.com> > > > wrote: > > > > > >> The export handler does not do distributed search. So if you have a > > >> multi-shard collection you may have to use Streaming Expressions to > get > > >> exports from all shards. > > >> > > >> > > >> Joel Bernstein > > >> http://joelsolr.blogspot.com/ > > >> > > >> > > >> On Thu, Sep 27, 2018 at 4:32 AM Jan Høydahl <jan....@cominvent.com> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > Yes, you can choose which to use, it should give you about same > > result. > > >> > If you already work with the Solr search API it would be the easiest > > for > > >> > you to consume /export as you don't need to learn the new syntax and > > >> parse > > >> > the Tuple response. However, if you need to do stuff with the docs > as > > >> you > > >> > stream them from Solr, then streaming expressions lets you enrich > the > > >> docs, > > >> > modify, join etc on the fly. > > >> > > > >> > PS: When the /export docs says it uses a streaming tecnique, it does > > NOT > > >> > mean that it has uses the solr feature streaming expressions :) > > >> > > > >> > -- > > >> > Jan Høydahl, search solution architect > > >> > Cominvent AS - www.cominvent.com > > >> > > > >> > > 27. sep. 2018 kl. 09:07 skrev Kamal Kishore Aggarwal < > > >> > kkroyal....@gmail.com>: > > >> > > > > >> > > Hi, > > >> > > > > >> > > I have a requirement to fetch all data from a collection. One way > is > > >> to > > >> > use > > >> > > streaming expression and other way is to use export. > > >> > > > > >> > > Streaming expression documentation says *streaming functions are > > >> designed > > >> > > to work with entire result sets rather then the top N results like > > >> normal > > >> > > search. This is supported by the /export handler.* > > >> > > > > >> > > Also, Export handler documentation says *this feature uses a > stream > > >> > sorting > > >> > > technique that begins to send records within milliseconds and > > >> continues > > >> > to > > >> > > stream results until the entire result set has been sorted and > > >> exported.* > > >> > > > > >> > > These two statements concludes to me that for fetching entire > > results > > >> > > streaming expressions uses export handler and export handler uses > > >> stream, > > >> > > so, whether I use streaming expression or export handler, they are > > >> > > internally same and would have same performance. I am correct over > > >> here > > >> > to > > >> > > say so ? > > >> > > > > >> > > > > >> > > Ref Links: > > >> > > > > >> > > > https://lucene.apache.org/solr/guide/6_6/streaming-expressions.html > > >> > > > https://lucene.apache.org/solr/guide/6_6/exporting-result-sets.html > > >> > > > > >> > > > > >> > > Regards > > >> > > Kamal Kishore > > >> > > > >> > > > >> > > > > > >
