Just out of curiosity, why are you using SET to set the solr collection? I'm not sure if you're using an out of the box Load/Store Func, but if I were to design it, I would use the "location" of a Load/Store Func to specify which solr collection to write to.
Is it possible for you to redesign this way? On Thu, Jan 15, 2015 at 9:41 PM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: > Thanks > > SET sets the SOLR collection name. When the STORE is invoked, the data > will be ingested into the collection name set before. > > So, the problem must be because the second set is overriding the > collection name and the STORE is failing. > > Is there any way to overcome this? Because most of the processing time is > taken in the load and I don't want to do it twice. > > Regards, > Vishnu Viswanath > > > On 16-Jan-2015, at 09:29, Cheolsoo Park <piaozhe...@gmail.com> wrote: > > > > What does "SET" do for Solr? Pig pre-processes all the set commands in > the > > entire script before executing any query, and values are overwritten if > the > > same key is set more than once. In your example, you have two set > commands. > > If you're thinking that different values will be applied in each section, > > that's not the case. e) will overwrite a). > > > > > > On Thu, Jan 15, 2015 at 7:46 PM, Vishnu Viswanath < > > vishnu.viswanat...@gmail.com> wrote: > > > >> Hi All, > >> > >> I am in indexing data into solr using pig script. > >> I have two such scripts, and I tried combining these two scripts into a > >> single one. > >> > >> i.e., i have script 1 that does > >> -------------------- > >> a)SET solr collection info for collection 1 > >> b)LOAD data > >> c)FILTER data for SOLR collection number 1 > >> d)STORE data to solr > >> > >> > >> and script 2 that does > >> ------------------- > >> a)SET solr collection info for collection 2 > >> b)LOAD data > >> c)FILTER data for SOLR collection number 2 > >> d)STORE data to solr > >> > >> > >> combined script looks something like > >> -------------- > >> a)SET solr collection info for collection 1 > >> b)LOAD data > >> c)FILTER data from (b) for SOLR collection number 1 > >> d)STORE data to solr > >> e)SET solr collection info for collection 2 > >> f)FILTER data from (b) for SOLR collection number 2 > >> g)STORE data to solr > >> > >> But the store function fails when I run the combined script where as it > >> runs fine if I run scripts 1 and 2 separately. > >> > >> Any idea? > >> > >> Regards, > >> Vishnu > >> >
