Re: retreive all the fields in join

[Ahmet Arslan]

Hi Karanti,

I was thinking the same. DocTransformer looks a good candidate for such 
implementation. What do you think? May be we can implement this and contribute 
back?




On Saturday, May 17, 2014 9:13 AM, Kranti Parisa  
wrote:
Aman,

The option you have got is:
- write custom components like request handlers, collectors & response
writers..
- first you would do the join, then apply the pagination
- you will get the docList in response writer, you would need to make a
call to the second core (you could be smart to use the FQs so that you
could hit the cache and hence the second call will be fast) and fetch the
documents
- use them for building the response

out of the box Solr won't do this for you..

Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa




On Mon, May 12, 2014 at 7:05 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> On Sun, May 11, 2014 at 12:14 PM, Aman Tandon  >wrote:
>
> > Is it possible?
>
>
> no.
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> 
>  
>

Re: retreive all the fields in join

[Kranti Parisa]

Aman,

The option you have got is:
- write custom components like request handlers, collectors & response
writers..
- first you would do the join, then apply the pagination
- you will get the docList in response writer, you would need to make a
call to the second core (you could be smart to use the FQs so that you
could hit the cache and hence the second call will be fast) and fetch the
documents
- use them for building the response

out of the box Solr won't do this for you..

Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa



On Mon, May 12, 2014 at 7:05 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> On Sun, May 11, 2014 at 12:14 PM, Aman Tandon  >wrote:
>
> > Is it possible?
>
>
> no.
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> 
>  
>

Re: retreive all the fields in join

[Mikhail Khludnev]

On Sun, May 11, 2014 at 12:14 PM, Aman Tandon wrote:

> Is it possible?


no.


-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

Re: retreive all the fields in join

[Aman Tandon]

Thanks Walter yeah we are now trying to use the external file field as
mentioned by you.
On May 12, 2014 11:11 PM, "Walter Underwood"  wrote:

> Top management has given requirements that force a broken design. They are
> requiring something that is impossible with Solr.
>
> 1. Flatten the data. You get one table, no joins.
>
> 2. 12M records is not a big Solr index. That should work fine.
>
> 3. If the supplier activity points are updated frequently, you could use
> an external file field for those, but they still need to be flattened.
>
> wunder
>
> On May 12, 2014, at 7:21 AM, Aman Tandon  wrote:
>
> > Yeah i understand but i got the requirement from the top management,
> > requirements are:
> > core1:  in this we want to keep the supplier activity points
> > case 2: we want to boost those records which are present in core1 by the
> > amount of supplier activity points.
> >
> > I know we can keep that supplier score in same core but this requires the
> > full indexing of 12M records and suppliers are of about 1lacs which won't
> > cost much.
> >
> > With Regards
> > Aman
> >
> >
> > On Mon, May 12, 2014 at 7:44 PM, Erick Erickson  >wrote:
> >
> >> Any time you find yourself trying to use Solr like a DB, stop.
> >>
> >> Solr joins are _not_ DB joins, the data from the "from" core is not
> >> returned (I think there are a few special cases where you can make
> >> this happen though).
> >>
> >> Try denormalizing your data if at all possible, that's what Solr docs
> >> best... search single records.
> >>
> >> Best,
> >> Erick
> >>
> >> On Sun, May 11, 2014 at 6:40 PM, Aman Tandon 
> >> wrote:
> >>> please help me out here!!
> >>>
> >>> With Regards
> >>> Aman Tandon
> >>>
> >>>
> >>> On Sun, May 11, 2014 at 1:44 PM, Aman Tandon  >>> wrote:
> >>>
>  Hi,
> 
>  Is there a way possible to retrieve all the fields present in both the
>  cores(core 1 and core2).
> 
>  e.g.
>  core1: {id:111,name: "abc" }
> 
>  core2: {page:17, type: "fiction"}
> 
>  I want is that, on querying both the cores I want to retrieve the
> >> results
>  containing all the 4 fields, fields id, name from core1 and page, type
> >> from
>  core2. Is it possible?
> 
>  With Regards
>  Aman Tandon
> 
> >>
>
> --
> Walter Underwood
> wun...@wunderwood.org
>
>
>
>

Re: retreive all the fields in join

[David Philip]

Hi Aman,

I think it is possible.

1. Use fl parameter.
2. Add all the 4 fields in both the schemas[schemas of core 1 and 2].
3. While querying use &fl=id,name,type,page.

It will return all the fields. The document that has no data for this
field, the field will be an empty string.
Ex:  {id:111,name: "abc", type:"", page""}
{page:17, type: "fiction", id:"", name:""}


Thanks







On Mon, May 12, 2014 at 7:10 AM, Aman Tandon wrote:

> please help me out here!!
>
> With Regards
> Aman Tandon
>
>
> On Sun, May 11, 2014 at 1:44 PM, Aman Tandon  >wrote:
>
> > Hi,
> >
> > Is there a way possible to retrieve all the fields present in both the
> > cores(core 1 and core2).
> >
> > e.g.
> > core1: {id:111,name: "abc" }
> >
> > core2: {page:17, type: "fiction"}
> >
> > I want is that, on querying both the cores I want to retrieve the results
> > containing all the 4 fields, fields id, name from core1 and page, type
> from
> > core2. Is it possible?
> >
> > With Regards
> > Aman Tandon
> >
>

Re: retreive all the fields in join

[Erick Erickson]

Well, you might explain to management that this isn't what Solr is
built to do.

12M records is actually quite small by Solr standards. I'd recommend
de-normalizing and demonstrating a working solution that uses Solr
as it's intended rather than how senior management wishes it were.
It'll be a lot easier/faster/more scalable I'd guess. and with 12M records,
it will very likely fit on commodity hardware.

I'm afraid I can't really help you otherwise..

Best
Erick

On Mon, May 12, 2014 at 7:21 AM, Aman Tandon  wrote:
> Yeah i understand but i got the requirement from the top management,
> requirements are:
> core1:  in this we want to keep the supplier activity points
> case 2: we want to boost those records which are present in core1 by the
> amount of supplier activity points.
>
> I know we can keep that supplier score in same core but this requires the
> full indexing of 12M records and suppliers are of about 1lacs which won't
> cost much.
>
> With Regards
> Aman
>
>
> On Mon, May 12, 2014 at 7:44 PM, Erick Erickson 
> wrote:
>
>> Any time you find yourself trying to use Solr like a DB, stop.
>>
>> Solr joins are _not_ DB joins, the data from the "from" core is not
>> returned (I think there are a few special cases where you can make
>> this happen though).
>>
>> Try denormalizing your data if at all possible, that's what Solr docs
>> best... search single records.
>>
>> Best,
>> Erick
>>
>> On Sun, May 11, 2014 at 6:40 PM, Aman Tandon 
>> wrote:
>> > please help me out here!!
>> >
>> > With Regards
>> > Aman Tandon
>> >
>> >
>> > On Sun, May 11, 2014 at 1:44 PM, Aman Tandon > >wrote:
>> >
>> >> Hi,
>> >>
>> >> Is there a way possible to retrieve all the fields present in both the
>> >> cores(core 1 and core2).
>> >>
>> >> e.g.
>> >> core1: {id:111,name: "abc" }
>> >>
>> >> core2: {page:17, type: "fiction"}
>> >>
>> >> I want is that, on querying both the cores I want to retrieve the
>> results
>> >> containing all the 4 fields, fields id, name from core1 and page, type
>> from
>> >> core2. Is it possible?
>> >>
>> >> With Regards
>> >> Aman Tandon
>> >>
>>

Re: retreive all the fields in join

[Walter Underwood]

Top management has given requirements that force a broken design. They are 
requiring something that is impossible with Solr.

1. Flatten the data. You get one table, no joins.

2. 12M records is not a big Solr index. That should work fine.

3. If the supplier activity points are updated frequently, you could use an 
external file field for those, but they still need to be flattened. 

wunder

On May 12, 2014, at 7:21 AM, Aman Tandon  wrote:

> Yeah i understand but i got the requirement from the top management,
> requirements are:
> core1:  in this we want to keep the supplier activity points
> case 2: we want to boost those records which are present in core1 by the
> amount of supplier activity points.
> 
> I know we can keep that supplier score in same core but this requires the
> full indexing of 12M records and suppliers are of about 1lacs which won't
> cost much.
> 
> With Regards
> Aman
> 
> 
> On Mon, May 12, 2014 at 7:44 PM, Erick Erickson 
> wrote:
> 
>> Any time you find yourself trying to use Solr like a DB, stop.
>> 
>> Solr joins are _not_ DB joins, the data from the "from" core is not
>> returned (I think there are a few special cases where you can make
>> this happen though).
>> 
>> Try denormalizing your data if at all possible, that's what Solr docs
>> best... search single records.
>> 
>> Best,
>> Erick
>> 
>> On Sun, May 11, 2014 at 6:40 PM, Aman Tandon 
>> wrote:
>>> please help me out here!!
>>> 
>>> With Regards
>>> Aman Tandon
>>> 
>>> 
>>> On Sun, May 11, 2014 at 1:44 PM, Aman Tandon >> wrote:
>>> 
 Hi,
 
 Is there a way possible to retrieve all the fields present in both the
 cores(core 1 and core2).
 
 e.g.
 core1: {id:111,name: "abc" }
 
 core2: {page:17, type: "fiction"}
 
 I want is that, on querying both the cores I want to retrieve the
>> results
 containing all the 4 fields, fields id, name from core1 and page, type
>> from
 core2. Is it possible?
 
 With Regards
 Aman Tandon
 
>> 

--
Walter Underwood
wun...@wunderwood.org

Re: retreive all the fields in join

[Aman Tandon]

Yeah i understand but i got the requirement from the top management,
requirements are:
core1:  in this we want to keep the supplier activity points
case 2: we want to boost those records which are present in core1 by the
amount of supplier activity points.

I know we can keep that supplier score in same core but this requires the
full indexing of 12M records and suppliers are of about 1lacs which won't
cost much.

With Regards
Aman


On Mon, May 12, 2014 at 7:44 PM, Erick Erickson wrote:

> Any time you find yourself trying to use Solr like a DB, stop.
>
> Solr joins are _not_ DB joins, the data from the "from" core is not
> returned (I think there are a few special cases where you can make
> this happen though).
>
> Try denormalizing your data if at all possible, that's what Solr docs
> best... search single records.
>
> Best,
> Erick
>
> On Sun, May 11, 2014 at 6:40 PM, Aman Tandon 
> wrote:
> > please help me out here!!
> >
> > With Regards
> > Aman Tandon
> >
> >
> > On Sun, May 11, 2014 at 1:44 PM, Aman Tandon  >wrote:
> >
> >> Hi,
> >>
> >> Is there a way possible to retrieve all the fields present in both the
> >> cores(core 1 and core2).
> >>
> >> e.g.
> >> core1: {id:111,name: "abc" }
> >>
> >> core2: {page:17, type: "fiction"}
> >>
> >> I want is that, on querying both the cores I want to retrieve the
> results
> >> containing all the 4 fields, fields id, name from core1 and page, type
> from
> >> core2. Is it possible?
> >>
> >> With Regards
> >> Aman Tandon
> >>
>

Re: retreive all the fields in join

[Erick Erickson]

Any time you find yourself trying to use Solr like a DB, stop.

Solr joins are _not_ DB joins, the data from the "from" core is not
returned (I think there are a few special cases where you can make
this happen though).

Try denormalizing your data if at all possible, that's what Solr docs
best... search single records.

Best,
Erick

On Sun, May 11, 2014 at 6:40 PM, Aman Tandon  wrote:
> please help me out here!!
>
> With Regards
> Aman Tandon
>
>
> On Sun, May 11, 2014 at 1:44 PM, Aman Tandon wrote:
>
>> Hi,
>>
>> Is there a way possible to retrieve all the fields present in both the
>> cores(core 1 and core2).
>>
>> e.g.
>> core1: {id:111,name: "abc" }
>>
>> core2: {page:17, type: "fiction"}
>>
>> I want is that, on querying both the cores I want to retrieve the results
>> containing all the 4 fields, fields id, name from core1 and page, type from
>> core2. Is it possible?
>>
>> With Regards
>> Aman Tandon
>>

Re: retreive all the fields in join

[Aman Tandon]

please help me out here!!

With Regards
Aman Tandon


On Sun, May 11, 2014 at 1:44 PM, Aman Tandon wrote:

> Hi,
>
> Is there a way possible to retrieve all the fields present in both the
> cores(core 1 and core2).
>
> e.g.
> core1: {id:111,name: "abc" }
>
> core2: {page:17, type: "fiction"}
>
> I want is that, on querying both the cores I want to retrieve the results
> containing all the 4 fields, fields id, name from core1 and page, type from
> core2. Is it possible?
>
> With Regards
> Aman Tandon
>

retreive all the fields in join

[Aman Tandon]

Hi,

Is there a way possible to retrieve all the fields present in both the
cores(core 1 and core2).

e.g.
core1: {id:111,name: "abc" }

core2: {page:17, type: "fiction"}

I want is that, on querying both the cores I want to retrieve the results
containing all the 4 fields, fields id, name from core1 and page, type from
core2. Is it possible?

With Regards
Aman Tandon