Hello,
I briefly described the similar problems at
http://blog.griddynamics.com/2015/07/how-to-import-structured-data-into-solr.html
Let me know if you have further questions
On Thu, Jul 23, 2015 at 7:55 PM, cbuxbaum <cbuxb...@tradestonesoftware.com>
wrote:
> That's OK, I have determined that caching is not relevant to our use case.
> However, I have a question about the full import queries that we are using:
>
> Here is the SQL from the top level entity:
>
> query="SELECT DISTINCT 'LEAP_PARTY' AS
> DOCUMENT_TYPE, VPARTY.OWNER AS PARTY_OWNER, VPARTY.PARTY_ID AS
> PARTY_PARTY_ID, VPARTY.PARTY_ID AS LEAP_PARTY_ID,VPARTY.OWNER AS
> LEAP_PARTY_OWNER FROM VPARTY"
>
> The values from the top level entity are being inserted one by one into the
> child entity SQL
>
> query="SELECT DISTINCT OFFER.REQUEST_NO AS OFFER_REQUEST_NO,OFFER.OWNER AS
> OFFER_OWNER,OFFER.OFFER_NO AS OFFER_OFFER_NO,OFFER.MODIFY_TS FROM OFFER
> INNER JOIN (SELECT DISTINCT REQUEST_NO, OWNER, OFFER_NO FROM OFFER WHERE
> SUPPLIER='${LEAP_PARTY.LEAP_PARTY_ID}' AND
> OWNER='${LEAP_PARTY.LEAP_PARTY_OWNER}') SQ ON
> OFFER.REQUEST_NO=SQ.REQUEST_NO
> AND OFFER.OWNER=SQ.OWNER AND SQ.OFFER_NO = OFFER.OFFER_NO ORDER BY
> OFFER.MODIFY_TS DESC"
>
> Say we have 1000000 party records. Then the child SQL will be run 1000000
> times (once for each party record). Isn't there a way to just run the
> child
> SQL on all of the party records at once with a join, using a GROUP BY and
> ORDER BY on the PARTY_ID? Then the results from that query could easily be
> placed in SOLR according to the primary key (party_id). Is there some part
> of the Data Import Handler that operates that way?
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/cache-implemetation-tp4218825p4218851.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mkhlud...@griddynamics.com>
