Re: Solr indexing not taking all values from DB.
Hi Noble, Thanks for your reply Sorry that I was not able to reply back in time. I followed your suggestions and it is now indexing fine up to an extend. 1) if the query is taking values from a single table, the full import will show the expected count itself. And when i do http://localhost:8983/solr/select/?q=(*:*%20AND%20rowtype:customers)version=2.2start=0rows=100indent=onwt=json it will display all the results. 2) if the query is taking values from multiple table(select * from EMPLOYEE, CUSTOMER where EMPLOYEE.pro_id= CUSTOMER.prod_id) the full import will show the expected count itself. But, When I do http://localhost:8983/solr/select/?q=(*:*%20AND%20rowtype:both)version=2.2start=0rows=100indent=onwt=json it is not displaying all the values. Since for full import, str name=Total Rows Fetched**/str is getting as expected in both cases, i suppose the indexing is successfully done. Then why my search doesn't yield all the values in the second case? Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: now just do a normal full-import do not enable debug . I guess it should be just fine On Mon, Oct 13, 2008 at 1:20 PM, con [EMAIL PROTECTED] wrote: Thanks Nobble I tried in the debug mode with rows=100 and it is accepting all the result sets. So i suppose there is nothing wrong in the query. But I am not able to update the index since this is available only in the debug mode. Can you please give some suggestions based on this. thanks con Noble Paul നോബിള് नोब्ळ् wrote: in debug mode it writes only 10 because there is a rows parameter which is by default set to 10 make it 100 or so and you should be seeing all docs. But in non-debug mode there is no such parameter On Sun, Oct 12, 2008 at 11:00 PM, con [EMAIL PROTECTED] wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying
Re: Solr indexing not taking all values from DB.
in debug mode it writes only 10 because there is a rows parameter which is by default set to 10 make it 100 or so and you should be seeing all docs. But in non-debug mode there is no such parameter On Sun, Oct 12, 2008 at 11:00 PM, con [EMAIL PROTECTED] wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul
Re: Solr indexing not taking all values from DB.
Thanks Nobble I tried in the debug mode with rows=100 and it is accepting all the result sets. So i suppose there is nothing wrong in the query. But I am not able to update the index since this is available only in the debug mode. Can you please give some suggestions based on this. thanks con Noble Paul നോബിള് नोब्ळ् wrote: in debug mode it writes only 10 because there is a rows parameter which is by default set to 10 make it 100 or so and you should be seeing all docs. But in non-debug mode there is no such parameter On Sun, Oct 12, 2008 at 11:00 PM, con [EMAIL PROTECTED] wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context:
Re: Solr indexing not taking all values from DB.
now just do a normal full-import do not enable debug . I guess it should be just fine On Mon, Oct 13, 2008 at 1:20 PM, con [EMAIL PROTECTED] wrote: Thanks Nobble I tried in the debug mode with rows=100 and it is accepting all the result sets. So i suppose there is nothing wrong in the query. But I am not able to update the index since this is available only in the debug mode. Can you please give some suggestions based on this. thanks con Noble Paul നോബിള് नोब्ळ् wrote: in debug mode it writes only 10 because there is a rows parameter which is by default set to 10 make it 100 or so and you should be seeing all docs. But in non-debug mode there is no such parameter On Sun, Oct 12, 2008 at 11:00 PM, con [EMAIL PROTECTED] wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html
Re: Solr indexing not taking all values from DB.
template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul
Re: Solr indexing not taking all values from DB.
I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr indexing not taking all values from DB.
What is your uniqueKey set to? Could it be you have duplicates in your uniqueKey setup (thus producing only 10 rows in index)? - Jon On Oct 12, 2008, at 1:30 PM, con wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on ) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/? q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full- import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/ str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr indexing not taking all values from DB.
Hi Jon I have my unique key set to false. uniqueKey required=falseEMP_ID/uniqueKey. I tried with enabling the unique key. Still the same problem exists. Thanks con jonbaer wrote: What is your uniqueKey set to? Could it be you have duplicates in your uniqueKey setup (thus producing only 10 rows in index)? - Jon On Oct 12, 2008, at 1:30 PM, con wrote: I wrote a jdbc program to implement the same query. But it is returning all the responses, 25 nos. But the solr is still indexing only 10 rows. Is there any optimization settings by default in the solrconfig.xml that restricts the responses to 10 ? thanks con. Noble Paul നോബിള് नोब्ळ् wrote: template transformer does not eat up rows. I am almost sure that the query returns only 10 rows in that case. could you write a quick jdbc program and verify that (not the oralce client) everything else looks fine On Sat, Oct 11, 2008 at 4:52 PM, con [EMAIL PROTECTED] wrote: Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on ) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/? q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full- import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/ str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19943817.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19948508.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr indexing not taking all values from DB.
Hi Noble Thanks for your reply In my data-config.xml I have; entity name=employees transformer=TemplateTransformer query=Select EMP_ID , EMP_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from EMPLOYEES field column=rowtype template=employees / field column=EMP_ID name=EMP_ID / field column=EMP_NAME name=EMP_NAME / field column=COMMENT name=COMMENT / /entity entity name=customers transformer=TemplateTransformer query=Select CUST_ID , CUST_NAME , NVL (COMMENT ,'-Nil-') as COMMENT from CUSTOMERS field column=rowtype template=customers / field column=CUST_ID name=CUST_ID / field column=CUST_NAME name=CUST_NAME / field column=COMMENT name=COMMENT / /entity Whether this, TemplateTransformer, is the one that is restricting the resultset count to 10? Where can I find it out? I need this TemplateTransformer because I want to query the responses of either one of these at a time using the URL like, http://localhost:8983/solr/select/?q=(Bob%20AND%20rowtype:customers)version=2.2start=0rows=10indent=onwt=json I tried in the debug mode: (http://localhost:8983/solr/dataimport?command=full-importdebug=onverbose=on) , But it is not all mentioning anything after the 10th document. Thanks and regards con Noble Paul നോബിള് नोब्ळ् wrote: The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19931736.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr indexing not taking all values from DB.
The DIH status says 10 rows which means only 10 rows got fetched for that query. Do you have any custom transformers which eats up rows? Try the debug page of DIH and see what is happening to the rest of the rows. On Fri, Oct 10, 2008 at 5:32 PM, con [EMAIL PROTECTED] wrote: A simple question: I performed the following steps to index data from a oracle db to solr index and then search: a) I have the configurations for indexing data from a oracle db b) started the server. c) Done a full-import: http://localhost:8983/solr/dataimport?command=full-import But when I do a search using http://localhost:8983/solr/select/?q= Not all the result sets that matches the search string are displayed. 1) Is the above steps enough for getting db values to solr index? My configurations (data-config.xml and schema.xml )are quite correct because I am getting SOME of the result sets as search result(not all). 2) Is there some value in sorconfig.xml, or some other files that limits the number of items being indexed? [For the time being I have only a few hundreds of records in my db. ] The query that I am specifying in data-config yields around 25 results if i execute it in a oracle client, where as the status of full-import is something like: str name=statusidle/str str name=importResponseConfiguration Re-loaded sucessfully/str lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-10-10 17:29:03/str str name=Time taken 0:0:0.513/str /lst -- View this message in context: http://www.nabble.com/Solr-indexing-not-taking-all-values-from-DB.-tp19916938p19916938.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul
