Re: Unable to index documents using DataImportHandler with MSSQL
Hah, I've just come on here to suggest you do the same thing! Thanks for getting back to me - and interesting we both came up with the same solution! Now I have the problem that running a delta update updates the 'dataimport.properties' file - but then just re-fetches all the data regardless! Weird! On Mon, Nov 28, 2011 at 11:59 AM, Husain, Yavar wrote: > Hi Ian > > I downloaded and build latest Solr (3.4) from sources and finally hit > following line of code in Solr (where I put my debug statement) : > > if(url != null){ > LOG.info("Yavar: getting handle to driver manager:"); > c = DriverManager.getConnection(url, initProps); > LOG.info("Yavar: got handle to driver manager:"); > } > > The call to Driver Manager was not returning. Here was the error!! The Driver > we were using was Microsoft Type 4 JDBC driver for SQL Server. I downloaded > another driver called jTDS jDBC driver and installed that. Problem got > fixed!!! > > So please follow the following steps: > > 1. Download jTDS jDBC driver from http://jtds.sourceforge.net/ > 2. Put the driver jar file into your Solr/lib directory where you had put > Microsoft JDBC driver. > 3. In the data-config.xml use this statement: > driver="net.sourceforge.jtds.jdbc.Driver" > 4. Also in data-config.xml mention url like this: > "url="jdbc:jTDS:sqlserver://localhost:1433;databaseName=XXX" > 5. Now run your indexing. > > It should solve the problem. > > Regards, > Yavar > > -Original Message- > From: Ian Grainger [mailto:i...@isfluent.com] > Sent: Monday, November 28, 2011 4:11 PM > To: Husain, Yavar > Cc: solr-user@lucene.apache.org > Subject: Re: Unable to index documents using DataImportHandler with MSSQL > > Right. > This is REALLY weird - I've now started from scratch on another > machine (this time Windows 7), and got _exactly_ the same problem !? > > > On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar wrote: >> Hi Ian >> >> I am having exactly the same problem what you are having on Win 7 and 2008 >> Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html >> >> I still have not received any replies which could solve my problem till now. >> Please do let me know if you have arrived at some solution for your problem. >> >> Thanks. >> >> Regards, >> Yavar >> >> -Original Message- >> From: Ian Grainger [mailto:i...@isfluent.com] >> Sent: Friday, November 25, 2011 10:59 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Unable to index documents using DataImportHandler with MSSQL >> >> Update on this: I've established: >> * It's not a problem in the DB (I can index from this DB into a Solr >> instance on another server) >> * It's not Tomcat (I get the same problem in Jetty) >> * It's not the schema (I have simplified it to one field) >> >> That leaves SolrConfig.xml and data-config. >> >> Only thing changed in SolrConfig.xml is adding: >> >> > regex="apache-solr-cell-\d.*\.jar" /> >> > regex="apache-solr-clustering-\d.*\.jar" /> >> > regex="apache-solr-dataimporthandler-\d.*\.jar" /> >> > class="org.apache.solr.handler.dataimport.DataImportHandler"> >> >> > name="config">D:/Software/Solr/example/solr/conf/data-config.xml >> >> >> >> And data-config.xml is pretty much as attached - except simpler. >> >> Any help or any advice on how to diagnose would be appreciated! >> >> >> On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger wrote: >>> Hi I have copied my Solr config from a working Windows server to a new >>> one, and it can't seem to run an import. >>> >>> They're both using win server 2008 and SQL 2008R2. This is the data >>> importer config >>> >>> >>> >> driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" >>> url="jdbc:sqlserver://localhost;databaseName=DB" >>> user="Solr" >>> password="pwd"/> >>> >>> >> query="EXEC SOLR_COMPANY_SEARCH_DATA" >>> deltaImportQuery="SELECT * FROM Company_Search_Data WHERE >>> [key]='${dataimporter.delta.key}'" >>> deltaQuery="SELECT [key] FROM Company_Search_Data W
RE: Unable to index documents using DataImportHandler with MSSQL
Hi Ian I downloaded and build latest Solr (3.4) from sources and finally hit following line of code in Solr (where I put my debug statement) : if(url != null){ LOG.info("Yavar: getting handle to driver manager:"); c = DriverManager.getConnection(url, initProps); LOG.info("Yavar: got handle to driver manager:"); } The call to Driver Manager was not returning. Here was the error!! The Driver we were using was Microsoft Type 4 JDBC driver for SQL Server. I downloaded another driver called jTDS jDBC driver and installed that. Problem got fixed!!! So please follow the following steps: 1. Download jTDS jDBC driver from http://jtds.sourceforge.net/ 2. Put the driver jar file into your Solr/lib directory where you had put Microsoft JDBC driver. 3. In the data-config.xml use this statement: driver="net.sourceforge.jtds.jdbc.Driver" 4. Also in data-config.xml mention url like this: "url="jdbc:jTDS:sqlserver://localhost:1433;databaseName=XXX" 5. Now run your indexing. It should solve the problem. Regards, Yavar -Original Message- From: Ian Grainger [mailto:i...@isfluent.com] Sent: Monday, November 28, 2011 4:11 PM To: Husain, Yavar Cc: solr-user@lucene.apache.org Subject: Re: Unable to index documents using DataImportHandler with MSSQL Right. This is REALLY weird - I've now started from scratch on another machine (this time Windows 7), and got _exactly_ the same problem !? On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar wrote: > Hi Ian > > I am having exactly the same problem what you are having on Win 7 and 2008 > Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html > > I still have not received any replies which could solve my problem till now. > Please do let me know if you have arrived at some solution for your problem. > > Thanks. > > Regards, > Yavar > > -Original Message- > From: Ian Grainger [mailto:i...@isfluent.com] > Sent: Friday, November 25, 2011 10:59 PM > To: solr-user@lucene.apache.org > Subject: Re: Unable to index documents using DataImportHandler with MSSQL > > Update on this: I've established: > * It's not a problem in the DB (I can index from this DB into a Solr > instance on another server) > * It's not Tomcat (I get the same problem in Jetty) > * It's not the schema (I have simplified it to one field) > > That leaves SolrConfig.xml and data-config. > > Only thing changed in SolrConfig.xml is adding: > > regex="apache-solr-cell-\d.*\.jar" /> > regex="apache-solr-clustering-\d.*\.jar" /> > regex="apache-solr-dataimporthandler-\d.*\.jar" /> > class="org.apache.solr.handler.dataimport.DataImportHandler"> > > D:/Software/Solr/example/solr/conf/data-config.xml > > > > And data-config.xml is pretty much as attached - except simpler. > > Any help or any advice on how to diagnose would be appreciated! > > > On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger wrote: >> Hi I have copied my Solr config from a working Windows server to a new >> one, and it can't seem to run an import. >> >> They're both using win server 2008 and SQL 2008R2. This is the data >> importer config >> >> >> > driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" >> url="jdbc:sqlserver://localhost;databaseName=DB" >> user="Solr" >> password="pwd"/> >> >> > query="EXEC SOLR_COMPANY_SEARCH_DATA" >> deltaImportQuery="SELECT * FROM Company_Search_Data WHERE >> [key]='${dataimporter.delta.key}'" >> deltaQuery="SELECT [key] FROM Company_Search_Data WHERE modify_dt >>> '${dataimporter.last_index_time}'"> >> > name="WorkDesc_Comments_Split" /> >> > /> >> >> >> >> >> I can use MS SQL Profiler to watch the Solr user log in successfully, >> but then nothing. It doesn't seem to even try and execute the stored >> procedure. Any ideas why this would be working one server and not on >> another? >> >> FTR the only thing in the tomcat catalina log is: >> >> org.apache.solr.handler.dataimport.JdbcDataSource$1 call >> INFO: Creating a connection for entity data with URL: >> jdbc:sqlserver://localhost;databaseName=CATLive >> >> -- >> Ian >> >> i...@isfluent.com >> +44 (0)1223 257903 >> > > > > -- > Ian > > i...@isfluent.com > +44 (0)1223 257903 >
Re: Unable to index documents using DataImportHandler with MSSQL
Right. This is REALLY weird - I've now started from scratch on another machine (this time Windows 7), and got _exactly_ the same problem !? On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar wrote: > Hi Ian > > I am having exactly the same problem what you are having on Win 7 and 2008 > Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html > > I still have not received any replies which could solve my problem till now. > Please do let me know if you have arrived at some solution for your problem. > > Thanks. > > Regards, > Yavar > > -Original Message- > From: Ian Grainger [mailto:i...@isfluent.com] > Sent: Friday, November 25, 2011 10:59 PM > To: solr-user@lucene.apache.org > Subject: Re: Unable to index documents using DataImportHandler with MSSQL > > Update on this: I've established: > * It's not a problem in the DB (I can index from this DB into a Solr > instance on another server) > * It's not Tomcat (I get the same problem in Jetty) > * It's not the schema (I have simplified it to one field) > > That leaves SolrConfig.xml and data-config. > > Only thing changed in SolrConfig.xml is adding: > > regex="apache-solr-cell-\d.*\.jar" /> > regex="apache-solr-clustering-\d.*\.jar" /> > regex="apache-solr-dataimporthandler-\d.*\.jar" /> > class="org.apache.solr.handler.dataimport.DataImportHandler"> > > D:/Software/Solr/example/solr/conf/data-config.xml > > > > And data-config.xml is pretty much as attached - except simpler. > > Any help or any advice on how to diagnose would be appreciated! > > > On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger wrote: >> Hi I have copied my Solr config from a working Windows server to a new >> one, and it can't seem to run an import. >> >> They're both using win server 2008 and SQL 2008R2. This is the data >> importer config >> >> >> > driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" >> url="jdbc:sqlserver://localhost;databaseName=DB" >> user="Solr" >> password="pwd"/> >> >> > query="EXEC SOLR_COMPANY_SEARCH_DATA" >> deltaImportQuery="SELECT * FROM Company_Search_Data WHERE >> [key]='${dataimporter.delta.key}'" >> deltaQuery="SELECT [key] FROM Company_Search_Data WHERE modify_dt >>> '${dataimporter.last_index_time}'"> >> > name="WorkDesc_Comments_Split" /> >> > /> >> >> >> >> >> I can use MS SQL Profiler to watch the Solr user log in successfully, >> but then nothing. It doesn't seem to even try and execute the stored >> procedure. Any ideas why this would be working one server and not on >> another? >> >> FTR the only thing in the tomcat catalina log is: >> >> org.apache.solr.handler.dataimport.JdbcDataSource$1 call >> INFO: Creating a connection for entity data with URL: >> jdbc:sqlserver://localhost;databaseName=CATLive >> >> -- >> Ian >> >> i...@isfluent.com >> +44 (0)1223 257903 >> > > > > -- > Ian > > i...@isfluent.com > +44 (0)1223 257903 > ** > This message may contain confidential or proprietary information intended > only for the use of the > addressee(s) named above or may contain information that is legally > privileged. If you are > not the intended addressee, or the person responsible for delivering it to > the intended addressee, > you are hereby notified that reading, disseminating, distributing or copying > this message is strictly > prohibited. If you have received this message by mistake, please immediately > notify us by > replying to the message and delete the original message and any copies > immediately thereafter. > > Thank you.- > ** > FAFLD > > -- Ian i...@isfluent.com +44 (0)1223 257903
RE: Unable to index documents using DataImportHandler with MSSQL
Hi Ian I am having exactly the same problem what you are having on Win 7 and 2008 Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html I still have not received any replies which could solve my problem till now. Please do let me know if you have arrived at some solution for your problem. Thanks. Regards, Yavar -Original Message- From: Ian Grainger [mailto:i...@isfluent.com] Sent: Friday, November 25, 2011 10:59 PM To: solr-user@lucene.apache.org Subject: Re: Unable to index documents using DataImportHandler with MSSQL Update on this: I've established: * It's not a problem in the DB (I can index from this DB into a Solr instance on another server) * It's not Tomcat (I get the same problem in Jetty) * It's not the schema (I have simplified it to one field) That leaves SolrConfig.xml and data-config. Only thing changed in SolrConfig.xml is adding: D:/Software/Solr/example/solr/conf/data-config.xml And data-config.xml is pretty much as attached - except simpler. Any help or any advice on how to diagnose would be appreciated! On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger wrote: > Hi I have copied my Solr config from a working Windows server to a new > one, and it can't seem to run an import. > > They're both using win server 2008 and SQL 2008R2. This is the data > importer config > > > driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" > url="jdbc:sqlserver://localhost;databaseName=DB" > user="Solr" > password="pwd"/> > > query="EXEC SOLR_COMPANY_SEARCH_DATA" > deltaImportQuery="SELECT * FROM Company_Search_Data WHERE > [key]='${dataimporter.delta.key}'" > deltaQuery="SELECT [key] FROM Company_Search_Data WHERE modify_dt >> '${dataimporter.last_index_time}'"> > name="WorkDesc_Comments_Split" /> > /> > > > > > I can use MS SQL Profiler to watch the Solr user log in successfully, > but then nothing. It doesn't seem to even try and execute the stored > procedure. Any ideas why this would be working one server and not on > another? > > FTR the only thing in the tomcat catalina log is: > > org.apache.solr.handler.dataimport.JdbcDataSource$1 call > INFO: Creating a connection for entity data with URL: > jdbc:sqlserver://localhost;databaseName=CATLive > > -- > Ian > > i...@isfluent.com > +44 (0)1223 257903 > -- Ian i...@isfluent.com +44 (0)1223 257903 ** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you.- ** FAFLD
Re: Unable to index documents using DataImportHandler with MSSQL
Update on this: I've established: * It's not a problem in the DB (I can index from this DB into a Solr instance on another server) * It's not Tomcat (I get the same problem in Jetty) * It's not the schema (I have simplified it to one field) That leaves SolrConfig.xml and data-config. Only thing changed in SolrConfig.xml is adding: D:/Software/Solr/example/solr/conf/data-config.xml And data-config.xml is pretty much as attached - except simpler. Any help or any advice on how to diagnose would be appreciated! On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger wrote: > Hi I have copied my Solr config from a working Windows server to a new > one, and it can't seem to run an import. > > They're both using win server 2008 and SQL 2008R2. This is the data > importer config > > > driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" > url="jdbc:sqlserver://localhost;databaseName=DB" > user="Solr" > password="pwd"/> > > query="EXEC SOLR_COMPANY_SEARCH_DATA" > deltaImportQuery="SELECT * FROM Company_Search_Data WHERE > [key]='${dataimporter.delta.key}'" > deltaQuery="SELECT [key] FROM Company_Search_Data WHERE modify_dt >> '${dataimporter.last_index_time}'"> > name="WorkDesc_Comments_Split" /> > /> > > > > > I can use MS SQL Profiler to watch the Solr user log in successfully, > but then nothing. It doesn't seem to even try and execute the stored > procedure. Any ideas why this would be working one server and not on > another? > > FTR the only thing in the tomcat catalina log is: > > org.apache.solr.handler.dataimport.JdbcDataSource$1 call > INFO: Creating a connection for entity data with URL: > jdbc:sqlserver://localhost;databaseName=CATLive > > -- > Ian > > i...@isfluent.com > +44 (0)1223 257903 > -- Ian i...@isfluent.com +44 (0)1223 257903
Unable to index documents using DataImportHandler with MSSQL
Hi I have copied my Solr config from a working Windows server to a new one, and it can't seem to run an import. They're both using win server 2008 and SQL 2008R2. This is the data importer config I can use MS SQL Profiler to watch the Solr user log in successfully, but then nothing. It doesn't seem to even try and execute the stored procedure. Any ideas why this would be working one server and not on another? FTR the only thing in the tomcat catalina log is: org.apache.solr.handler.dataimport.JdbcDataSource$1 call INFO: Creating a connection for entity data with URL: jdbc:sqlserver://localhost;databaseName=CATLive -- Ian i...@isfluent.com +44 (0)1223 257903