So disabling convertType does provide a workaround for my problem with datetime column. But the problem still exists when convertType is enabled because DIH is not doing the conversion correctly for a solr date field. Solr date field does have a time portion but java.sql.Date does not. So DIH should not be calling ResultSet.getDate() for a solr date field. It should really be calling ResultSet.getTimestamp() instead. Is the fix this simple? Am I missing anything?
If the fix is this simple I can submit and commit a patch to DIH. Bill On Sat, Jun 29, 2013 at 12:13 PM, Bill Au <bill.w...@gmail.com> wrote: > Setting convertType=false does solve the datetime issue. But there are > now other columns that were working before but not working now. Since I > have already done some research into the datetime to date issue and not > been able to find a solution, I think I will have to keep convertType set > to false and deal with the other column type that are not working now. > > Thanks for your help. > > Bill > > > On Sat, Jun 29, 2013 at 10:24 AM, Bill Au <bill.w...@gmail.com> wrote: > >> I just double check my config. We are using convertType=true. Someone >> else came up with the config so I am not sure why we are using it. I will >> try with it set to false to see if something else will break. Thanks for >> pointing that out. >> >> This is my first time using DIH. I really like what I have seen so far. >> >> Bill >> >> >> On Sat, Jun 29, 2013 at 1:45 AM, Shalin Shekhar Mangar < >> shalinman...@gmail.com> wrote: >> >>> The default in JdbcDataSource is to use ResultSet.getObject which >>> returns the underlying database's type. The type specific methods in >>> ResultSet are not invoked unless you are using convertType="true". >>> >>> Is MySQL actually returning java.sql.Timestamp objects? >>> >>> On Sat, Jun 29, 2013 at 5:22 AM, Bill Au <bill.w...@gmail.com> wrote: >>> > I am running Solr 4.3.0, using DIH to import data from MySQL. I am >>> running >>> > into a very strange problem where data from a datetime column being >>> > imported with the right date but the time is 00:00:00. I tried using >>> SQL >>> > DATE_FORMAT() and also DIH DateFormatTransformer but nothing works. >>> The >>> > raw debug response of DIH, it looks like the time porting of the >>> datetime >>> > data is already 00:00:00 in Solr jdbc query result. >>> > >>> > So I looked at the source code of DIH JdbcDataSource class. It is >>> using >>> > java.sql.ResultSet and its getDate() method to handle date column. The >>> > getDate() method returns java.sql.Date. The java api doc for >>> java.sql.Date >>> > >>> > http://docs.oracle.com/javase/6/docs/api/java/sql/Date.html >>> > >>> > states that: >>> > >>> > "To conform with the definition of SQL DATE, the millisecond values >>> wrapped >>> > by a java.sql.Date instance must be 'normalized' by setting the hours, >>> > minutes, seconds, and milliseconds to zero in the particular time zone >>> with >>> > which the instance is associated." >>> > >>> > This seems to be describing exactly my problem. Has anyone else notice >>> > this problem? Has anyone use DIH to index SQL datetime successfully? >>> If >>> > so can you send me the relevant portion of the DIH config? >>> > >>> > Bill >>> >>> >>> >>> -- >>> Regards, >>> Shalin Shekhar Mangar. >>> >> >> >
