Michael, Please update your code, I just committed a fix for the bug you reported (commit c89f8a39fa1faa34374d8a6e92ae9c2467deeda7). Please test this with your code as well.
With regards to the 64bit FS issue, it would be nice if you could provide a test and a fix for this (using some #define hacks or our cmake scripts). I'm just so swamped at the moment that I'm afraid I won't be able to do this myself anytime soon. I can provide pointers if necessary. Itamar. -----Original Message----- From: Michael Levin [mailto:mele...@stanford.edu] Sent: Thursday, November 05, 2009 10:07 PM To: clucene-developers@lists.sourceforge.net Subject: Re: [CLucene-dev] Cannot write >2gb index file Itamar Syn-Hershko wrote: > Michael, > > Thanks. However, the O_LARGEFILE flag isn't supported on Windows (for > all versions as far as I can tell), and might not be supported on > other Linux distributions, and on Mac. That being said, this is > something we need to test and find a solution for (probably another > cmake check). I'm no cross-platform wiz, so anyone willing to take this up please be my guest. Thanks for looking into this. It's true that Windows doesn't support this flag. I believe on Windows you don't have the open64 style functions either so you must use the Windows API equivalents (e.g. CreateFile()). I can see how inconvenient this can get though as you probably won't be able to get away with a platform-agnostic _cl_open() function... > On that note, I haven't tested your code to see if it crashes on > Windows as well. Might be interesting to see tho. I believe it should though definitely something worth testing. > I see no reason why this will break StandardAnalyzer. Can you provide > more details please? Honestly I don't know why it would either. It may not have been my changes actually, cel tix44 just sent out an email saying the last two commits broke the StandardAnalyzer ("[CLucene-dev] StandardAnalyzer broken - GIT 364c21b6c3f54fbb90df223621b660197366fb93"). I was using git to switch from my branch to head and I thought that the StandardAnalyzer was working in HEAD though I may have made a mistake... The exact problem is missing norms. When I generate a new file and open it in Luke or query with CLucene only the first term processed with StandardAnalyzer has a norm (of 1.0) and every other term has zero norm and won't appear in search results. I am currently using StopAnalyzer and it works fine so I wonder if the problem is somewhere in StandardFilter? > Itamar. > > -----Original Message----- > From: Michael Levin [mailto:mele...@stanford.edu] > Sent: Thursday, November 05, 2009 11:55 AM > To: clucene-developers@lists.sourceforge.net > Subject: Re: [CLucene-dev] Cannot write >2gb index file > > (Sorry for the email spam...) > > This change seems to break StandardAnalyzer though. I can't figure out > why... all of the other analyzers work fine. :-\ > > Michael Levin wrote: >> Really easy fix, please add "O_LARGEFILE" flag everywhere _cl_open() >> is used. E.g.: >> >> _cl_open(buffer, O_RDWR, _S_IWRITE) --> >> _cl_open(buffer, O_RDWR | O_LARGEFILE, _S_IWRITE) >> >> The required header define is already defined in config files and >> adding this flag shouldn't affect 64-bit machines in any way. Thanks! -- Michael Levin <mele...@stanford.edu> ---------------------------------------------------------------------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
