[jira] Updated: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-16 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2814: Attachment: LUCENE-2814.patch First iteration. Passes all tests except TestNRTThreads

[jira] Commented: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-16 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12972298#action_12972298 ] Earwin Burrfoot commented on LUCENE-2814: - So, what's the plan? stop writing

[jira] Commented: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-16 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12972316#action_12972316 ] Earwin Burrfoot commented on LUCENE-2814: - Instead of you pulling out docstore

[jira] Updated: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-16 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2814: Attachment: LUCENE-2814.patch Patch updated to trunk, no nocommits, no *.closeDocStore

[jira] Commented: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-14 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971248#action_12971248 ] Earwin Burrfoot commented on LUCENE-2814: - bq. We should verify the back-compat

[jira] Commented: (LUCENE-2811) SegmentInfo should explicitly track whether that segment wrote term vectors

2010-12-14 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971303#action_12971303 ] Earwin Burrfoot commented on LUCENE-2811: - I think SegmentInfo.hasVectors should

[jira] Commented: (LUCENE-2811) SegmentInfo should explicitly track whether that segment wrote term vectors

2010-12-14 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971510#action_12971510 ] Earwin Burrfoot commented on LUCENE-2811: - From IRC: SegmentMerger.hasVectors

[jira] Commented: (LUCENE-2814) stop writing shared doc stores across segments

2010-12-13 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971057#action_12971057 ] Earwin Burrfoot commented on LUCENE-2814: - I'll take this. I think. stop writing

[jira] Commented: (LUCENE-2611) IntelliJ IDEA setup

2010-12-13 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971058#action_12971058 ] Earwin Burrfoot commented on LUCENE-2611: - [quote] bq. I wonder if several .iml

[jira] Issue Comment Edited: (LUCENE-2611) IntelliJ IDEA setup

2010-12-13 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971058#action_12971058 ] Earwin Burrfoot edited comment on LUCENE-2611 at 12/13/10 5:36 PM

[jira] Commented: (LUCENE-2802) DirectoryReader ignores NRT SegmentInfos in #isOptimized()

2010-12-06 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12968503#action_12968503 ] Earwin Burrfoot commented on LUCENE-2802: - Patch looks cool. DirectoryReader

[jira] Commented: (LUCENE-2799) MMapDirectory not designed for inheritance

2010-12-04 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966865#action_12966865 ] Earwin Burrfoot commented on LUCENE-2799: - I think it's always best to copy-paste

[jira] Commented: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-03 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966550#action_12966550 ] Earwin Burrfoot commented on LUCENE-2790: - Ok, let's commit? There's no need

Re: Consolidate MP and LMP

2010-12-02 Thread Earwin Burrfoot
Actually, in trunk IW doesn't break on anything else. There's one private no-longer-used method I forgot to delete on my drop-all-deprecations spree. And there's a block in addIndexes, that explicitly checks instanceof, and only then casts to LMP. I'm against consolidating MP and LMP. MP is a

Re: Consolidate MP and LMP

2010-12-02 Thread Earwin Burrfoot
Hmm .. now that I look closely at it, MP has useCompundFile/DocStore methods, and LMP just adds getUseCompoundFile(). Why? And IndexWriter.addIndexes(IndexReader...) queries instanceof LMP, instead of calling mp.useCompoundFile()? getUseCompoundFile - is a setting on LMP. MP.useCompoundFile

Re: Consolidate MP and LMP

2010-12-02 Thread Earwin Burrfoot
On Thu, Dec 2, 2010 at 14:19, Shai Erera ser...@gmail.com wrote: You can't remove it on 3x, it's used by a host of deprecated methods that access LMP's settings through IW. Remove means deprecate in 3x and remove in trunk. Should have been more clear about that. We can drop it from trunk

[jira] Commented: (LUCENE-2789) Let codec decide to use compound file system or not

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966071#action_12966071 ] Earwin Burrfoot commented on LUCENE-2789: - I'd like to a see a switch like

Re: Consolidate MP and LMP

2010-12-02 Thread Earwin Burrfoot
On Thu, Dec 2, 2010 at 15:04, Shai Erera ser...@gmail.com wrote: Earwin: LogMergePolicy.getUseCompoundFile() is a public and not private API on trunk, not deprecated and used. Perhaps you are talking about something else? I was speaking of getLogMergePolicy that you mentioned: 1) Fix IW to

[jira] Commented: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966103#action_12966103 ] Earwin Burrfoot commented on LUCENE-2790: - Fails addIndexesWithThreads

[jira] Issue Comment Edited: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966108#action_12966108 ] Earwin Burrfoot edited comment on LUCENE-2790 at 12/2/10 8:12 AM

[jira] Updated: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2790: Attachment: LUCENE-2790.patch Check this patch out. It moves noCFS ratio

[jira] Commented: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966112#action_12966112 ] Earwin Burrfoot commented on LUCENE-2790: - bq. I checked who implements

[jira] Updated: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2790: Attachment: LUCENE-2790.patch Okay, this patch fixes remaining threading issue

[jira] Updated: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2790: Attachment: LUCENE-2790.patch Fixed your test failure IndexWriter should call

[jira] Commented: (LUCENE-2790) IndexWriter should call MP.useCompoundFile and not LogMP.getUseCompoundFile

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966285#action_12966285 ] Earwin Burrfoot commented on LUCENE-2790: - Shai, what about: bq. My only concern

[jira] Closed: (LUCENE-2307) Spurious exception in TestIndexWriter

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot closed LUCENE-2307. --- Resolution: Cannot Reproduce Never reproduced since, closing as stale and outdated

[jira] Commented: (LUCENE-2471) Supporting bulk copies in Directory

2010-12-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966358#action_12966358 ] Earwin Burrfoot commented on LUCENE-2471: - Hmmm. Are we going to do

Re: Document aware analyzers was Re: deprecating Versions

2010-12-01 Thread Earwin Burrfoot
I agree with Robert that minimizing analysis - indexer interface is the way to go. For me, one of Lucene's problems is that it wants to do too much stuff out of the box, and is tightly coupled, so you can't drop much of the things you never need. Having minimal interface for the indexer allows us

Re: svn commit: r1040145 - /lucene/dev/branches/branch_3x/lucene/src/java/org/apache/lucene/store/RAMDirectory.java

2010-11-30 Thread Earwin Burrfoot
On Tue, Nov 30, 2010 at 07:48, Shai Erera ser...@gmail.com wrote: The break was only in MockRAMDir, and even that is because I changed fileMap type from HashMap to Map, which IMO should have been defined like that from the beginning. As a piece of trivia. I did some benchmarks and declaring

Re: API Semantics and Backwards

2010-11-30 Thread Earwin Burrfoot
We can try writing tests that only check binary compatibility for public/protected members? And use these for back-compat testing. On Tue, Nov 30, 2010 at 12:47, Shai Erera ser...@gmail.com wrote: I realize the benefits of not storing the backwards source -- I don't care too much about the size

Re: API Semantics and Backwards

2010-11-30 Thread Earwin Burrfoot
Oh, Shai already said this, so +1. On Tue, Nov 30, 2010 at 13:11, Earwin Burrfoot ear...@gmail.com wrote: We can try writing tests that only check binary compatibility for public/protected members? And use these for back-compat testing. On Tue, Nov 30, 2010 at 12:47, Shai Erera ser

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965194#action_12965194 ] Earwin Burrfoot commented on LUCENE-2779: - bq. Cloning the keySet

[jira] Commented: (LUCENE-2785) TopFieldCollector throws AIOOBE if numHits is 0

2010-11-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965202#action_12965202 ] Earwin Burrfoot commented on LUCENE-2785: - A Collector, that counts - priceless

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965296#action_12965296 ] Earwin Burrfoot commented on LUCENE-2779: - Quoting Sun JDK 1.6: {code} public

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965380#action_12965380 ] Earwin Burrfoot commented on LUCENE-2779: - bq. So I ended up writing the following

Re: deprecating Versions

2010-11-29 Thread Earwin Burrfoot
On Mon, Nov 29, 2010 at 13:34, Robert Muir rcm...@gmail.com wrote: On Mon, Nov 29, 2010 at 2:50 AM, Earwin Burrfoot ear...@gmail.com wrote: And for indexes: * Index compatibility is guaranteed across two adjacent major releases. eg 2.x - 3.x, 3.x - 4.x.  That includes both binary compat

Re: deprecating Versions

2010-11-29 Thread Earwin Burrfoot
. And for that the factories are needed. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Earwin Burrfoot [mailto:ear...@gmail.com] Sent: Monday, November 29, 2010 11:53 AM To: dev@lucene.apache.org Subject: Re

Re: deprecating Versions

2010-11-29 Thread Earwin Burrfoot
I'm talking about the analyzers we provide in lucene itself. there is no reason, no advantage towards these being .java code: it just causes problems. I see little difference between public class StockAnalyzers { public static final Analyzer STANDARD_30 = new AnalyzerBuilder(). add(new

Re: deprecating Versions

2010-11-29 Thread Earwin Burrfoot
On Mon, Nov 29, 2010 at 15:28, Robert Muir rcm...@gmail.com wrote: On Mon, Nov 29, 2010 at 7:21 AM, Earwin Burrfoot ear...@gmail.com wrote: There's no reason, no advantage towards using .xml files for configuration, when said configuration can easily be expressed programmatically. It just

[jira] Commented: (LUCENE-2781) Drop deprecations from trunk

2010-11-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964749#action_12964749 ] Earwin Burrfoot commented on LUCENE-2781: - bq. unfortunately we cant yet remove

[jira] Commented: (LUCENE-2781) Drop deprecations from trunk

2010-11-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964751#action_12964751 ] Earwin Burrfoot commented on LUCENE-2781: - Hmm.. and regarding this exact case

[jira] Commented: (LUCENE-2781) Drop deprecations from trunk

2010-11-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964755#action_12964755 ] Earwin Burrfoot commented on LUCENE-2781: - Ok, restore it then and fix deprecation

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964920#action_12964920 ] Earwin Burrfoot commented on LUCENE-2779: - I don't believe cloning the keySet

Re: deprecating Versions

2010-11-29 Thread Earwin Burrfoot
On Mon, Nov 29, 2010 at 20:51, DM Smith dmsmith...@gmail.com wrote: The other thing I'd like is for the spec to be save along side of the index as a manifest. From earlier threads, I can see that there might need to be one for writing and another for reading. I'm not interested in using it to

Re: Lucene-Solr-tests-only-3.x - Build # 1950 - Failure

2010-11-29 Thread Earwin Burrfoot
Or you can make threadlocal RNGs. On Mon, Nov 29, 2010 at 23:20, Yonik Seeley yo...@lucidimagination.com wrote: On Mon, Nov 29, 2010 at 2:52 PM, Michael McCandless luc...@mikemccandless.com wrote: Though why doesn't the random seed reproduce it? OK, I think I see why. For reproducibility,

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964982#action_12964982 ] Earwin Burrfoot commented on LUCENE-2779: - Maybe we should commit it to 4.0 only

[jira] Commented: (LUCENE-2779) Use ReadWriteLock in RAMDirectory

2010-11-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964511#action_12964511 ] Earwin Burrfoot commented on LUCENE-2779: - If you don't write, you don't care

[jira] Commented: (LUCENE-2779) Use ReadWriteLock in RAMDirectory

2010-11-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964512#action_12964512 ] Earwin Burrfoot commented on LUCENE-2779: - I mean, even if aquiring locks costed

[jira] Commented: (LUCENE-2779) Use ReadWriteLock in RAMDirectory

2010-11-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964515#action_12964515 ] Earwin Burrfoot commented on LUCENE-2779: - My primary point was that you're

[jira] Commented: (LUCENE-2779) Use ConcurrentHashMap in RAMDirectory

2010-11-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964556#action_12964556 ] Earwin Burrfoot commented on LUCENE-2779: - I'm happy with CHM. The only thing

[jira] Commented: (LUCENE-2781) Drop deprecations from trunk

2010-11-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12964640#action_12964640 ] Earwin Burrfoot commented on LUCENE-2781: - bq. i thought you said you weren't

Re: deprecating Versions

2010-11-28 Thread Earwin Burrfoot
optimize your 3.x indexes when going through 4.x to 5.x. On Mon, Nov 29, 2010 at 05:35, Robert Muir rcm...@gmail.com wrote: On Sat, Nov 27, 2010 at 3:44 PM, Earwin Burrfoot ear...@gmail.com wrote: I think we should deprecate and remove Version constants as Lucene progresses? well one idea

deprecating Versions

2010-11-27 Thread Earwin Burrfoot
I think we should deprecate and remove Version constants as Lucene progresses? Imagine there's a number of features in 4.x that get deprecated and un-defaulted in 5.x, then removed in 6.x Our user compiled with Version.4_0, it was cool in 4.x, then it still worked in 5.x, as we preserved index

[jira] Updated: (LUCENE-2781) Drop deprecations from trunk

2010-11-27 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2781: Attachment: drop-deprecations.patch New patch. Current status: * Lucene's deprecations

[jira] Updated: (LUCENE-2781) Drop deprecations from trunk

2010-11-27 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2781: Attachment: drop-deprecations.patch New patch. Version 2x dropped, 3x deprecated. Builds

[jira] Updated: (LUCENE-2781) Drop deprecations from trunk

2010-11-27 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2781: Attachment: drop-deprecations.patch Same, rebased on latest trunk. Drop deprecations

[jira] Updated: (LUCENE-2781) Drop deprecations from trunk

2010-11-26 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2781: Attachment: drop-deprecations.patch Stab one. Everything works, except Solr - this fails

[jira] Commented: (LUCENE-2506) A Stateful Filter That Works Across Index Segments

2010-11-25 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935755#action_12935755 ] Earwin Burrfoot commented on LUCENE-2506: - bq. Some better Filter API is required

[jira] Commented: (LUCENE-2691) Consolidate Near Real Time and Reopen API semantics

2010-11-25 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935794#action_12935794 ] Earwin Burrfoot commented on LUCENE-2691: - {quote} bq. You're still okay

[jira] Commented: (LUCENE-2779) Use ReadWriteLock in RAMDirectory

2010-11-25 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935813#action_12935813 ] Earwin Burrfoot commented on LUCENE-2779: - Using RWLock is needless. synchronized

[jira] Commented: (LUCENE-2779) Use ReadWriteLock in RAMDirectory

2010-11-25 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935816#action_12935816 ] Earwin Burrfoot commented on LUCENE-2779: - In fact, uncontended synchronized block

Re: [jira] Commented: (LUCENE-2691) Consolidate Near Real Time and Reopen API semantics

2010-11-25 Thread Earwin Burrfoot
be a proxy and notify it of anything else besides opening and closing files. Proxying can be completely transparent. On Thu, Nov 25, 2010 at 6:00 PM, Earwin Burrfoot (JIRA) j...@apache.org wrote:    [ https://issues.apache.org/jira/browse/LUCENE-2691?page

[jira] Commented: (LUCENE-2506) A Stateful Filter That Works Across Index Segments

2010-11-25 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935899#action_12935899 ] Earwin Burrfoot commented on LUCENE-2506: - bq. we are doing an SQL query, get back

[jira] Created: (LUCENE-2781) Drop deprecations from trunk

2010-11-25 Thread Earwin Burrfoot (JIRA)
Drop deprecations from trunk Key: LUCENE-2781 URL: https://issues.apache.org/jira/browse/LUCENE-2781 Project: Lucene - Java Issue Type: Task Affects Versions: 4.0 Reporter: Earwin Burrfoot

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935400#action_12935400 ] Earwin Burrfoot commented on LUCENE-2755: - bq. Refactor IW, MS and MP so that MS

[jira] Commented: (LUCENE-2691) Consolidate Near Real Time and Reopen API semantics

2010-11-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935569#action_12935569 ] Earwin Burrfoot commented on LUCENE-2691: - You're still okay with an API

[jira] Commented: (LUCENE-2771) Remove norms() support from non-atomic IndexReaders

2010-11-19 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12934078#action_12934078 ] Earwin Burrfoot commented on LUCENE-2771: - SegmentReader and AllOtherReaders

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-17 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932864#action_12932864 ] Earwin Burrfoot commented on LUCENE-2755: - {quote} If we proceed w/ your proposal

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-16 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932561#action_12932561 ] Earwin Burrfoot commented on LUCENE-2755: - Shai: bq. The thing is - the second

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-15 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932124#action_12932124 ] Earwin Burrfoot commented on LUCENE-2755: - Whatever solution for block-on-add you

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-15 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932200#action_12932200 ] Earwin Burrfoot commented on LUCENE-2755: - bq. There was some reason why

[jira] Commented: (LUCENE-1799) Unicode compression

2010-11-15 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932207#action_12932207 ] Earwin Burrfoot commented on LUCENE-1799: - .. and not the Codec, as was suggested

[jira] Commented: (LUCENE-1799) Unicode compression

2010-11-15 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932206#action_12932206 ] Earwin Burrfoot commented on LUCENE-1799: - Returning to this issue, right now

[jira] Commented: (LUCENE-2755) Some improvements to CMS

2010-11-15 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932266#action_12932266 ] Earwin Burrfoot commented on LUCENE-2755: - bq. But then you accumulate too many

[jira] Commented: (LUCENE-2167) Implement StandardTokenizer with the UAX#29 Standard

2010-11-08 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929587#action_12929587 ] Earwin Burrfoot commented on LUCENE-2167: - bq. No thanks, i dont want to read my

[jira] Commented: (LUCENE-2167) Implement StandardTokenizer with the UAX#29 Standard

2010-11-07 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929407#action_12929407 ] Earwin Burrfoot commented on LUCENE-2167: - bq. Would it somehow be possible

Re: fast bitset

2010-11-05 Thread Earwin Burrfoot
It's okay, trunk has iteration-based filters. Filters with low selectivity might be faster if used in oldstyle random-access way, though. If one wants to exploit this, compressed bitmaps are no go. On Sat, Nov 6, 2010 at 00:29, Peter Karich peat...@yahoo.de wrote:  And they're not

Re: inconsistency/performance trap of empty terms

2010-10-30 Thread Earwin Burrfoot
On Fri, Oct 29, 2010 at 21:50, Robert Muir rcm...@gmail.com wrote: I was suggesting that mathematically, the empty term makes no sense in an inverted index, and we shouldn't allow it. Its one solution. Mathematically an inverted index is keyed by strings. Any strings. Empty term is just a case

Re: inconsistency/performance trap of empty terms

2010-10-30 Thread Earwin Burrfoot
I'd say support them everywhere, and slip LengthFilter into all the standard Analyzers, so people won't hit empty terms unless they opt-in for it. This is a most consistent approach. On Sat, Oct 30, 2010 at 15:06, Robert Muir rcm...@gmail.com wrote: On Sat, Oct 30, 2010 at 7:01 AM, Earwin

Re: inconsistency/performance trap of empty terms

2010-10-30 Thread Earwin Burrfoot
On Sat, Oct 30, 2010 at 18:49, Uwe Schindler u...@thetaphi.de wrote: In my opinion, we should not have analyzers at all (just my personal opinion). new Field(name, TokenStream) is much enough from consistency standpoint! Indeed, my friend! -- Kirill Zakharenko/Кирилл Захаренко

Re: is multi-threads searcher feasible idea to speed up?

2010-10-04 Thread Earwin Burrfoot
Thread-per-segment approach should run well with Zoie MergePolicy. On Tue, Sep 28, 2010 at 16:17, Michael McCandless luc...@mikemccandless.com wrote: This is an excellent idea! And, desperately needed. It's high time Lucene can take advantage of concurrency when running a single query.  

Re: instantiated contrib

2010-08-26 Thread Earwin Burrfoot
IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough queries, and RAMDir eventually outperforms it. On Thu, Aug 26, 2010 at 11:24, Li Li fancye...@gmail.com wrote: I have about 70k document, the total indexed size is about 15MB(the orginal text files' size).                dir=new

Re: instantiated contrib

2010-08-26 Thread Earwin Burrfoot
you mean that InstantiatedIndex is not as fast as its document says? 2010/8/26 Earwin Burrfoot ear...@gmail.com: IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough queries, and RAMDir eventually outperforms it. On Thu, Aug 26, 2010 at 11:24, Li Li fancye...@gmail.com wrote: I

[jira] Commented: (LUCENE-2593) disk full can cause index corruption in certain cases

2010-08-11 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897325#action_12897325 ] Earwin Burrfoot commented on LUCENE-2593: - Yeehaw! This looks very much like a bug

Re: index format error because disk full - possible Lucene bug?

2010-07-09 Thread Earwin Burrfoot
I believe I've seen a similar condition a few times. A segments file referring zero-length segment files after a disk full event. On Fri, Jul 9, 2010 at 13:37, Michael McCandless luc...@mikemccandless.com wrote: I responded on the original thread. Disk full should never cause index corruption

Re: Proposal: Scorer api change

2010-06-09 Thread Earwin Burrfoot
Lies, lies, lies :) I mean, Sun JIT is overrelied on. Especially in regards to inlining. But, there are some cases when you can trust it. I.e. if you call a virtual method and this exact call-site gets refs to different objects at runtime (meaning here - you wrap different Queries in your

Re: Proposal: Scorer api change

2010-06-09 Thread Earwin Burrfoot
Can we represent the Query state in some general structure, that no matter which Query you get, you'll know how to score it? No. You could go for unified interface that allows you to express different query states, like a set of untyped key-values, but you'll end up switching on these

Re: Proposal: Scorer api change

2010-06-09 Thread Earwin Burrfoot
the best way to achieve that. Shai On Wed, Jun 9, 2010 at 2:24 PM, Earwin Burrfoot ear...@gmail.com wrote: Can we represent the Query state in some general structure, that no matter which Query you get, you'll know how to score it? No. You could go for unified interface that allows you

Re: Proposal: Scorer api change

2010-06-09 Thread Earwin Burrfoot
On Wed, Jun 9, 2010 at 15:39, Doron Cohen cdor...@gmail.com wrote: I think you'd still not modify a nicely extendible/wrapable API just to avoid the extra call, unless benchmarking shows that the cost is high. Current Query API is NOT nicely extensible :) Look above for BM25BooleanQuery

Re: Proposal: Scorer api change

2010-06-08 Thread Earwin Burrfoot
The problem with your proposal is that, currently, Lucene uses current iteration state to compute score. I.e. it already knows which of SHOULD BQ clauses matched for current doc, so it's easier to calculate the score. If you change API to allow scoring arbitrary documents (even those that didn't

Re: Proposal: Scorer api change

2010-06-08 Thread Earwin Burrfoot
Shai, his wrapper Scorer will just look like: DISI getDISI() { return delegate.getDISI(); } float score(int doc) { return calcMyAwesomeScore(doc); } this saves delegate.nextDoc(), delegate.advance() indirection calls. But I already offered a better alternative :) On Tue, Jun 8, 2010 at

Re: Proposal: Scorer api change

2010-06-08 Thread Earwin Burrfoot
, Earwin Burrfoot ear...@gmail.com wrote: To compute a score you have to see which of your subqueries did not match, which did, and what are the docfreqs/positions for them. When iterating, and calling score() only for current doc - parts of this data (maybe even all of it, not sure) is already

[jira] Commented: (LUCENE-2491) Extend Codec with a SegmentInfos writer / reader

2010-06-07 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12876203#action_12876203 ] Earwin Burrfoot commented on LUCENE-2491: - Or we can force the same Codec

[jira] Commented: (LUCENE-2311) Pass potent SR to IRWarmer.warm(), and also call warm() for new segments

2010-06-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12874634#action_12874634 ] Earwin Burrfoot commented on LUCENE-2311: - bq. Does your pending patch (what's

[jira] Commented: (LUCENE-2485) IndexWriter should also warm flushed segments

2010-06-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12874650#action_12874650 ] Earwin Burrfoot commented on LUCENE-2485: - bq. As long as warming a new segment

[jira] Commented: (LUCENE-2480) Remove support for pre-3.0 indexes

2010-05-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873465#action_12873465 ] Earwin Burrfoot commented on LUCENE-2480: - Wow! So fast! :) bq. You didn't remove

[jira] Commented: (LUCENE-2480) Remove support for pre-3.0 indexes

2010-05-30 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873469#action_12873469 ] Earwin Burrfoot commented on LUCENE-2480: - bq. Strange, there were lines in my

[jira] Commented: (LUCENE-2480) Remove support for pre-3.0 indexes

2010-05-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873292#action_12873292 ] Earwin Burrfoot commented on LUCENE-2480: - Doing that now, plus some additions

[jira] Updated: (LUCENE-2480) Remove support for pre-3.0 indexes

2010-05-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2480: Comment: was deleted (was: Doing that now, plus some additions to Shai's patch) Remove

Re: Thoughts on CMS and SMS

2010-05-29 Thread Earwin Burrfoot
I disagree about time limiting MS. It may not be useful in many cases, true. But I have a scenario in which machines are used to perform all sorts of tasks and the are windows in which I'm allowed to do 'heavy operations'. It's true I can just choose not to merge large segments, but I

<    1   2   3   4   5   6   7   >