[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10274 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10274/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 303 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/303/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10291 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10291/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10273 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10273/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 307 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/307/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090023#comment-13090023 ] Uwe Schindler commented on LUCENE-3218: --- I would also rename CFIndexInput to SliceIndexInput, it's private so does not matter, but wozuld be nice to have. Otherwise I agree with committing to trunk. As far as I see, the format did not change in trunk, so once we get this back into 3.x we are at the state pre-revert? > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Solr-3.x - Build # 444 - Failure
Build: https://builds.apache.org/job/Solr-3.x/444/ No tests ran. Build Log (for compile errors): [...truncated 29 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 302 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/302/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10290 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10290/ No tests ran. Build Log (for compile errors): [...truncated 27 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 306 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/306/ No tests ran. Build Log (for compile errors): [...truncated 23 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10272 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10272/ No tests ran. Build Log (for compile errors): [...truncated 32 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-trunk - Build # 1661 - Still Failing
Build: https://builds.apache.org/job/Lucene-trunk/1661/ No tests ran. Build Log (for compile errors): [...truncated 12972 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314160802)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314160802.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314155581)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314155581.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3399) Enable replace-able field caches
[ https://issues.apache.org/jira/browse/LUCENE-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-3399: - Attachment: LUCENE-3399.patch A cut of this. > Enable replace-able field caches > > > Key: LUCENE-3399 > URL: https://issues.apache.org/jira/browse/LUCENE-3399 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3399.patch > > > For LUCENE-2312 we need to be able to synchronously replace field cache > values and receive events on when new field cache values are created. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314148142)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314148142.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravind Srini reopened SOLR-2727: - > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Need help with lucene 3.2/3.2
Hello there I need some help with a odd behave with Lucene since version 2.9 I have a project with Lucene 2.41 that runs fine with 600k document where I use two index folder to separate two set of index, one primary and another one secondary, this allow me to switch the indexreader/indexsearcher by the time I want to build a fresh index and also keep a safe copy of it, While I am using Lucene 2.4.1 I have no problem with my application regarding have a web application hitting many time the searcher, which is shared, when I have to switch the index, I do check which one is stale and then close the old one and open the new one, even having many threads into my container, it can safely close and open this, without having any problem. This days, since version 3 arrives, I've being trying to upgrade my application for all matters, by the end of this, everything seems OK, BUT DID NOT, I used JMeter to put it to burn, then I got this : java12650 root 290r REG 253,11 702777186193160 /index/secondary/spell/_12.frq (deleted)java12650 root 291r REG 253,11 313120836193167 /index/secondary/spell/_10.prxjava12650 root 292r REG 253,11 240317416193158 //index/secondary/spell/_12.tis (deleted)java12650 root 293r REG 253,11 240317416193158 /index/secondary/spell/_12.tis (deleted)java 12650 root 294r REG 253,11 702777186193160 /index/secondary/spell/_12.frq (deleted)java12650 root 295r REG 253,11 240317416193158 /index/secondary/spell/_12.tis (deleted)java 12650 root 296r REG 253,11 252614906193159 /index/secondary/spell/_10.fdtjava12650 root 297r REG 253,11 61809686193162 /index/secondary/spell/_12.nrm (deleted)java12650 root 298r REG 253,11 123665726193163 /index/secondary/spell/_10.fdxjava12650 root 299r REG 253,11 312960756193161 /index/secondary/spell/_12.prx (deleted)java12650 root 300r REG 253,11 252497926193156 /index/secondary/spell/_12.fdt (deleted)java12650 root 301r REG 253,11 123619326193157 /index/secondary/spell/_12.fdx (deleted)java 12650 root 311r REG 253,1161832886193168 /index/secondary/spell/_10.nrmetc... so, somehow my container keeps old indexsearcher/indexreader opened, even the files no longer exists, that making it to reach max open files from the System, and you know what comes next. Even look like complex, for the time it grown the many file opened is invert when use Lucene 2.4.1, which manage and close all old indexsearcher/indexreader and keeping clean the opened files from lsof command from linux, where I run this application As I can not change my entire application to use Solr, which seems to have a good approach, and I also can not walk back to Lucene 2.4.1 and let it there forever, I ask for help here. If anyone from there can help me, please let me know Best regards and very thanks in advanced Wilson
[jira] [Commented] (LUCENE-2748) Convert all Lucene web properties to use the ASF CMS
[ https://issues.apache.org/jira/browse/LUCENE-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089912#comment-13089912 ] Andi Vajda commented on LUCENE-2748: I completely agree, there should one css for the whole site. Consistent look and feel is a good thing. Currently, this is not how it's setup, each template gets its own .css. That would simple to change, of course. Ok, I'm going to try to tweak the pylucene.css to something more legible (I'm as incompetent with .css as anyone but I can copy/paste). Once I like what I see, I'll commit that and the other templates are welcome to adopt it. If someones feel like intervening there first, by all means :-) > Convert all Lucene web properties to use the ASF CMS > > > Key: LUCENE-2748 > URL: https://issues.apache.org/jira/browse/LUCENE-2748 > Project: Lucene - Java > Issue Type: Bug >Reporter: Grant Ingersoll >Assignee: Grant Ingersoll > > The new CMS has a lot of nice features (and some kinks to still work out) and > Forrest just doesn't cut it anymore, so we should move to the ASF CMS: > http://apache.org/dev/cms.html -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314140461)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314140461.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Solr support for stored procedures
Does Solr support calling stored procedures in the data-config.xml? Thanks! Maria
trunk test failure (1314136801)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314136801.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2664) Disable/enable autocommit on the fly
[ https://issues.apache.org/jira/browse/SOLR-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089793#comment-13089793 ] Mark Miller commented on SOLR-2664: --- I don't think it is possible - I think with Yonik's idea, all clients would have to coordinate, or you wouldn't really be guaranteed to have a batch you could avoid committing. > Disable/enable autocommit on the fly > > > Key: SOLR-2664 > URL: https://issues.apache.org/jira/browse/SOLR-2664 > Project: Solr > Issue Type: New Feature > Components: update >Affects Versions: 4.0 >Reporter: Simon Rosenthal >Priority: Minor > Fix For: 4.0 > > Attachments: SOLR-2664.patch > > > There are occasions when although autocommit is configured, it would be > desirable to disable it temporarily - for instance when batch adding/updating > a set of documents which should be committed atomically (e.g. a set of price > changes). > The patch adds and commands to > XMLUpdateHandler, and also adds a disableAutoCommit=true|false attribute to > the element - this will disable autocommit until the terminating > at the end of the XML document is reached. > At present, the autocommit state will not survive a core reload. > It should be possible to extend this functionality to SolrJ, CSVUpdatehandler > ( and JSONUpdateHandler ?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2748) Convert all Lucene web properties to use the ASF CMS
[
https://issues.apache.org/jira/browse/LUCENE-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089787#comment-13089787
]
Hoss Man commented on LUCENE-2748:
--
{quote}
And last but not least, I need to mess with the .css so that the pages
are legible, links visible, fonts balanced, etc...
Is there a plan to have a global CSS for all pages or is this left up to each
area ?
{quote}
I think one of the goals was to try and make the entire "Lucene" site more
consistent in it's look/feel/navigation (and if it wasn't, then i vote for it
to be a goal) so i would say having a global css for the entire site is wise --
particularly if you're seeing problems with legibility, visibility, and fonts.
we should fix that everywhere.
> Convert all Lucene web properties to use the ASF CMS
>
>
> Key: LUCENE-2748
> URL: https://issues.apache.org/jira/browse/LUCENE-2748
> Project: Lucene - Java
> Issue Type: Bug
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>
> The new CMS has a lot of nice features (and some kinks to still work out) and
> Forrest just doesn't cut it anymore, so we should move to the ASF CMS:
> http://apache.org/dev/cms.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2664) Disable/enable autocommit on the fly
[ https://issues.apache.org/jira/browse/SOLR-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089783#comment-13089783 ] Simon Rosenthal commented on SOLR-2664: --- bq. Rather than disabling autoCommit globally (for all clients), shouldn't it just disable it for that particular client? A different client may be adding some time sensitive documents. That's a much better approach! I like the "batch" parameter. Is it in fact now possible to autocommit (or not, in this case) only for a particular content stream/batch, when multiple ones are being indexed simultaneoulsy ? (my understanding has always been that commits/autocommits were global in effect). > Disable/enable autocommit on the fly > > > Key: SOLR-2664 > URL: https://issues.apache.org/jira/browse/SOLR-2664 > Project: Solr > Issue Type: New Feature > Components: update >Affects Versions: 4.0 >Reporter: Simon Rosenthal >Priority: Minor > Fix For: 4.0 > > Attachments: SOLR-2664.patch > > > There are occasions when although autocommit is configured, it would be > desirable to disable it temporarily - for instance when batch adding/updating > a set of documents which should be committed atomically (e.g. a set of price > changes). > The patch adds and commands to > XMLUpdateHandler, and also adds a disableAutoCommit=true|false attribute to > the element - this will disable autocommit until the terminating > at the end of the XML document is reached. > At present, the autocommit state will not survive a core reload. > It should be possible to extend this functionality to SolrJ, CSVUpdatehandler > ( and JSONUpdateHandler ?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn van Groningen updated SOLR-2066: Attachment: SOLR-2066.patch Minor update. Small code cleanup. > Search Grouping: support distributed search > --- > > Key: SOLR-2066 > URL: https://issues.apache.org/jira/browse/SOLR-2066 > Project: Solr > Issue Type: Sub-task >Reporter: Yonik Seeley > Fix For: 3.4, 4.0 > > Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, > SOLR-2066.patch > > > Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2664) Disable/enable autocommit on the fly
[ https://issues.apache.org/jira/browse/SOLR-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089760#comment-13089760 ] Yonik Seeley commented on SOLR-2664: Rather than disabling autoCommit globally (for all clients), shouldn't it just disable it for that particular client? A different client may be adding some time sensitive documents. We could add a "batch=true" that is a higher level concept, and does the following: - won't trigger autocommits - updates aren't guaranteed to be seen by real-time-get (and thus logging all of the document stored fields wouldn't be necessary) Another random thought: if an explicit "commit=true" is specified in the URL, perhaps auto commit could be disabled? That could either take the place of a "batch" parameter, or at least imply "batch=true". > Disable/enable autocommit on the fly > > > Key: SOLR-2664 > URL: https://issues.apache.org/jira/browse/SOLR-2664 > Project: Solr > Issue Type: New Feature > Components: update >Affects Versions: 4.0 >Reporter: Simon Rosenthal >Priority: Minor > Fix For: 4.0 > > Attachments: SOLR-2664.patch > > > There are occasions when although autocommit is configured, it would be > desirable to disable it temporarily - for instance when batch adding/updating > a set of documents which should be committed atomically (e.g. a set of price > changes). > The patch adds and commands to > XMLUpdateHandler, and also adds a disableAutoCommit=true|false attribute to > the element - this will disable autocommit until the terminating > at the end of the XML document is reached. > At present, the autocommit state will not survive a core reload. > It should be possible to extend this functionality to SolrJ, CSVUpdatehandler > ( and JSONUpdateHandler ?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3399) Enable replace-able field caches
Enable replace-able field caches Key: LUCENE-3399 URL: https://issues.apache.org/jira/browse/LUCENE-3399 Project: Lucene - Java Issue Type: Improvement Components: core/index Affects Versions: 4.0 Reporter: Jason Rutherglen Priority: Minor For LUCENE-2312 we need to be able to synchronously replace field cache values and receive events on when new field cache values are created. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Separate issue for appendable field caches / doc values
Also a FieldCache getCaches(Reader) method is needed. I'm not sure exactly what getCaches would return. It's the entries? So maybe it would be called getEntries(Reader). That way the DWPT can get the existing field caches. Oh, and also it would need to be able to add an event listener that notifies when a new field cache [entry?] is created for a given reader / key. I think if this makes sense / is workable then this can be a separate issue. On Tue, Aug 23, 2011 at 4:00 PM, Jason Rutherglen wrote: > I just examined the field cache code. I don't think replacing FCs > needs to be difficult. Lets make the CachedArray values variable > volatile. values is already public. > > On Tue, Aug 23, 2011 at 3:02 PM, Jason Rutherglen > wrote: >>> Oh duh right -- you should be able to alloc too-large arrays and only >>> realloc once you run out. So amortized cost is low but some reopens >>> will take a hit to grow >> >> This hit will be minimal, ie, less than what we're doing now with >> cloned deleted docs bit vectors, which feasibly happen on each >> getReader() call. Growing the field cache will occur far less. >> >> Given there isn't a use case for the appendable field cache outside of >> RT / LUCENE-2312. I may bake it in, it's hard to extract, it's hard >> to maintain two patches. However the discussion was good. >> >>> DV is also provided by the IR (perDocValues method) so the RT reader >> >> Ok, it's not clear when / how DVs are used instead of field caches, >> and why their access isn't merged together? >> >> On Tue, Aug 23, 2011 at 12:30 PM, Michael McCandless >> wrote: >>> On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen >>> wrote: > But, we are trying to move FC under IR control (Martijn has a patch), > at which point an RT IR could have its own appending impl...? LUCENE-3360? It's placing the field cache into IR / SR. >>> >>> Yes! >>> The RAM reader could return it's own impl where the underlying array can be atomically replaced (when a larger sized array is needed). I agree that is logical. >>> >>> Good. >>> > But then... FC still returns fixed arrays so you can't append until we > fix that? I don't think anything should depend on the size of the field cache array. If it does, it seems odd. Are you preferring moving field cache access to method calls? What is the difference between that and primitive array access? >>> >>> Oh duh right -- you should be able to alloc too-large arrays and only >>> realloc once you run out. So amortized cost is low but some reopens >>> will take a hit to grow >>> For now I will create an independent field cache implementation that is appendable. It will only be associate-able with the DWPT / RAM reader. Maybe somehow it can work with LUCENE-3360 and / or the existing static FC access system. >>> >>> Sounds good. >>> Still not sure how doc values fits in. >>> >>> DV is also provided by the IR (perDocValues method) so the RT reader >>> should be able to impl its own. Each lookup is a method call so it >>> should be easier to back that w/ a more RT friendly data structure... >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >> > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Separate issue for appendable field caches / doc values
I just examined the field cache code. I don't think replacing FCs needs to be difficult. Lets make the CachedArray values variable volatile. values is already public. On Tue, Aug 23, 2011 at 3:02 PM, Jason Rutherglen wrote: >> Oh duh right -- you should be able to alloc too-large arrays and only >> realloc once you run out. So amortized cost is low but some reopens >> will take a hit to grow > > This hit will be minimal, ie, less than what we're doing now with > cloned deleted docs bit vectors, which feasibly happen on each > getReader() call. Growing the field cache will occur far less. > > Given there isn't a use case for the appendable field cache outside of > RT / LUCENE-2312. I may bake it in, it's hard to extract, it's hard > to maintain two patches. However the discussion was good. > >> DV is also provided by the IR (perDocValues method) so the RT reader > > Ok, it's not clear when / how DVs are used instead of field caches, > and why their access isn't merged together? > > On Tue, Aug 23, 2011 at 12:30 PM, Michael McCandless > wrote: >> On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen >> wrote: But, we are trying to move FC under IR control (Martijn has a patch), at which point an RT IR could have its own appending impl...? >>> >>> LUCENE-3360? It's placing the field cache into IR / SR. >> >> Yes! >> >>> The RAM >>> reader could return it's own impl where the underlying array can be >>> atomically replaced (when a larger sized array is needed). I agree >>> that is logical. >> >> Good. >> But then... FC still returns fixed arrays so you can't append until we fix that? >>> >>> I don't think anything should depend on the size of the field cache >>> array. If it does, it seems odd. Are you preferring moving field >>> cache access to method calls? What is the difference between that and >>> primitive array access? >> >> Oh duh right -- you should be able to alloc too-large arrays and only >> realloc once you run out. So amortized cost is low but some reopens >> will take a hit to grow >> >>> For now I will create an independent field cache implementation that >>> is appendable. It will only be associate-able with the DWPT / RAM >>> reader. Maybe somehow it can work with LUCENE-3360 and / or the >>> existing static FC access system. >> >> Sounds good. >> >>> Still not sure how doc values fits in. >> >> DV is also provided by the IR (perDocValues method) so the RT reader >> should be able to impl its own. Each lookup is a method call so it >> should be easier to back that w/ a more RT friendly data structure... >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2664) Disable/enable autocommit on the fly
[ https://issues.apache.org/jira/browse/SOLR-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089690#comment-13089690 ] Simon Rosenthal commented on SOLR-2664: --- In working on extending the functionality to SolrJ/CSV/JSON, I've seen some shortcomings and ambiguities in the initial API I was working on. here's a clearer approach: The anticipated use case would be to disable autocommits temporarily during update processing from a content stream (or from a server.add() in solrJ). So for /update handlers where a content stream is specified, append the parameter &deferAutoCommit=true to the URL. Autocommits will be disabled while the content stream is processed, and automatically re-enabled at the end of processing (regardless of success or failure). This will also be a recognized attribute in an element for solr XML processing. Additionally, no content stream is specified, one can specify the single operations /update?disableAutoCommit=true and subsequently /update/enableAutoCommit=true, in the same way as you can specify commit as the only parameter. However you're on your own if you do the one without the other SolrJ will have new server#disableAutoCommit and server#enableAutocommit methods, and also add(doc, boolean deferAutoCommit). > Disable/enable autocommit on the fly > > > Key: SOLR-2664 > URL: https://issues.apache.org/jira/browse/SOLR-2664 > Project: Solr > Issue Type: New Feature > Components: update >Affects Versions: 4.0 >Reporter: Simon Rosenthal >Priority: Minor > Fix For: 4.0 > > Attachments: SOLR-2664.patch > > > There are occasions when although autocommit is configured, it would be > desirable to disable it temporarily - for instance when batch adding/updating > a set of documents which should be committed atomically (e.g. a set of price > changes). > The patch adds and commands to > XMLUpdateHandler, and also adds a disableAutoCommit=true|false attribute to > the element - this will disable autocommit until the terminating > at the end of the XML document is reached. > At present, the autocommit state will not survive a core reload. > It should be possible to extend this functionality to SolrJ, CSVUpdatehandler > ( and JSONUpdateHandler ?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3373) waitForMerges deadlocks if background merge fails
[
https://issues.apache.org/jira/browse/LUCENE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089674#comment-13089674
]
Tim Smith commented on LUCENE-3373:
---
waitForMerges should continue to wait until all merges are complete (regardless
of if they all end up failing)
i would suggest updating the MergeThread to catch all exceptions and allow
processing the next merge. right now, any merge failure results in a
ThreadDeath, which seems rather nasty. should probably just catch the exception
and log a index trace message
> waitForMerges deadlocks if background merge fails
> -
>
> Key: LUCENE-3373
> URL: https://issues.apache.org/jira/browse/LUCENE-3373
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
>Affects Versions: 3.0.3
>Reporter: Tim Smith
>
> waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
> this is because the merge thread will die, but pending merges are still
> available
> normally, the merge thread will pick up the next merge once it finishes the
> previous merge, but in the event of a merge exception, the pending work is
> not resumed, but waitForMerges won't complete until all pending work is
> complete
> i worked around this by overriding doMerge() like so:
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {
> super.doMerge(merge);
> } catch (Throwable exc) {
> // Just logging the exception and not rethrowing
> // insert logging code here
> }
> }
> {code}
> Here's the rough steps i used to reproduce this issue:
> override doMerge like so
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {Thread.sleep(500L);} catch (InterruptedException e) { }
> super.doMerge(merge);
> throw new IOException("fail");
> }
> {code}
> then, if you do the following:
> loop 50 times:
> addDocument // any doc
> commit
> waitForMerges // This will deadlock sometimes
> SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089666#comment-13089666 ] Ryan McKinley commented on SOLR-2727: - the two issues seem related, but not necessarily a duplicate. It would be great to get someone with httpcomponent-4.x know how to look at how the http client is used thoughout solr. In /trunk (4.x) we can change the solr APIs to depend on the newer httpcomponent stuff > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Separate issue for appendable field caches / doc values
> Oh duh right -- you should be able to alloc too-large arrays and only > realloc once you run out. So amortized cost is low but some reopens > will take a hit to grow This hit will be minimal, ie, less than what we're doing now with cloned deleted docs bit vectors, which feasibly happen on each getReader() call. Growing the field cache will occur far less. Given there isn't a use case for the appendable field cache outside of RT / LUCENE-2312. I may bake it in, it's hard to extract, it's hard to maintain two patches. However the discussion was good. > DV is also provided by the IR (perDocValues method) so the RT reader Ok, it's not clear when / how DVs are used instead of field caches, and why their access isn't merged together? On Tue, Aug 23, 2011 at 12:30 PM, Michael McCandless wrote: > On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen > wrote: >>> But, we are trying to move FC under IR control (Martijn has a patch), >>> at which point an RT IR could have its own appending impl...? >> >> LUCENE-3360? It's placing the field cache into IR / SR. > > Yes! > >> The RAM >> reader could return it's own impl where the underlying array can be >> atomically replaced (when a larger sized array is needed). I agree >> that is logical. > > Good. > >>> But then... FC still returns fixed arrays so you can't append until we fix >>> that? >> >> I don't think anything should depend on the size of the field cache >> array. If it does, it seems odd. Are you preferring moving field >> cache access to method calls? What is the difference between that and >> primitive array access? > > Oh duh right -- you should be able to alloc too-large arrays and only > realloc once you run out. So amortized cost is low but some reopens > will take a hit to grow > >> For now I will create an independent field cache implementation that >> is appendable. It will only be associate-able with the DWPT / RAM >> reader. Maybe somehow it can work with LUCENE-3360 and / or the >> existing static FC access system. > > Sounds good. > >> Still not sure how doc values fits in. > > DV is also provided by the IR (perDocValues method) so the RT reader > should be able to impl its own. Each lookup is a method call so it > should be easier to back that w/ a more RT friendly data structure... > > Mike McCandless > > http://blog.mikemccandless.com > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3373) waitForMerges deadlocks if background merge fails
[
https://issues.apache.org/jira/browse/LUCENE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089662#comment-13089662
]
Michael McCandless commented on LUCENE-3373:
Hmm nice catch!
Any ides on how should we fix this...? What do we want to happen? Should
waitForMerges() return if any merge hits an exc? It could return a boolean
indicating an error occurred?
> waitForMerges deadlocks if background merge fails
> -
>
> Key: LUCENE-3373
> URL: https://issues.apache.org/jira/browse/LUCENE-3373
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
>Affects Versions: 3.0.3
>Reporter: Tim Smith
>
> waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
> this is because the merge thread will die, but pending merges are still
> available
> normally, the merge thread will pick up the next merge once it finishes the
> previous merge, but in the event of a merge exception, the pending work is
> not resumed, but waitForMerges won't complete until all pending work is
> complete
> i worked around this by overriding doMerge() like so:
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {
> super.doMerge(merge);
> } catch (Throwable exc) {
> // Just logging the exception and not rethrowing
> // insert logging code here
> }
> }
> {code}
> Here's the rough steps i used to reproduce this issue:
> override doMerge like so
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {Thread.sleep(500L);} catch (InterruptedException e) { }
> super.doMerge(merge);
> throw new IOException("fail");
> }
> {code}
> then, if you do the following:
> loop 50 times:
> addDocument // any doc
> commit
> waitForMerges // This will deadlock sometimes
> SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089651#comment-13089651 ] Simon Willnauer commented on LUCENE-3218: - bq. Why don't we get it committed to trunk and let it chill for a while, let it hit random testing for a while, get used by adventurous users, and then make the decision? +1 > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3398) TestCompoundFile fails on windows
[ https://issues.apache.org/jira/browse/LUCENE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-3398: Attachment: LUCENE-3398.patch here is a patch basically extracted the fix from LUCENE-3218 which tries to delete the file after the stream referencing it is closed. > TestCompoundFile fails on windows > - > > Key: LUCENE-3398 > URL: https://issues.apache.org/jira/browse/LUCENE-3398 > Project: Lucene - Java > Issue Type: Bug >Affects Versions: 4.0 >Reporter: Robert Muir >Assignee: Simon Willnauer > Attachments: LUCENE-3398.patch > > > ant test-core -Dtestcase=TestCompoundFile -Dtestmethod=testReadNestedCFP > -Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089653#comment-13089653 ] Simon Willnauer commented on LUCENE-3218: - I don't really like the name IndexInputHandle what about * IndexInputFactory * IndexInputProducer * IndexInputSlicer more ideas? > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-3398) TestCompoundFile fails on windows
[ https://issues.apache.org/jira/browse/LUCENE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer reassigned LUCENE-3398: --- Assignee: Simon Willnauer > TestCompoundFile fails on windows > - > > Key: LUCENE-3398 > URL: https://issues.apache.org/jira/browse/LUCENE-3398 > Project: Lucene - Java > Issue Type: Bug >Affects Versions: 4.0 >Reporter: Robert Muir >Assignee: Simon Willnauer > > ant test-core -Dtestcase=TestCompoundFile -Dtestmethod=testReadNestedCFP > -Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089639#comment-13089639 ] Mark Miller commented on LUCENE-3218: - bq. this seems close, the question is if we want to backport this to 3.x too? Why don't we get it committed to trunk and let it chill for a while, let it hit random testing for a while, get used by adventurous users, and then make the decision? > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3398) TestCompoundFile fails on windows
[ https://issues.apache.org/jira/browse/LUCENE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089638#comment-13089638 ] Simon Willnauer commented on LUCENE-3398: - I had this failure while working on LUCENE-3218 so there is a fix for this issue. However I will upload a patch then you can run it (I have no windows ready) and see if that fixes this issue too. Is it reproducible for you on Windows? > TestCompoundFile fails on windows > - > > Key: LUCENE-3398 > URL: https://issues.apache.org/jira/browse/LUCENE-3398 > Project: Lucene - Java > Issue Type: Bug >Affects Versions: 4.0 >Reporter: Robert Muir > > ant test-core -Dtestcase=TestCompoundFile -Dtestmethod=testReadNestedCFP > -Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089631#comment-13089631 ] Michael McCandless commented on LUCENE-3218: This approach looks nice! Maybe rename IndexInputHandle to IndexInputProvider? IndexInputSlicer? SliceCreator? Maybe rename CSIndexInput -> SlicedIndexInput? In SimpleFSDir we may as well move that static Descriptor class out? Rather than having to import it to itself. > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3398) TestCompoundFile fails on windows
[
https://issues.apache.org/jira/browse/LUCENE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089615#comment-13089615
]
Robert Muir commented on LUCENE-3398:
-
{noformat}
junit-sequential:
[junit] Testsuite: org.apache.lucene.index.TestCompoundFile
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.227 sec
[junit]
[junit] - Standard Error -
[junit] NOTE: reproduce with: ant test -Dtestcase=TestCompoundFile
-Dtestmethod=testReadNestedCFP
-Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c
[junit] NOTE: test params are: codec=RandomCodecProvider: {}, locale=de_DE,
timezone=Asia/Gaza
[junit] NOTE: all tests run in this JVM:
[junit] [TestCompoundFile]
[junit] NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23
(32-bit)/cpus=4,threads=1,free=11693688,total=16252928
[junit] - ---
[junit] Testcase:
testReadNestedCFP(org.apache.lucene.index.TestCompoundFile): Caused an
ERROR
[junit] Cannot delete
C:\Users\rmuir\workspace\trunky\lucene\build\test\1\test5610056101tmp\b_1.xyz
[junit] java.io.IOException: Cannot delete
C:\Users\rmuir\workspace\trunky\lucene\build\test\1\test5610056101tmp\b_1.xyz
[junit] at
org.apache.lucene.store.FSDirectory.deleteFile(FSDirectory.java:287)
[junit] at
org.apache.lucene.store.CompoundFileWriter.copyFileEntry(CompoundFileWriter.java:205)
[junit] at
org.apache.lucene.store.CompoundFileWriter.prunePendingEntries(CompoundFileWriter.java:272)
[junit] at
org.apache.lucene.store.CompoundFileWriter.access$200(CompoundFileWriter.java:59)
[junit] at
org.apache.lucene.store.CompoundFileWriter$DirectCFSIndexOutput.close(CompoundFileWriter.java:335)
[junit] at
org.apache.lucene.index.TestCompoundFile.testReadNestedCFP(TestCompoundFile.java:722)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1530)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1432)
[junit]
[junit]
[junit] Test org.apache.lucene.index.TestCompoundFile FAILED
{noformat}
> TestCompoundFile fails on windows
> -
>
> Key: LUCENE-3398
> URL: https://issues.apache.org/jira/browse/LUCENE-3398
> Project: Lucene - Java
> Issue Type: Bug
>Affects Versions: 4.0
>Reporter: Robert Muir
>
> ant test-core -Dtestcase=TestCompoundFile -Dtestmethod=testReadNestedCFP
> -Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3398) TestCompoundFile fails on windows
TestCompoundFile fails on windows - Key: LUCENE-3398 URL: https://issues.apache.org/jira/browse/LUCENE-3398 Project: Lucene - Java Issue Type: Bug Affects Versions: 4.0 Reporter: Robert Muir ant test-core -Dtestcase=TestCompoundFile -Dtestmethod=testReadNestedCFP -Dtests.seed=-61cb66ec0d71d1ac:-46685c36ec38fd32:568c63299214892c -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089578#comment-13089578 ] Steven Rowe commented on SOLR-2727: --- bq. SOLR-2020 addresses the same. Not quite? AFAICT, SOLR-2020 doesn't address internal uses of httpclient 3.x - it's only about providing an httpcomponent-4.x-based client. > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314116041)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314116041.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravind Srini resolved SOLR-2727. - Resolution: Duplicate SOLR-2020 addresses the same. > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Separate issue for appendable field caches / doc values
On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen wrote: >> But, we are trying to move FC under IR control (Martijn has a patch), >> at which point an RT IR could have its own appending impl...? > > LUCENE-3360? It's placing the field cache into IR / SR. Yes! > The RAM > reader could return it's own impl where the underlying array can be > atomically replaced (when a larger sized array is needed). I agree > that is logical. Good. >> But then... FC still returns fixed arrays so you can't append until we fix >> that? > > I don't think anything should depend on the size of the field cache > array. If it does, it seems odd. Are you preferring moving field > cache access to method calls? What is the difference between that and > primitive array access? Oh duh right -- you should be able to alloc too-large arrays and only realloc once you run out. So amortized cost is low but some reopens will take a hit to grow > For now I will create an independent field cache implementation that > is appendable. It will only be associate-able with the DWPT / RAM > reader. Maybe somehow it can work with LUCENE-3360 and / or the > existing static FC access system. Sounds good. > Still not sure how doc values fits in. DV is also provided by the IR (perDocValues method) so the RT reader should be able to impl its own. Each lookup is a method call so it should be easier to back that w/ a more RT friendly data structure... Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089564#comment-13089564 ] Steven Rowe commented on SOLR-2727: --- Some of this work has already been done: SOLR-2020. > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314111661)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314111661.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: trunk test failure (1314109382)
This is really annoying. Can somebody take care of this failure here:
ant test -Dtestcase=AutoCommitTest
-Dtestmethod=testSoftAndHardCommitMaxTime
-Dtests.seed=-5ee7e08a027abd02:-253353831f8290ef:1632dafde7a5bd7f
-Dtests.multiplier=3
[junit] NOTE: test params are: codec=RandomCodecProvider:
{range_facet_si=Standard(minBlockSize=35 maxBlockSize=87),
intDefault=SimpleText,
range_facet_l=MockVariableIntBlock(baseBlockSize=80),
range_facet_sl=MockSep, timestamp=SimpleText,
multiDefault=Standard(minBlockSize=35 maxBlockSize=87),
id=MockFixedIntBlock(blockSize=83), subject=Pulsing(freqCutoff=18
minBlockSize=18 maxBlockSize=42), text=MockSep, field_t=MockRandom},
locale=et_EE_PREEURO, timezone=America/Sao_Paulo
simon
On Tue, Aug 23, 2011 at 5:00 PM, Charlie Cron wrote:
> A test failure occurred running the tests.
>
> revert! revert! revert!
>
> You can see the entire build log at
> http://sierranevada.servebeer.com/1314109382.log
>
> Thanks,
> Charlie Cron
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314109382)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314109382.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Favre updated LUCENE-3392: -- Component/s: modules/analysis > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/analysis >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4, 4.0 > > Attachments: ComboAnalyzer-lucene-trunk.patch, > ComboAnalyzer-lucene3x.patch, ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Favre updated LUCENE-3392: -- Attachment: ComboAnalyzer-lucene3x.patch Moved analysis related changes into contrib/analysers/common, like the patch for the trunk. Small changes: - 2 space indentation (was 4 before, my personal default value) - removed a few useless imports - simplified ComboTokenStream, and fixes, as I saw functions have become final in the trunk. > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/analysis >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4, 4.0 > > Attachments: ComboAnalyzer-lucene-trunk.patch, > ComboAnalyzer-lucene3x.patch, ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 295 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/295/ 1 tests failed. REGRESSION: org.apache.solr.update.AutoCommitTest.testSoftAndHardCommitMaxTime Error Message: null Stack Trace: junit.framework.AssertionFailedError: at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1530) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1432) at org.apache.solr.update.AutoCommitTest.testSoftAndHardCommitMaxTime(AutoCommitTest.java:454) Build Log (for compile errors): [...truncated 11162 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Favre updated LUCENE-3392: -- Fix Version/s: 4.0 > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4, 4.0 > > Attachments: ComboAnalyzer-lucene-trunk.patch, > ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Favre updated LUCENE-3392: -- Attachment: ComboAnalyzer-lucene-trunk.patch Patch for lucene-trunk. Tested with sun's Java 1.6.0_26-b03. Adds support for Reader cloning in lucene's core, and the analysis stuff in modules/analysis/common > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4 > > Attachments: ComboAnalyzer-lucene-trunk.patch, > ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
[ https://issues.apache.org/jira/browse/SOLR-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089500#comment-13089500 ] Aravind Srini commented on SOLR-2727: - Initial patch to be worked on, w.r.t trunk ,targeting the 4.x . > Upgrade httpclient to 4.1.2 (from 3.0.1 ) > -- > > Key: SOLR-2727 > URL: https://issues.apache.org/jira/browse/SOLR-2727 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.3 >Reporter: Aravind Srini > Fix For: 4.0 > > > Currently solr depends on commons-httpclient 3.x. EOL has been announced , > for some time , for that release line. > Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 > . > jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" > thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2727) Upgrade httpclient to 4.1.2 (from 3.0.1 )
Upgrade httpclient to 4.1.2 (from 3.0.1 ) -- Key: SOLR-2727 URL: https://issues.apache.org/jira/browse/SOLR-2727 Project: Solr Issue Type: Improvement Affects Versions: 3.3 Reporter: Aravind Srini Fix For: 4.0 Currently solr depends on commons-httpclient 3.x. EOL has been announced , for some time , for that release line. Need to upgrade the same, to httpclient 4.1.x , to begin with. Targeting 4.0 . jira logged as per the discussion of "solr - httpclient from 3.x to 4.1.x" thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: solr - httpclient from 3.x to 4.1.x
Thanks, Ryan. SOLR-2727 logged , to track the same. On Tue, Aug 23, 2011 at 6:37 PM, Ryan McKinley wrote: > I think upgrading to 4.x would be great -- for sure in /trunk, but > probably not in 3.x (since back compatibility would be complicated) > > Migration has not happened because the 4.x migration is non-trivial > and back compatibility issues. > > If you are willing to make the patch, i will review shepherd it along > > thanks > ryan > > > > On Tue, Aug 23, 2011 at 2:57 AM, Arvind Srini > wrote: > > Solr Server and hence, solrj, depends on commons-httpclient at the > moment. > > As we use the various solrj libraries in our app, we are using > httpclient > > 4.1.x (eol has been announced for httpclient 3.x sometime ago). > > Going through archives, I see some interest expressed in the form of > > SOLR-861, say to move to httpclient 4.1.x codebase. I am trying to > > understand the sense of the community towards that long pending migration > > and am willing to work on the patch, but am interested in knowing some > > thoughts before I spend time on the same. Thanks ! > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
[jira] [Commented] (LUCENE-2959) [GSoC] Implementing State of the Art Ranking for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089453#comment-13089453 ] Robert Muir commented on LUCENE-2959: - I think we can defer that one or just leave it open for consideration later. As far as this issue, lets keep it open until we merge branch to trunk! > [GSoC] Implementing State of the Art Ranking for Lucene > --- > > Key: LUCENE-2959 > URL: https://issues.apache.org/jira/browse/LUCENE-2959 > Project: Lucene - Java > Issue Type: New Feature > Components: core/query/scoring, general/javadocs, modules/examples >Reporter: David Mark Nemeskey >Assignee: Robert Muir > Labels: gsoc2011, lucene-gsoc-11, mentor > Fix For: flexscoring branch > > Attachments: LUCENE-2959_mockdfr.patch, implementation_plan.pdf, > proposal.pdf > > > Lucene employs the Vector Space Model (VSM) to rank documents, which compares > unfavorably to state of the art algorithms, such as BM25. Moreover, the > architecture is > tailored specically to VSM, which makes the addition of new ranking functions > a non- > trivial task. > This project aims to bring state of the art ranking methods to Lucene and to > implement a > query architecture with pluggable ranking functions. > The wiki page for the project can be found at > http://wiki.apache.org/lucene-java/SummerOfCode2011ProjectRanking. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-1863) spellchecker leaks file on core reload
[ https://issues.apache.org/jira/browse/SOLR-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arne de Bruijn closed SOLR-1863. Resolution: Fixed Fix Version/s: 1.4.1 This was no longer a problem with solr 1.4.1 > spellchecker leaks file on core reload > -- > > Key: SOLR-1863 > URL: https://issues.apache.org/jira/browse/SOLR-1863 > Project: Solr > Issue Type: Bug > Components: spellchecker > Environment: linux i386 (ubuntu 8.04) >Reporter: Arne de Bruijn > Fix For: 1.4.1 > > Attachments: SOLR-1863.patch > > > When reloading a core of a multicore solr 1.4.0 instance with > /admin/cores?action=RELOAD&core=name an extra reference to the spellchecker > cfs file appears in the list of open files of the java process. A forced gc > (with jconsole) does not help. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2959) [GSoC] Implementing State of the Art Ranking for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089449#comment-13089449 ] David Mark Nemeskey commented on LUCENE-2959: - Robert: maybe we could resolve this issue as well? Once we decide what to do with 3173 -- perhaps a won'tfix? > [GSoC] Implementing State of the Art Ranking for Lucene > --- > > Key: LUCENE-2959 > URL: https://issues.apache.org/jira/browse/LUCENE-2959 > Project: Lucene - Java > Issue Type: New Feature > Components: core/query/scoring, general/javadocs, modules/examples >Reporter: David Mark Nemeskey >Assignee: Robert Muir > Labels: gsoc2011, lucene-gsoc-11, mentor > Fix For: flexscoring branch > > Attachments: LUCENE-2959_mockdfr.patch, implementation_plan.pdf, > proposal.pdf > > > Lucene employs the Vector Space Model (VSM) to rank documents, which compares > unfavorably to state of the art algorithms, such as BM25. Moreover, the > architecture is > tailored specically to VSM, which makes the addition of new ranking functions > a non- > trivial task. > This project aims to bring state of the art ranking methods to Lucene and to > implement a > query architecture with pluggable ranking functions. > The wiki page for the project can be found at > http://wiki.apache.org/lucene-java/SummerOfCode2011ProjectRanking. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3393) Rename EasySimilarity to SimilarityBase
[ https://issues.apache.org/jira/browse/LUCENE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-3393. - Resolution: Fixed thanks David! > Rename EasySimilarity to SimilarityBase > --- > > Key: LUCENE-3393 > URL: https://issues.apache.org/jira/browse/LUCENE-3393 > Project: Lucene - Java > Issue Type: Sub-task > Components: core/query/scoring, general/javadocs, modules/examples >Reporter: David Mark Nemeskey >Assignee: David Mark Nemeskey > Fix For: flexscoring branch > > Attachments: LUCENE-3393.patch > > Original Estimate: 1h > Remaining Estimate: 1h > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Could possible donate webapp for dynamic core create/deletion.
Sure thing... Basically, we mimicked the core creation script and procedure described here: http://blog.dustinrue.com/archives/690 We wrapped it that process in a RESTful web service. A client can post a schema to the service, which will create the file for you then POST to SOLR to create the core. The web service is configured using a properties file right now, which among other things has a list of hosts. It will loop through the hosts and perform this operation on each host. If one fails, it rolls the core creation back on each host. If you want, I could pass along the WADL that we have for the service. -brian From: mohit soni [mailto:mohitsoni1...@gmail.com] Sent: Tuesday, August 23, 2011 5:32 AM To: dev@lucene.apache.org Subject: Re: Could possible donate webapp for dynamic core create/deletion. Hi Brian Can you share a brief summary of the work done, features offered, etc. ~mohit On Mon, Aug 22, 2011 at 6:43 PM, Brian O'Neill wrote: All, My team has developed a small web app that can dynamically create/delete cores in a cluster of SOLR instances. Is this feature already under development? Is anyone interested in it? If so, we might be able to donate it. -brian -- Brian O'Neill Lead Architect, Software Development Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406 p: 215.588.6024 www.healthmarketscience.com
[jira] [Commented] (LUCENE-3393) Rename EasySimilarity to SimilarityBase
[ https://issues.apache.org/jira/browse/LUCENE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089445#comment-13089445 ] Robert Muir commented on LUCENE-3393: - Looks good to me, seems like reasonable names at the moment. > Rename EasySimilarity to SimilarityBase > --- > > Key: LUCENE-3393 > URL: https://issues.apache.org/jira/browse/LUCENE-3393 > Project: Lucene - Java > Issue Type: Sub-task > Components: core/query/scoring, general/javadocs, modules/examples >Reporter: David Mark Nemeskey >Assignee: David Mark Nemeskey > Fix For: flexscoring branch > > Attachments: LUCENE-3393.patch > > Original Estimate: 1h > Remaining Estimate: 1h > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3387) Get javadoc for the similarities package in shape
[ https://issues.apache.org/jira/browse/LUCENE-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089443#comment-13089443 ] David Mark Nemeskey commented on LUCENE-3387: - bq. This is because of an out of date regexp in the javadocs construction. I've found that, I just didn't know what to make of it. Since as far as I know a similarities package hadn't existed before I added the new sims, I assumed it was there on purpose. > Get javadoc for the similarities package in shape > - > > Key: LUCENE-3387 > URL: https://issues.apache.org/jira/browse/LUCENE-3387 > Project: Lucene - Java > Issue Type: Sub-task > Components: core/query/scoring, general/javadocs >Reporter: David Mark Nemeskey >Assignee: David Mark Nemeskey > Labels: gsoc, gsoc2011, javadoc > Fix For: flexscoring branch > > Attachments: LUCENE-3387.patch, LUCENE-3387.patch > > > 1. Create a package.html in the similarities package. > 2. Update the javadoc of the search package (package.html mentions > Similarity)? > 3. Compile the javadoc to see if there are any warnings. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3391) Make EasySimilarityProvider a full-fledged class
[
https://issues.apache.org/jira/browse/LUCENE-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-3391.
-
Resolution: Fixed
Thanks David!
> Make EasySimilarityProvider a full-fledged class
> -
>
> Key: LUCENE-3391
> URL: https://issues.apache.org/jira/browse/LUCENE-3391
> Project: Lucene - Java
> Issue Type: Sub-task
> Components: core/query/scoring, general/javadocs
>Reporter: David Mark Nemeskey
>Assignee: David Mark Nemeskey
> Labels: gsoc, gsoc2011, rank, similarity
> Fix For: flexscoring branch
>
> Attachments: LUCENE-3391.patch, LUCENE-3391.patch, LUCENE-3391.patch,
> LUCENE-3391.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The {{EasySimilarityProvider}} in {{TestEasySimilarity}} would be a good
> candidate for a full-fledged class. Both {{DefaultSimilarity}} and
> {{BM25Similarity}} have their own providers, which are effectively the
> same,so I don't see why we couldn't add one generic provider for convenience.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: solr - httpclient from 3.x to 4.1.x
I think upgrading to 4.x would be great -- for sure in /trunk, but probably not in 3.x (since back compatibility would be complicated) Migration has not happened because the 4.x migration is non-trivial and back compatibility issues. If you are willing to make the patch, i will review shepherd it along thanks ryan On Tue, Aug 23, 2011 at 2:57 AM, Arvind Srini wrote: > Solr Server and hence, solrj, depends on commons-httpclient at the moment. > As we use the various solrj libraries in our app, we are using httpclient > 4.1.x (eol has been announced for httpclient 3.x sometime ago). > Going through archives, I see some interest expressed in the form of > SOLR-861, say to move to httpclient 4.1.x codebase. I am trying to > understand the sense of the community towards that long pending migration > and am willing to work on the patch, but am interested in knowing some > thoughts before I spend time on the same. Thanks ! - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3387) Get javadoc for the similarities package in shape
[ https://issues.apache.org/jira/browse/LUCENE-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-3387. - Resolution: Fixed thanks, no more javadocs warnings! > Get javadoc for the similarities package in shape > - > > Key: LUCENE-3387 > URL: https://issues.apache.org/jira/browse/LUCENE-3387 > Project: Lucene - Java > Issue Type: Sub-task > Components: core/query/scoring, general/javadocs >Reporter: David Mark Nemeskey >Assignee: David Mark Nemeskey > Labels: gsoc, gsoc2011, javadoc > Fix For: flexscoring branch > > Attachments: LUCENE-3387.patch, LUCENE-3387.patch > > > 1. Create a package.html in the similarities package. > 2. Update the javadoc of the search package (package.html mentions > Similarity)? > 3. Compile the javadoc to see if there are any warnings. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3387) Get javadoc for the similarities package in shape
[
https://issues.apache.org/jira/browse/LUCENE-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089433#comment-13089433
]
Robert Muir commented on LUCENE-3387:
-
Thanks David, looks good.
{quote}
The similarities package shows up in the 'core', even though it is classified
as 'contrib' for javadocs-all. However, since the class Similarity is now in
similarities, shouldn't it be core as well?
{quote}
This is because of an out of date regexp in the javadocs construction. I'll
nuke this before committing :)
{noformat}
{noformat}
> Get javadoc for the similarities package in shape
> -
>
> Key: LUCENE-3387
> URL: https://issues.apache.org/jira/browse/LUCENE-3387
> Project: Lucene - Java
> Issue Type: Sub-task
> Components: core/query/scoring, general/javadocs
>Reporter: David Mark Nemeskey
>Assignee: David Mark Nemeskey
> Labels: gsoc, gsoc2011, javadoc
> Fix For: flexscoring branch
>
> Attachments: LUCENE-3387.patch, LUCENE-3387.patch
>
>
> 1. Create a package.html in the similarities package.
> 2. Update the javadoc of the search package (package.html mentions
> Similarity)?
> 3. Compile the javadoc to see if there are any warnings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3386) Integrate MockBM25Similarity and MockLMSimilarity into the framework
[
https://issues.apache.org/jira/browse/LUCENE-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-3386.
-
Resolution: Fixed
Thanks David, good thorough refactor here :)
> Integrate MockBM25Similarity and MockLMSimilarity into the framework
>
>
> Key: LUCENE-3386
> URL: https://issues.apache.org/jira/browse/LUCENE-3386
> Project: Lucene - Java
> Issue Type: Sub-task
> Components: core/query/scoring
>Reporter: David Mark Nemeskey
>Assignee: David Mark Nemeskey
> Labels: gsoc, gsoc2011, rank
> Fix For: flexscoring branch
>
> Attachments: LUCENE-3386.patch, LUCENE-3386.patch, LUCENE-3386.patch,
> LUCENE-3386.patch, LUCENE-3386.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Steps:
> 1. Decide if {{MockLMSimilarity}} is needed at all (we have
> {{LMDirichletSimilarity}})
> 2. Move the classes to the similarities package
> 3. Move the similarities package to src/
> 4. Move all sims (inc. Similarity) to similarities
> 5. Make MockBM25Similarity a subclass of EasySimilarity?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3396) Make TokenStream Reuse Mandatory for Analyzers
[ https://issues.apache.org/jira/browse/LUCENE-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089417#comment-13089417 ] Chris Male commented on LUCENE-3396: Done. LUCENE-3397 > Make TokenStream Reuse Mandatory for Analyzers > -- > > Key: LUCENE-3396 > URL: https://issues.apache.org/jira/browse/LUCENE-3396 > Project: Lucene - Java > Issue Type: Improvement > Components: modules/analysis >Reporter: Chris Male > > In LUCENE-2309 it became clear that we'd benefit a lot from Analyzer having > to return reusable TokenStreams. This is a big chunk of work, but its time > to bite the bullet. > I plan to attack this in the following way: > - Collapse the logic of ReusableAnalyzerBase into Analyzer > - Add a ReuseStrategy abstraction to Analyzer which controls whether the > TokenStreamComponents are reused globally (as they are today) or per-field. > - Convert all Analyzers over to using TokenStreamComponents. I've already > seen that some of the TokenStreams created in tests need some work to be > reusable (even if they aren't reused). > - Remove Analyzer.reusableTokenStream and convert everything over to using > .tokenStream (which will now be returning reusable TokenStreams). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3397) Cleanup Test TokenStreams so they are reusable
Cleanup Test TokenStreams so they are reusable -- Key: LUCENE-3397 URL: https://issues.apache.org/jira/browse/LUCENE-3397 Project: Lucene - Java Issue Type: Sub-task Components: modules/analysis Reporter: Chris Male Many TokenStreams created in tests are not reusable. Some do some really messy things which prevent their reuse so we may have to change the tests themselves. We'll target back porting this to 3x. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089415#comment-13089415 ] Olivier Favre commented on LUCENE-3392: --- The proposed implementation may a have tight bond with the JVM implementation of some classes (StringReader, BufferedReader and FilterReader), as they rely on a named private field (respectively "str", "in" and "in"). This can be avoided, but any Reader should then be fully read and stored as a String or a char[], which can have a huge overhead. Considering each clone would get read relatively at the same speed (well, only for word delimiting analysis, not for a KeywordAnalyzer) an implementation could only retain in memory the portion read by at least one cloned reader but not all clones, in order to implement a "multi read head" reader. Another implementation would be to change the API to give a CloneableReader interface with a "giveAClone()" function instead of a Reader for tokenStream and reusableTokenStream functions. But this involves massive refactoring (>13,000 lines) and introduces an important API break. The proposed implementation is the best solution I found. Any suggestions are welcome! > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4 > > Attachments: ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3396) Make TokenStream Reuse Mandatory for Analyzers
[ https://issues.apache.org/jira/browse/LUCENE-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089414#comment-13089414 ] Robert Muir commented on LUCENE-3396: - yeah: just thinking it could be a nice cleanup? Some of these are messy :) > Make TokenStream Reuse Mandatory for Analyzers > -- > > Key: LUCENE-3396 > URL: https://issues.apache.org/jira/browse/LUCENE-3396 > Project: Lucene - Java > Issue Type: Improvement > Components: modules/analysis >Reporter: Chris Male > > In LUCENE-2309 it became clear that we'd benefit a lot from Analyzer having > to return reusable TokenStreams. This is a big chunk of work, but its time > to bite the bullet. > I plan to attack this in the following way: > - Collapse the logic of ReusableAnalyzerBase into Analyzer > - Add a ReuseStrategy abstraction to Analyzer which controls whether the > TokenStreamComponents are reused globally (as they are today) or per-field. > - Convert all Analyzers over to using TokenStreamComponents. I've already > seen that some of the TokenStreams created in tests need some work to be > reusable (even if they aren't reused). > - Remove Analyzer.reusableTokenStream and convert everything over to using > .tokenStream (which will now be returning reusable TokenStreams). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3392) Combining analyzers output
[ https://issues.apache.org/jira/browse/LUCENE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Favre updated LUCENE-3392: -- Attachment: ComboAnalyzer-lucene3x.patch Patch for lucene-3x. Tested with Sun's Java 1.6.0_26-b03. Uses a special factory for cloning Readers, some implementation use reflection to gain access to private fields in order to reduce the need to read and copy a Readers' content. > Combining analyzers output > -- > > Key: LUCENE-3392 > URL: https://issues.apache.org/jira/browse/LUCENE-3392 > Project: Lucene - Java > Issue Type: New Feature >Reporter: Olivier Favre >Priority: Minor > Labels: analysis > Fix For: 3.4 > > Attachments: ComboAnalyzer-lucene3x.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > It should be easy to combine the output of multiple Analyzers, or > TokenStreams. > A ComboAnalyzer and a ComboTokenStream class would take multiple instances, > and multiplex their output, keeping a rough order of tokens like increasing > position then increasing start offset then increasing end offset. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314099241)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314099241.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3396) Make TokenStream Reuse Mandatory for Analyzers
[ https://issues.apache.org/jira/browse/LUCENE-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089396#comment-13089396 ] Chris Male commented on LUCENE-3396: Do you mean correcting the TokenStreams in tests so they are reusable? > Make TokenStream Reuse Mandatory for Analyzers > -- > > Key: LUCENE-3396 > URL: https://issues.apache.org/jira/browse/LUCENE-3396 > Project: Lucene - Java > Issue Type: Improvement > Components: modules/analysis >Reporter: Chris Male > > In LUCENE-2309 it became clear that we'd benefit a lot from Analyzer having > to return reusable TokenStreams. This is a big chunk of work, but its time > to bite the bullet. > I plan to attack this in the following way: > - Collapse the logic of ReusableAnalyzerBase into Analyzer > - Add a ReuseStrategy abstraction to Analyzer which controls whether the > TokenStreamComponents are reused globally (as they are today) or per-field. > - Convert all Analyzers over to using TokenStreamComponents. I've already > seen that some of the TokenStreams created in tests need some work to be > reusable (even if they aren't reused). > - Remove Analyzer.reusableTokenStream and convert everything over to using > .tokenStream (which will now be returning reusable TokenStreams). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3396) Make TokenStream Reuse Mandatory for Analyzers
[ https://issues.apache.org/jira/browse/LUCENE-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089394#comment-13089394 ] Robert Muir commented on LUCENE-3396: - can we create a separate sub-tasks to fix the tests (so we can backport at least a majority of the changes)? > Make TokenStream Reuse Mandatory for Analyzers > -- > > Key: LUCENE-3396 > URL: https://issues.apache.org/jira/browse/LUCENE-3396 > Project: Lucene - Java > Issue Type: Improvement > Components: modules/analysis >Reporter: Chris Male > > In LUCENE-2309 it became clear that we'd benefit a lot from Analyzer having > to return reusable TokenStreams. This is a big chunk of work, but its time > to bite the bullet. > I plan to attack this in the following way: > - Collapse the logic of ReusableAnalyzerBase into Analyzer > - Add a ReuseStrategy abstraction to Analyzer which controls whether the > TokenStreamComponents are reused globally (as they are today) or per-field. > - Convert all Analyzers over to using TokenStreamComponents. I've already > seen that some of the TokenStreams created in tests need some work to be > reusable (even if they aren't reused). > - Remove Analyzer.reusableTokenStream and convert everything over to using > .tokenStream (which will now be returning reusable TokenStreams). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1863) spellchecker leaks file on core reload
[ https://issues.apache.org/jira/browse/SOLR-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089390#comment-13089390 ] Markus Jelsma commented on SOLR-1863: - Is this still an issue in 3.3 or 4.x? > spellchecker leaks file on core reload > -- > > Key: SOLR-1863 > URL: https://issues.apache.org/jira/browse/SOLR-1863 > Project: Solr > Issue Type: Bug > Components: spellchecker > Environment: linux i386 (ubuntu 8.04) >Reporter: Arne de Bruijn > Attachments: SOLR-1863.patch > > > When reloading a core of a multicore solr 1.4.0 instance with > /admin/cores?action=RELOAD&core=name an extra reference to the spellchecker > cfs file appears in the list of open files of the java process. A forced gc > (with jconsole) does not help. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314095281)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314095281.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2726) NullPointerException when using spellcheck.q
[ https://issues.apache.org/jira/browse/SOLR-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] valentin updated SOLR-2726: --- Description: When I use spellcheck.q in my query to define what will be "spellchecked", I always have this error, for every configuration I try : java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:476) at org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:131) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:202) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) All my other functions works great, this is the only thing which doesn't work at all, just when i add "&spellcheck.q=my%20sentence" in the query... Example of a query : http://localhost:8983/solr/db/suggest_full?q=american%20israel&spellcheck.q=american%20israel In solrconfig.xml : suggestTextFull suggest_full org.apache.solr.spelling.suggest.Suggester org.apache.solr.spelling.suggest.tst.TSTLookup text_suggest_full suggestTextFull true suggest_full 10 true suggest_full I'm using SolR 3.3, and I tried it too on SolR 4.0 was: When I use spellcheck.q in my query to define what will be "spellchecked", I always have this error, for every configuration I try : java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:476) at org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:131) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:202) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at o
[jira] [Updated] (SOLR-2726) NullPointerException when using spellcheck.q
[ https://issues.apache.org/jira/browse/SOLR-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] valentin updated SOLR-2726: --- Description: When I use spellcheck.q in my query to define what will be "spellchecked", I always have this error, for every configuration I try : java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:476) at org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:131) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:202) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) All my other functions works great, this is the only thing which doesn't work at all, just when i add "&spellcheck.q=my%20sentence" in the query... I'm using SolR 3.3, and I tried it too on SolR 4.0 was: When I use spellcheck.q in my query to define what will be "spellchecked", I always have this error, for every configuration I try : java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:476) at org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:131) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:202) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) All my other functions works great, this is the only thing which doesn't work at all, just when i add "&spellcheck.q=my%20sentence" in the query... I'm using SolR 3.3. Affects Version/s: 4.0 > NullPointerException when using spellcheck.q > > > Key: SOLR-2726 > URL: https://issues.apache.org/jira/browse/SOLR-2726 >
Re: Could possible donate webapp for dynamic core create/deletion.
Hi Brian Can you share a brief summary of the work done, features offered, etc. ~mohit On Mon, Aug 22, 2011 at 6:43 PM, Brian O'Neill < bone...@healthmarketscience.com> wrote: > All, > > ** ** > > My team has developed a small web app that can dynamically create/delete > cores in a cluster of SOLR instances. Is this feature already under > development? Is anyone interested in it? If so, we might be able to donate > it. > > ** ** > > -brian > > ** ** > > -- > Brian O'Neill > Lead Architect, Software Development > Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406 > p: 215.588.6024 > *www.healthmarketscience.com* > > ** ** >
[jira] [Updated] (LUCENE-3345) docvalues FNFE
[ https://issues.apache.org/jira/browse/LUCENE-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-3345: Attachment: LUCENE-3345.patch here is a patch that makes the CFS option private to the codec. Currently all codecs default to use CFS for docvalues. For testcoverage I added a MockRandomDocValues codec that always uses noCFS. If you want to change to noCFS for docvalues you need to subclass a codec and give it a name since those settings are final now. > docvalues FNFE > -- > > Key: LUCENE-3345 > URL: https://issues.apache.org/jira/browse/LUCENE-3345 > Project: Lucene - Java > Issue Type: Bug >Reporter: Robert Muir >Assignee: Simon Willnauer > Attachments: LUCENE-3345.patch > > > I created a test for LUCENE-3335, and it found an unrelated bug in docvalues. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3218) Make CFS appendable
[ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-3218: Attachment: LUCENE-3218.patch next iteration * removed 3.4 support for the current format (cfe file) in SI * regenerated the BW test indices (not in the patch) * add javadoc for IndexInputHandle * added IndexInputHandle#openFullSlice to get a slice spanning the entire file. * Track indexInputHandle instances in MockDirectoryWrapper to ensure they are closed. * Use the IndexInputHandle ie. the underlying file handle to create all streams in CFS (uwes suggestion - thanks for that) I didn't include the generated indices for bw tests in the patch for size / readability. Yet, if you want to run the tests you need to generate them otherwise TestBackwardsCompatibility will fail. this seems close, the question is if we want to backport this to 3.x too? > Make CFS appendable > - > > Key: LUCENE-3218 > URL: https://issues.apache.org/jira/browse/LUCENE-3218 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Simon Willnauer >Priority: Blocker > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, > LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch, LUCENE-3218_tests.patch > > > Currently CFS is created once all files are written during a flush / merge. > Once on disk the files are copied into the CFS format which is basically a > unnecessary for some of the files. We can at any time write at least one file > directly into the CFS which can save a reasonable amount of IO. For instance > stored fields could be written directly during indexing and during a Codec > Flush one of the written files can be appended directly. This optimization is > a nice sideeffect for lucene indexing itself but more important for DocValues > and LUCENE-3216 we could transparently pack per field files into a single > file only for docvalues without changing any code once LUCENE-3216 is > resolved. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3396) Make TokenStream Reuse Mandatory for Analyzers
Make TokenStream Reuse Mandatory for Analyzers -- Key: LUCENE-3396 URL: https://issues.apache.org/jira/browse/LUCENE-3396 Project: Lucene - Java Issue Type: Improvement Components: modules/analysis Reporter: Chris Male In LUCENE-2309 it became clear that we'd benefit a lot from Analyzer having to return reusable TokenStreams. This is a big chunk of work, but its time to bite the bullet. I plan to attack this in the following way: - Collapse the logic of ReusableAnalyzerBase into Analyzer - Add a ReuseStrategy abstraction to Analyzer which controls whether the TokenStreamComponents are reused globally (as they are today) or per-field. - Convert all Analyzers over to using TokenStreamComponents. I've already seen that some of the TokenStreams created in tests need some work to be reusable (even if they aren't reused). - Remove Analyzer.reusableTokenStream and convert everything over to using .tokenStream (which will now be returning reusable TokenStreams). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2726) NullPointerException when using spellcheck.q
NullPointerException when using spellcheck.q Key: SOLR-2726 URL: https://issues.apache.org/jira/browse/SOLR-2726 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.3 Environment: ubuntu Reporter: valentin When I use spellcheck.q in my query to define what will be "spellchecked", I always have this error, for every configuration I try : java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:476) at org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:131) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:202) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) All my other functions works great, this is the only thing which doesn't work at all, just when i add "&spellcheck.q=my%20sentence" in the query... I'm using SolR 3.3. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
trunk test failure (1314083161)
A test failure occurred running the tests. revert! revert! revert! You can see the entire build log at http://sierranevada.servebeer.com/1314083161.log Thanks, Charlie Cron - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3233) HuperDuperSynonymsFilter™
[ https://issues.apache.org/jira/browse/LUCENE-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089319#comment-13089319 ] Glen Stampoultzis commented on LUCENE-3233: --- I'll try and put together a simple test case just to make sure I've got something repeatable and post to the list. I think it might have been related to synonym fields tokenized using KeywordTokenizerFactory. > HuperDuperSynonymsFilter™ > - > > Key: LUCENE-3233 > URL: https://issues.apache.org/jira/browse/LUCENE-3233 > Project: Lucene - Java > Issue Type: Improvement >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.4, 4.0 > > Attachments: LUCENE-3223.patch, LUCENE-3233.patch, LUCENE-3233.patch, > LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, > LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, > LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, LUCENE-3233.patch, > LUCENE-3233.patch, LUCENE-3233.patch, synonyms.zip > > > The current synonymsfilter uses a lot of ram and cpu, especially at build > time. > I think yesterday I heard about "huge synonyms files" three times. > So, I think we should use an FST-based structure, sharing the inputs and > outputs. > And we should be more efficient with the tokenStream api, e.g. using > save/restoreState instead of cloneAttributes() -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
