[Lucene.Net] Medium trust security issue
Hi I am running into problems using Lucence 2.9.2 in a medium trust environment, namely Rackspace cloud. I have added the following line to assembleyinfo.cs [assembly: AllowPartiallyTrustedCallers()] However the following code produces the error below FSDirectory directory = FSDirectory.Open(new DirectoryInfo(Server.MapPath(~/App_Data/LuceneIndex))); Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_29); var writer = new IndexWriter(directory, analyzer, true, IndexWriter.MaxFieldLength.LIMITED); writer.AddDocument(...); writer.Optimize(); writer.Close(); The directory LuceneIndex is being created, does anyone have a fix for this? Security Exception Description: The application attempted to perform an operation not allowed by the security policy. To grant this application the required permission please contact your system administrator or change the application's trust level in the configuration file. Exception Details: System.Security.SecurityException: Request failed. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace: [SecurityException: Request failed.] FileSupport.Sync(FileStream fileStream) +0 Lucene.Net.Store.FSDirectory.Sync(String name) +157 Lucene.Net.Index.SegmentInfos.FinishCommit(Directory dir) +184 Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean create, Boolean closeDir, IndexDeletionPolicy deletionPolicy, Boolean autoCommit, Int32 maxFieldLength, IndexingChain indexingChain, IndexCommit commit) +293 Lucene.Net.Index.IndexWriter..ctor(Directory d, Analyzer a, Boolean create, MaxFieldLength mfl) +413 Mvc.Cms.Controllers.LuceneController.Index() +1066 lambda_method(Closure , ControllerBase , Object[] ) +40 System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17 System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters) +188 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +27 System.Web.Mvc.c__DisplayClass15.InvokeActionMethodWithFiltersb__12() +56 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilte r filter, ActionExecutingContext preContext, Func`1 continuation) +267 System.Web.Mvc.c__DisplayClass17.InvokeActionMethodWithFiltersb__14() +20 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(Control lerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +190 System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +329 System.Web.Mvc.Controller.ExecuteCore() +115 System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +94 System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestCont ext requestContext) +10 System.Web.Mvc.c__DisplayClassb.BeginProcessRequestb__5() +37 System.Web.Mvc.Async.c__DisplayClass1.MakeVoidDelegateb__0() +21 System.Web.Mvc.Async.c__DisplayClass8`1.BeginSynchronousb__7(IAsyncResul t _) +12 System.Web.Mvc.Async.WrappedAsyncResult`1.End() +55 System.Web.Mvc.c__DisplayClasse.EndProcessRequestb__d() +31 System.Web.Mvc.SecurityUtil.GetCallInAppTrustThunkb__0(Action f) +7 System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action) +23 System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +59 System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAs yncResult result) +9 System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionSte p.Execute() +8841105 System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean completedSynchronously) +184 _ Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.1 Many Thanks Rippo
[Lucene.Net] Incubator PMC/Board report for May 2011 (lucene-net-dev@lucene.apache.org)
Dear Lucene.NET Developers, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Thurs, 19 May 2011, 10 am Pacific. The report for your podling will form a part of the Incubator PMC report. The Incubator PMC requires your report to be submitted one week before the board meeting, to allow sufficient time for review. Please submit your report with sufficient time to allow the incubator PMC, and subsequently board members to review and digest. Again, the very latest you should submit your report is one week prior to the board meeting. Thanks, The Apache Incubator PMC Submitting your Report -- Your report should contain the following: * Your project name * A brief description of your project, which assumes no knowledge of the project or necessarily of its field * A list of the three most important issues to address in the move towards graduation. * Any issues that the Incubator PMC or ASF Board might wish/need to be aware of * How has the community developed since the last report * How has the project developed since the last report. This should be appended to the Incubator Wiki page at: http://wiki.apache.org/incubator/May2011 Note: This manually populated. You may need to wait a little before this page is created from a template. Mentors --- Mentors should review reports for their project(s) and sign them off on the Incubator wiki page. Signing off reports shows that you are following the project - projects that are not signed may raise alarms for the Incubator PMC. Incubator PMC
RE: [Lucene.Net] Medium trust security issue
Richard,
This is because the call to FileSupport.Sync ultimately calls
SupportClass.Sync(FileStream) which ends up calling the FlushFileBuffers API
function through the P/Invoke layer, which is disallowed in medium trust
environment.
However, this should be mitigated by the fact that you have set the
assembly to allow partially trusted callers (are you doing this as a check
in to the tree? If so, have you done a full security analysis? Setting
this attribute on an assembly as big as Lucene.NET has major security
implications).
It would seem to me that you might not have given Lucene.NET a
strong name; this is required for AllowPartiallyTrustedCallers to take
effect.
This issue was seen by Simone Chiaretta and was discussed in the
group a while ago:
http://web.archiveorange.com/archive/v/3k9XU33O4yJyW15fWfMd
However, at the time, Lucene.NET was built on .NET 2.0 (IIRC) and
didn't have access to the overload of the Flush method which was used to
guarantee everything was flushed to disk:
http://web.archiveorange.com/archive/v/3k9XU33O4yJyW15fWfMd#MhNDlmKgnUj5fOj
Since you are now working in .NET 4.0, you should be able to replace
the following code in SupportClass.cs
(https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/C%23/src/Lucene
.Net/SupportClass.cs):
public static void Sync(System.IO.FileStream fileStream)
{
if (fileStream == null)
throw new
ArgumentNullException(fileStream);
fileStream.Flush();
if (OS.IsWindows)
{
if (!FlushFileBuffers(fileStream.Handle))
throw new System.IO.IOException();
}
else if (OS.IsUnix)
{
if (fsync(fileStream.Handle) != IntPtr.Zero)
throw new System.IO.IOException();
}
else
{
throw new NotImplementedException();
}
}
With this:
public static void Sync(System.IO.FileStream fileStream)
{
if (fileStream == null)
throw new
ArgumentNullException(fileStream);
fileStream.Flush(true);
}
One could make the argument that this should be taken out of SupportClass
and moved into FSDirectory, but that might break some of your line-for-line
port code, so best to keep it in SupportClass.
- Nicholas Paldino [.NET/C# MVP]
-Original Message-
From: Richard Wilde [mailto:rich...@wildesoft.net]
Sent: Sunday, May 01, 2011 6:01 AM
To: lucene-net-...@incubator.apache.org
Subject: [Lucene.Net] Medium trust security issue
Hi
I am running into problems using Lucence 2.9.2 in a medium trust
environment, namely Rackspace cloud. I have added the following line to
assembleyinfo.cs
[assembly: AllowPartiallyTrustedCallers()]
However the following code produces the error below
FSDirectory directory = FSDirectory.Open(new
DirectoryInfo(Server.MapPath(~/App_Data/LuceneIndex)));
Analyzer analyzer = new
StandardAnalyzer(Version.LUCENE_29);
var writer = new IndexWriter(directory, analyzer, true,
IndexWriter.MaxFieldLength.LIMITED);
writer.AddDocument(...);
writer.Optimize();
writer.Close();
The directory LuceneIndex is being created, does anyone have a fix for
this?
Security Exception
Description: The application attempted to perform an operation not allowed
by the security policy. To grant this application the required permission
please contact your system administrator or change the application's trust
level in the configuration file.
Exception Details: System.Security.SecurityException: Request failed.
Source Error:
An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.
Stack Trace:
[SecurityException: Request failed.]
FileSupport.Sync(FileStream fileStream) +0
Lucene.Net.Store.FSDirectory.Sync(String name) +157
Lucene.Net.Index.SegmentInfos.FinishCommit(Directory dir) +184
Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean
create, Boolean closeDir, IndexDeletionPolicy deletionPolicy, Boolean
autoCommit, Int32 maxFieldLength, IndexingChain indexingChain, IndexCommit
commit) +293
Lucene.Net.Index.IndexWriter..ctor(Directory d, Analyzer a, Boolean
create, MaxFieldLength mfl) +413
Mvc.Cms.Controllers.LuceneController.Index() +1066
lambda_method(Closure ,
[jira] [Commented] (LUCENE-3023) Land DWPT on trunk
[ https://issues.apache.org/jira/browse/LUCENE-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027439#comment-13027439 ] Simon Willnauer commented on LUCENE-3023: - bq. In ThreadAffinityDocumentsWriterThreadPool#getAndLock() we had talked about switching from a per-threadstate queue (safeway model) to a single queue (whole foods). I'm wondering if we should do that before we commit or change that later as a separate patch? I will create a new issue for this I don't think this should block the commit merge! all other minor stuff can be done after committing. I will take care of these issues. Land DWPT on trunk -- Key: LUCENE-3023 URL: https://issues.apache.org/jira/browse/LUCENE-3023 Project: Lucene - Java Issue Type: Task Affects Versions: CSF branch, 4.0 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.0 Attachments: LUCENE-3023-svn-diff.patch, LUCENE-3023-ws-changes.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_iw_iwc_jdoc.patch, LUCENE-3023_simonw_review.patch, LUCENE-3023_svndiff.patch, LUCENE-3023_svndiff.patch, diffMccand.py, diffSources.patch, diffSources.patch, realtime-TestAddIndexes-3.txt, realtime-TestAddIndexes-5.txt, realtime-TestIndexWriterExceptions-assert-6.txt, realtime-TestIndexWriterExceptions-npe-1.txt, realtime-TestIndexWriterExceptions-npe-2.txt, realtime-TestIndexWriterExceptions-npe-4.txt, realtime-TestOmitTf-corrupt-0.txt With LUCENE-2956 we have resolved the last remaining issue for LUCENE-2324 so we can proceed landing the DWPT development on trunk soon. I think one of the bigger issues here is to make sure that all JavaDocs for IW etc. are still correct though. I will start going through that first. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Link to nightly build test reports on main Lucene site needs updating
I fixed the nightly docs, once the webserver mirrors them from SVN they should appear. The developer-resources page was completely broken. It now also contains references to the stable 3.x branch as most users would prefer that one to fix latest bugs but don’t want to have a backwards-incompatible version. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Simon Willnauer [mailto:simon.willna...@googlemail.com] Sent: Saturday, April 30, 2011 9:42 PM To: java-u...@lucene.apache.org Cc: dev@lucene.apache.org Subject: Re: Link to nightly build test reports on main Lucene site needs updating thanks tom, I cced dev@l.a.o simon On Fri, Apr 29, 2011 at 11:14 PM, Burton-West, Tom tburt...@umich.edu wrote: Hello, I went to look at the Hudson nightly builds and tried to follow the link from the main Lucene page http://lucene.apache.org/java/docs/developer-resources.html#Nightly The links to the Clover Test Coverage Reports point to http://hudson.zones.apache.org/hudson/view/Lucene/job/Lucene- trunk/lastSuccessfulBuild/clover/ but apparently hudson.zones.apache.org is no longer being used. I think the link should point to somewhere on https://builds.apache.org/hudson/job/Lucene-trunk/. Is this the right list to alert whoever maintains the main Lucene pages on lucene.apache.org? Tom - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3023) Land DWPT on trunk
[ https://issues.apache.org/jira/browse/LUCENE-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027447#comment-13027447 ] Uwe Schindler commented on LUCENE-3023: --- Thanks Simon! As noted before, this is minor and should be done after the merge was committed, else it would produce lot of useless work :( I will commit this tomorrow morning MEST. After committing, I will move the branch as a new SVN tag (realtime_DWPT_latest or like that) and if somebody wants to start again with more realtime work, we should branch again (according to the SVN book: a reintegrated branch cannot be used for development anymore and should be removed). Land DWPT on trunk -- Key: LUCENE-3023 URL: https://issues.apache.org/jira/browse/LUCENE-3023 Project: Lucene - Java Issue Type: Task Affects Versions: CSF branch, 4.0 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.0 Attachments: LUCENE-3023-svn-diff.patch, LUCENE-3023-ws-changes.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_iw_iwc_jdoc.patch, LUCENE-3023_simonw_review.patch, LUCENE-3023_svndiff.patch, LUCENE-3023_svndiff.patch, diffMccand.py, diffSources.patch, diffSources.patch, realtime-TestAddIndexes-3.txt, realtime-TestAddIndexes-5.txt, realtime-TestIndexWriterExceptions-assert-6.txt, realtime-TestIndexWriterExceptions-npe-1.txt, realtime-TestIndexWriterExceptions-npe-2.txt, realtime-TestIndexWriterExceptions-npe-4.txt, realtime-TestOmitTf-corrupt-0.txt With LUCENE-2956 we have resolved the last remaining issue for LUCENE-2324 so we can proceed landing the DWPT development on trunk soon. I think one of the bigger issues here is to make sure that all JavaDocs for IW etc. are still correct though. I will start going through that first. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: svn commit: r1098242 - in /lucene/java/site: docs/ docs/skin/ src/documentation/content/xdocs/
BTW: The broken hatcher3 picture is not my fault, it's also broken on Manning's page! - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: uschind...@apache.org [mailto:uschind...@apache.org] Sent: Sunday, May 01, 2011 12:03 PM To: java-comm...@lucene.apache.org Subject: svn commit: r1098242 - in /lucene/java/site: docs/ docs/skin/ src/documentation/content/xdocs/ Author: uschindler Date: Sun May 1 10:03:16 2011 New Revision: 1098242 URL: http://svn.apache.org/viewvc?rev=1098242view=rev Log: Fix nightly build docs. Modified: lucene/java/site/docs/.htaccess lucene/java/site/docs/developer-resources.html lucene/java/site/docs/developer-resources.pdf lucene/java/site/docs/features.pdf lucene/java/site/docs/index.pdf lucene/java/site/docs/irc.pdf lucene/java/site/docs/linkmap.pdf lucene/java/site/docs/mailinglists.pdf lucene/java/site/docs/releases.pdf lucene/java/site/docs/skin/basic.css lucene/java/site/docs/skin/print.css lucene/java/site/docs/skin/profile.css lucene/java/site/docs/skin/screen.css lucene/java/site/docs/systemrequirements.pdf lucene/java/site/docs/whoweare.pdf lucene/java/site/src/documentation/content/xdocs/developer- resources.xml Modified: lucene/java/site/docs/.htaccess URL: http://svn.apache.org/viewvc/lucene/java/site/docs/.htaccess?rev=1098242 r1=1098241r2=1098242view=diff == --- lucene/java/site/docs/.htaccess (original) +++ lucene/java/site/docs/.htaccess Sun May 1 10:03:16 2011 @@ -2,5 +2,5 @@ #ignoring the meta http-equiv charset tags AddDefaultCharset off -Redirect Permanent /java/docs/api http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/all -Redirect Permanent /java/docs/nightly/api http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/all +Redirect Permanent /java/docs/api +https://builds.apache.org/hudson/job/Lucene-trunk/javadoc +Redirect Permanent /java/docs/nightly/api +https://builds.apache.org/hudson/job/Lucene-trunk/javadoc Modified: lucene/java/site/docs/developer-resources.html URL: http://svn.apache.org/viewvc/lucene/java/site/docs/developer- resources.html?rev=1098242r1=1098241r2=1098242view=diff == --- lucene/java/site/docs/developer-resources.html (original) +++ lucene/java/site/docs/developer-resources.html Sun May 1 10:03:16 +++ 2011 @@ -272,59 +272,106 @@ document.write(Last Published: + docu a name=N10011/aa name=source/a h2 class=boxedSource Code/h2 div class=section -pThe source files are now stored using Subversion (see http://subversion.tigris.org/ and http://svnbook.red-bean.com/) - /p -p - -span class=codefragsvn checkout http://svn.apache.org/repos/asf/lucene/dev/trunk lucene/dev/trunk/span +pThe source files are now stored using Subversion (see +http://subversion.tigris.org/ and http://svnbook.red-bean.com/)/p +ul + +liLatest btrunk/b: + span class=codefragsvn checkout +http://svn.apache.org/repos/asf/lucene/dev/trunk lucene_trunk/span + +/li -/p +liLatest bstable 3.x branch/b: + span class=codefragsvn checkout +http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x +lucene_3x/span + +/li + +/ul /div -a name=N10021/aa name=Changes/a +a name=N1002F/aa name=Changes/a h2 class=boxedChanges List/h2 div class=section -pCurrent changes list (as of the most recent nightly build) is available -a href=http://hudson.zones.apache.org/hudson/view/Lucene/job/Lucene- trunk/lastSuccessfulBuild/artifact/lucene/build/docs/changes/Changes.html -here/a./p +ul + +liCurrent changes list em(as of the most recent nightly btrunk/b build)/em is available +a href=https://builds.apache.org/hudson/job/Lucene- trunk/lastSuccessfulBuild/artifact/artifacts/changes/Changes.html +here/a./li + +liCurrent changes list em(as of the most recent nightly bstable 3.x branch/b build)/em is available +a href=https://builds.apache.org/hudson/job/Lucene- 3.x/lastSuccessfulBuild/artifact/artifacts/changes/Changes.html +here/a./li + +/ul /div -a name=N1002E/aa name=Making a Contribution/a +a name=N10052/aa name=Making a Contribution/a h2 class=boxedMaking a Contribution/h2 div class=section - pLucene Java is always looking for contributions, especially in the areas of documentation. See the a href=http://wiki.apache.org/lucene- javaWiki/a for details on contributing. /p +pLucene Java is always looking for contributions, especially in the +areas of documentation. See the a +href=http://wiki.apache.org/lucene-java;Wiki/a for details on +contributing./p /div -a name=N1003B/aa name=Nightly/a +a name=N1005F/aa name=Nightly/a h2 class=boxedNightly
Re: I was accepted in GSoC!!!
Welcome Vinicius! This is a badly needed feature and I'm glad you're working on it :) Mike http://blog.mikemccandless.com On Sat, Apr 30, 2011 at 9:18 PM, Vinicius Barros viniciusbarros.g...@yahoo.com.br wrote: Hi, That's great, I am waiting next instructions from google, it seems there is some paperwork to do. Regards, Vinicius Barros --- Em seg, 25/4/11, no-re...@socghop.appspotmail.com no-re...@socghop.appspotmail.com escreveu: De: no-re...@socghop.appspotmail.com no-re...@socghop.appspotmail.com Assunto: Congratulations! Para: viniciusbarros.g...@yahoo.com.br Data: Segunda-feira, 25 de Abril de 2011, 15:48 Dear Vinicius, Congratulations! Your proposal LUCENE-1768: NumericRange support for new query parser as submitted to Apache Software Foundation has been accepted for Google Summer of Code 2011. Over the next few days, we will add you to the private Google Summer of Code Student Discussion List. Over the next few weeks, we will send instructions to this list regarding turn in proof of enrollment, tax forms, etc. Now that you've been accepted, please take the opportunity to speak with your mentors about plans for the Community Bonding Period: what documentation should you be reading, what version control system will you need to set up, etc., before start of coding begins on May 23rd. Welcome to Google Summer of Code 2011! We look forward to having you with us. With best regards, The Google Summer of Code Program Administration Team - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027448#comment-13027448 ] Michael McCandless commented on LUCENE-3057: Patch looks great Simon; thanks for catching this selckin! Maybe you can rip out the random number generator in your machine and donate it to Apache!!! LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: I was accepted in GSoC!!!
Welcome Vinicius, I am glad to hear that you (my mentee) are one of the 5 students that are working for Apache Lucene/Solr this year. Until the coding officially starts, we should also sort out the infrastructure things like where to put the code and make a plan how to start. We should keep in close contact. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de http://www.thetaphi.de/ eMail: u...@thetaphi.de From: Vinicius Barros [mailto:viniciusbarros.g...@yahoo.com.br] Sent: Sunday, May 01, 2011 3:18 AM To: dev@lucene.apache.org; uschind...@apache.org Subject: I was accepted in GSoC!!! Hi, That's great, I am waiting next instructions from google, it seems there is some paperwork to do. Regards, Vinicius Barros --- Em seg, 25/4/11, no-re...@socghop.appspotmail.com no-re...@socghop.appspotmail.com escreveu: De: no-re...@socghop.appspotmail.com no-re...@socghop.appspotmail.com Assunto: Congratulations! Para: viniciusbarros.g...@yahoo.com.br Data: Segunda-feira, 25 de Abril de 2011, 15:48 Dear Vinicius, Congratulations! Your proposal LUCENE-1768: NumericRange support for new query parser as submitted to Apache Software Foundation has been accepted for Google Summer of Code 2011. Over the next few days, we will add you to the private Google Summer of Code Student Discussion List. Over the next few weeks, we will send instructions to this list regarding turn in proof of enrollment, tax forms, etc. Now that you've been accepted, please take the opportunity to speak with your mentors about plans for the Community Bonding Period: what documentation should you be reading, what version control system will you need to set up, etc., before start of coding begins on May 23rd. Welcome to Google Summer of Code 2011! We look forward to having you with us. With best regards, The Google Summer of Code Program Administration Team
Re: [HUDSON] Lucene-trunk - Build # 1547 - Failure
OOME Mike http://blog.mikemccandless.com On Sat, Apr 30, 2011 at 10:43 PM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-trunk/1547/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestNRTThreads.testNRTThreads Error Message: Some threads threw uncaught exceptions! Stack Trace: junit.framework.AssertionFailedError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1246) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1175) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:535) Build Log (for compile errors): [...truncated 11917 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [HUDSON] Lucene-trunk - Build # 1547 - Failure
The hprof is there: https://builds.apache.org/hudson/job/Lucene-trunk/ws/checkout/lucene/build/ I am working on a completely separate dir for the hprofs (where I also gzip them), so Jenkins can pick them up as artifacts and archive them. Using this technique its easier for us to pick up the ones from half hourly builds. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Michael McCandless [mailto:luc...@mikemccandless.com] Sent: Sunday, May 01, 2011 12:46 PM To: dev@lucene.apache.org Subject: Re: [HUDSON] Lucene-trunk - Build # 1547 - Failure OOME Mike http://blog.mikemccandless.com On Sat, Apr 30, 2011 at 10:43 PM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-trunk/1547/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestNRTThreads.testNRTThreads Error Message: Some threads threw uncaught exceptions! Stack Trace: junit.framework.AssertionFailedError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(Lu ceneTestCase.java:1246) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(Lu ceneTestCase.java:1175) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:535 ) Build Log (for compile errors): [...truncated 11917 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [HUDSON] Lucene-trunk - Build # 1547 - Failure
I implemented the separate archiving of the heap dumps in all Hudson jobs (except maven, as I have no idea how they work and how to enable heap dumps there). You should from now on find them in the project's artifacts (if they exist). - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, May 01, 2011 12:56 PM To: dev@lucene.apache.org Subject: RE: [HUDSON] Lucene-trunk - Build # 1547 - Failure The hprof is there: https://builds.apache.org/hudson/job/Lucene- trunk/ws/checkout/lucene/build/ I am working on a completely separate dir for the hprofs (where I also gzip them), so Jenkins can pick them up as artifacts and archive them. Using this technique its easier for us to pick up the ones from half hourly builds. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Michael McCandless [mailto:luc...@mikemccandless.com] Sent: Sunday, May 01, 2011 12:46 PM To: dev@lucene.apache.org Subject: Re: [HUDSON] Lucene-trunk - Build # 1547 - Failure OOME Mike http://blog.mikemccandless.com On Sat, Apr 30, 2011 at 10:43 PM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-trunk/1547/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestNRTThreads.testNRTThreads Error Message: Some threads threw uncaught exceptions! Stack Trace: junit.framework.AssertionFailedError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(Lu ceneTestCase.java:1246) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(Lu ceneTestCase.java:1175) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:535 ) Build Log (for compile errors): [...truncated 11917 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Solr Config XML DTD's
If not a DTD, can we put some more customized form of validation for Solr's configuration? In general, I think servers should be anal on startup, refusing to start if there's anything off in their configuration. (Of course, along with this, the error messaging has to be *excellent* so you know precisely where the problem is, what's wrong, how to fix it). If you take the lenient/forgiving approach then you wind up with Solr instances in unknown states -- the app developer thinks they turned X on, everything starts fine, but then, silently, inexplicably, it's not working. This then leads to frustration, thinking Solr is buggy, not using this feature, blogging about problems, etc. Mike http://blog.mikemccandless.com On Tue, Mar 29, 2011 at 7:15 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : Hi, this is my first post to the mailing list. I'm working on a commercial Welcome! : My DTD works for our internal version of queryElevation.xml, but since the : ATTRIB name of the doc/ tag could be anything, I'm not sure how to write a : DTD that would validate any valid query elevation file. right .. this is one of the reasons why we've never tried to publish a DTD for the solrconfig.xml or schema.xml files either. there are lots of cases where plugins can define arbitrary attributes on the XML nodes. If i had the chance to do it all over again, and i better understood xml back when yonik first showed me what the configs would look like, i would have suggested using xml namespaces .. but that ship kind of sailed a while ago. we're getting a little better -- moving towards using the same type of NamedList backed XML for the initialization anytime new plugins are added, but i don't see it being feasible to have a config DTD anytime soon. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[HUDSON] Lucene-Solr-tests-only-3.x - Build # 7607 - Failure
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7607/ All tests passed Build Log (for compile errors): [...truncated 12585 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Link to nightly build test reports on main Lucene site needs updating
Thanks for fixing uwe! On Sun, May 1, 2011 at 12:04 PM, Uwe Schindler u...@thetaphi.de wrote: I fixed the nightly docs, once the webserver mirrors them from SVN they should appear. The developer-resources page was completely broken. It now also contains references to the stable 3.x branch as most users would prefer that one to fix latest bugs but don’t want to have a backwards-incompatible version. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Simon Willnauer [mailto:simon.willna...@googlemail.com] Sent: Saturday, April 30, 2011 9:42 PM To: java-u...@lucene.apache.org Cc: dev@lucene.apache.org Subject: Re: Link to nightly build test reports on main Lucene site needs updating thanks tom, I cced dev@l.a.o simon On Fri, Apr 29, 2011 at 11:14 PM, Burton-West, Tom tburt...@umich.edu wrote: Hello, I went to look at the Hudson nightly builds and tried to follow the link from the main Lucene page http://lucene.apache.org/java/docs/developer-resources.html#Nightly The links to the Clover Test Coverage Reports point to http://hudson.zones.apache.org/hudson/view/Lucene/job/Lucene- trunk/lastSuccessfulBuild/clover/ but apparently hudson.zones.apache.org is no longer being used. I think the link should point to somewhere on https://builds.apache.org/hudson/job/Lucene-trunk/. Is this the right list to alert whoever maintains the main Lucene pages on lucene.apache.org? Tom - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3059) PulsingTermState.clone leaks memory
PulsingTermState.clone leaks memory --- Key: LUCENE-3059 URL: https://issues.apache.org/jira/browse/LUCENE-3059 Project: Lucene - Java Issue Type: Bug Affects Versions: 4.0 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.0 Attachments: LUCENE-3059.patch I looked at the heap dump from the OOME this morning (thank you Uwe for turning this on!), and I think it's a real memory leak. Well, not really a leak; rather, the cloned PulsingTermState, which we cache in the terms dict cache, is hanging onto large byte[] unnecessarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3059) PulsingTermState.clone leaks memory
[ https://issues.apache.org/jira/browse/LUCENE-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-3059: --- Attachment: LUCENE-3059.patch Patch. I fixed PulsingTermState's clone to make a new PulsingTerm state and then copy (like we do for StandardCodec). I did the same for SepCodec, though I don't think that one is leaking memory. PulsingTermState.clone leaks memory --- Key: LUCENE-3059 URL: https://issues.apache.org/jira/browse/LUCENE-3059 Project: Lucene - Java Issue Type: Bug Affects Versions: 4.0 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.0 Attachments: LUCENE-3059.patch I looked at the heap dump from the OOME this morning (thank you Uwe for turning this on!), and I think it's a real memory leak. Well, not really a leak; rather, the cloned PulsingTermState, which we cache in the terms dict cache, is hanging onto large byte[] unnecessarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3053) improve test coverage for Multi*
[ https://issues.apache.org/jira/browse/LUCENE-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-3053. - Resolution: Fixed Fix Version/s: 3.2 Committed revision 1098303, 1098329 (branch_3x) improve test coverage for Multi* Key: LUCENE-3053 URL: https://issues.apache.org/jira/browse/LUCENE-3053 Project: Lucene - Java Issue Type: Task Reporter: Robert Muir Assignee: Robert Muir Fix For: 3.2, 4.0 Attachments: LUCENE-3053.patch, LUCENE-3053.patch, LUCENE-3053.patch It seems like an easy win that when the test calls newSearcher(), it should sometimes wrap the reader with a SlowMultiReaderWrapper. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: jira issues falling off the radar -- Next JIRA version
I don't think Next is necessarily dangerous; if used then 3.2, shouldn't exist until it's time to release. Having both is dangerous. But sure, I'm in favor of removing Next. A committer would have to adjudicate all issues and substitute either 3.2 or 4.0. Speaking of which, I think simply assigning a fix-version for 3.2 implies that it will be for any future release (including 4.0) and so I don't see there's a point in assigning both of these fix-versions. There are rare exceptions to this and I am aware of course that our dual-development branches implies having to make two commits. RE v1.5, fortunately, most of the issues are already assigned a suitable fix-version other than 1.5 so those can be ignored. Once the few remainder are addressed, v1.5 can be deleted. RE Next, Any issue with a resolution status of Fixed should be re-assigned a suitable fix-version if it doesn't have any. There are very few of these two worry about. Those without a resolution status should probably just be assigned to 3.2 -- so yes I agree that's what we should do. Then Next can be deleted. ~ David Smiley Michael McCandless-2 wrote: On Fri, Apr 29, 2011 at 12:12 AM, David Smiley (@MITRE.org) lt;dsmi...@mitre.orggt; wrote: (Comments on SOLR-2191 between Mark I were starting to get off-topic with respect to the issue so I am continuing the conversation here) A lot of JIRA issues seem to fall off the radar, IMO. I'm talking about issues that have patches and are basically ready to go. There are multiple ways to address this but at the moment I am going to just bring up one. Looking at the versions in JIRA one can assign an issue to https://issues.apache.org/jira/browse/SOLR#selectedTab=com.atlassian.jira.plugin.system.project%3Aversions-panel I see the version named Next, with this description: Placeholder for commiters to track issues that are not ready to commit, but seem close enough to being ready to warrant focus before the next feature release. This version and what it implies is a common pattern in use of JIRA that I too use for projects I manage for my employer. It appears that for the 3.1 release, nobody looked through the issues assigned to Next, and consequently, some issues like SOLR-2191 were forgotten despite being ready to go. Looking through the wiki I see information on how to do a release http://wiki.apache.org/solr/HowToRelease and release suggestions but no information on what to do in advance of a release. I also don't see any administrative tasks on managing the Next version in JIRA. So I think either the Next version should be used effectively, or if that isn't going to happen then delete this version. I agree Next is dangerous! It'd be nice if Jira could auto-magically treat Next as whatever release really is next. EG, say we all agree 3.2 is our next release, then ideally Jira would treat all Next issues as if they were marked with 3.2. But... lacking that, maybe we really shouldn't use Next at all, and just use 3.2? Having to step through these issues and move them to the next release on releasing is also healthy, ie, it's good that we see/review them, think about why we didn't get it done on the current release, etc. On a related note, I don't know what to make of the 1.5 version, nor what to make of issues marked as Closed for Next. Some house cleaning is in order. We should clean these up. Should we just roll them over to 3.2? Mike http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book-- View this message in context: http://lucene.472066.n3.nabble.com/Re-jira-issues-falling-off-the-radar-Next-JIRA-version-tp2886427p2886427.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 7613 - Failure
false fail caused by LUCENE-3053, I'll fix. On Sun, May 1, 2011 at 11:02 AM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7613/ 1 tests failed. REGRESSION: org.apache.lucene.search.TestDisjunctionMaxQuery.testSkipToFirsttimeHit Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.lucene.search.TestDisjunctionMaxQuery.testSkipToFirsttimeHit(TestDisjunctionMaxQuery.java:188) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 5279 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Setting the max number of merge threads across IndexWriters
Almost any design that keeps circular references between components is broken. Inability to share MergeSchedulers is just another testimonial to that. 2011/4/16 Shai Erera ser...@gmail.com: Hi This was raised in LUCENE-2755 (along with other useful refactoring to MS-IW-MP interaction). Here is the relevant comment which addresses Jason's particular issue: https://issues.apache.org/jira/browse/LUCENE-2755?focusedCommentId=12966029page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12966029 In short, we can refactor CMS to not hold to an IndexWriter member if we change a lot of the API. But IMO, an ExecutorServiceMS is the right way to go, if you don't mind giving up some CMS features, like controlling thread priority and stalling running threads. In fact, even w/ ExecutorServiceMS you can still achieve some (e.g., stalling), but some juggling will be required. Then, instead of trying to factor out IW members from this MS, you could share the same ES with all MS instances, each will keep a reference to a different IW member. This is just a thought though, I haven't tried it. Shai On Thu, Apr 14, 2011 at 8:23 PM, Earwin Burrfoot ear...@gmail.com wrote: Can't remember. Probably no. I started an experimental MS api rewrite (incorporating ability to share MSs between IWs) some time ago, but never had the time to finish it. On Thu, Apr 14, 2011 at 19:56, Simon Willnauer simon.willna...@googlemail.com wrote: On Thu, Apr 14, 2011 at 5:52 PM, Earwin Burrfoot ear...@gmail.com wrote: I proposed to decouple MergeScheduler from IW (stop keeping a reference to it). Then you can create a single CMS and pass it to all your IWs. Yep that was it... is there an issue for this? simon On Thu, Apr 14, 2011 at 19:40, Jason Rutherglen jason.rutherg...@gmail.com wrote: I think the proposal involved using a ThreadPoolExecutor, which seemed to not quite work as well as what we have. I think it'll be easier to simply pass a global context that keeps a counter of the actively running threads, and pass that into each IW's CMS? On Thu, Apr 14, 2011 at 8:25 AM, Simon Willnauer simon.willna...@googlemail.com wrote: On Thu, Apr 14, 2011 at 5:20 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Today the ConcurrentMergeScheduler allows setting the max thread count and is bound to a single IndexWriter. However in the [common] case of multiple IndexWriters running in the same process, this disallows one from managing the aggregate number of merge threads executing at any given time. I think this can be fixed, shall I open an issue? go ahead! I think I have seen this suggestion somewhere maybe you need to see if there is one already simon - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Setting the max number of merge threads across IndexWriters
You don't mean 'static var' under 'global'? I hope, very much. 2011/4/16 Jason Rutherglen jason.rutherg...@gmail.com: I'd rather not lose [important] functionality. I think a global max thread count is the least intrusive way to go, however I also need to see if that's possible. If so I'll open an issue and post a patch. 2011/4/15 Shai Erera ser...@gmail.com: Hi This was raised in LUCENE-2755 (along with other useful refactoring to MS-IW-MP interaction). Here is the relevant comment which addresses Jason's particular issue: https://issues.apache.org/jira/browse/LUCENE-2755?focusedCommentId=12966029page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12966029 In short, we can refactor CMS to not hold to an IndexWriter member if we change a lot of the API. But IMO, an ExecutorServiceMS is the right way to go, if you don't mind giving up some CMS features, like controlling thread priority and stalling running threads. In fact, even w/ ExecutorServiceMS you can still achieve some (e.g., stalling), but some juggling will be required. Then, instead of trying to factor out IW members from this MS, you could share the same ES with all MS instances, each will keep a reference to a different IW member. This is just a thought though, I haven't tried it. Shai On Thu, Apr 14, 2011 at 8:23 PM, Earwin Burrfoot ear...@gmail.com wrote: Can't remember. Probably no. I started an experimental MS api rewrite (incorporating ability to share MSs between IWs) some time ago, but never had the time to finish it. On Thu, Apr 14, 2011 at 19:56, Simon Willnauer simon.willna...@googlemail.com wrote: On Thu, Apr 14, 2011 at 5:52 PM, Earwin Burrfoot ear...@gmail.com wrote: I proposed to decouple MergeScheduler from IW (stop keeping a reference to it). Then you can create a single CMS and pass it to all your IWs. Yep that was it... is there an issue for this? simon On Thu, Apr 14, 2011 at 19:40, Jason Rutherglen jason.rutherg...@gmail.com wrote: I think the proposal involved using a ThreadPoolExecutor, which seemed to not quite work as well as what we have. I think it'll be easier to simply pass a global context that keeps a counter of the actively running threads, and pass that into each IW's CMS? On Thu, Apr 14, 2011 at 8:25 AM, Simon Willnauer simon.willna...@googlemail.com wrote: On Thu, Apr 14, 2011 at 5:20 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Today the ConcurrentMergeScheduler allows setting the max thread count and is bound to a single IndexWriter. However in the [common] case of multiple IndexWriters running in the same process, this disallows one from managing the aggregate number of merge threads executing at any given time. I think this can be fixed, shall I open an issue? go ahead! I think I have seen this suggestion somewhere maybe you need to see if there is one already simon - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Kirill Zakharenko/Кирилл Захаренко E-Mail/Jabber: ear...@gmail.com Phone: +7 (495) 683-567-4 ICQ: 104465785 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 7614 - Still Failing
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7614/ 1 tests failed. FAILED: org.apache.lucene.store.instantiated.TestRealTime.test Error Message: null Stack Trace: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.lucene.store.instantiated.InstantiatedIndexReader.norms(InstantiatedIndexReader.java:332) at org.apache.lucene.index.MultiReader.norms(MultiReader.java:308) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:106) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:517) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:332) at org.apache.lucene.store.instantiated.TestRealTime.test(TestRealTime.java:51) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 7743 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 7614 - Still Failing
This is likely also caused by LUCENE-3053... will take a look now On Sun, May 1, 2011 at 11:22 AM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7614/ 1 tests failed. FAILED: org.apache.lucene.store.instantiated.TestRealTime.test Error Message: null Stack Trace: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.lucene.store.instantiated.InstantiatedIndexReader.norms(InstantiatedIndexReader.java:332) at org.apache.lucene.index.MultiReader.norms(MultiReader.java:308) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:106) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:517) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:332) at org.apache.lucene.store.instantiated.TestRealTime.test(TestRealTime.java:51) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 7743 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 7615 - Still Failing
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7615/ 1 tests failed. FAILED: org.apache.lucene.store.instantiated.TestRealTime.test Error Message: null Stack Trace: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.lucene.store.instantiated.InstantiatedIndexReader.norms(InstantiatedIndexReader.java:332) at org.apache.lucene.index.MultiReader.norms(MultiReader.java:308) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:106) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:517) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:332) at org.apache.lucene.store.instantiated.TestRealTime.test(TestRealTime.java:51) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 7742 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 7615 - Still Failing
This is fixed in r1098359. this realtime capability of instantiated is incompatible with MultiReader, because when MR initializes it computes and caches the maxDoc, which becomes out of date with respect to the InstantiatedIndexReader if you use it in this way. The only reason the test didnt fail in trunk is because SlowMultiReaderWrapper does not do this caching, and this top-level norms is gone in trunk, MultiNorms just calls norms(field) and arraycopies itself. On Sun, May 1, 2011 at 11:47 AM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7615/ 1 tests failed. FAILED: org.apache.lucene.store.instantiated.TestRealTime.test Error Message: null Stack Trace: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.lucene.store.instantiated.InstantiatedIndexReader.norms(InstantiatedIndexReader.java:332) at org.apache.lucene.index.MultiReader.norms(MultiReader.java:308) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:106) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:517) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:332) at org.apache.lucene.store.instantiated.TestRealTime.test(TestRealTime.java:51) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 7742 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027469#comment-13027469 ] Simon Willnauer commented on LUCENE-3057: - bq. Maybe you can rip out the random number generator in your machine and donate it to Apache!!! +1 :) thanks for running tests that is of great value! LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer resolved LUCENE-3057. - Resolution: Fixed Committed revision 1098367. LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027470#comment-13027470 ] Robert Muir commented on LUCENE-3057: - should we backport to 3.x too?! LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027471#comment-13027471 ] Simon Willnauer commented on LUCENE-3057: - bq. should we backport to 3.x too?! oh right this stuff is in 3.x too - I will backport tomorrow! LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 7618 - Failure
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7618/ 1 tests failed. REGRESSION: org.apache.lucene.search.TestTermScorer.testNext Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.lucene.search.TestTermScorer.testNext(TestTermScorer.java:136) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 5267 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 7618 - Failure
fixed in revision 1098372. On Sun, May 1, 2011 at 1:13 PM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/7618/ 1 tests failed. REGRESSION: org.apache.lucene.search.TestTermScorer.testNext Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.lucene.search.TestTermScorer.testNext(TestTermScorer.java:136) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1039) Build Log (for compile errors): [...truncated 5267 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 7620 - Failure
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/7620/ 1 tests failed. REGRESSION: org.apache.lucene.store.TestLockFactory.testNativeFSLockFactoryPrefix Error Message: Lock prefix for lockDir outside of directory should be not null Stack Trace: junit.framework.AssertionFailedError: Lock prefix for lockDir outside of directory should be not null at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1261) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1190) at org.apache.lucene.store.TestLockFactory.testNativeFSLockFactoryPrefix(TestLockFactory.java:246) Build Log (for compile errors): [...truncated 3227 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 7620 - Failure
this one is somehow triggered by LUCENE-3057 On Sun, May 1, 2011 at 1:28 PM, Apache Jenkins Server hud...@hudson.apache.org wrote: Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/7620/ 1 tests failed. REGRESSION: org.apache.lucene.store.TestLockFactory.testNativeFSLockFactoryPrefix Error Message: Lock prefix for lockDir outside of directory should be not null Stack Trace: junit.framework.AssertionFailedError: Lock prefix for lockDir outside of directory should be not null at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1261) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1190) at org.apache.lucene.store.TestLockFactory.testNativeFSLockFactoryPrefix(TestLockFactory.java:246) Build Log (for compile errors): [...truncated 3227 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3057) LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception
[ https://issues.apache.org/jira/browse/LUCENE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-3057: Attachment: LUCENE-3057_bug.patch Hi Simon, I think you meant to set the lockfactory in the finally block? currently, if there is no exception instantiated the requested directory impl via the ctor (e.g. NIOFSDirectory), then the lockfactory parameter is never set. LuceneTestCase#newFSDirectoryImpl misses to set LockFactory if ctor call throws exception - Key: LUCENE-3057 URL: https://issues.apache.org/jira/browse/LUCENE-3057 Project: Lucene - Java Issue Type: Bug Components: Tests Affects Versions: 4.0 Reporter: Simon Willnauer Priority: Minor Fix For: 4.0 Attachments: LUCENE-3057.patch, LUCENE-3057_bug.patch selckin reported on IRC that if you run ant test -Dtestcase=TestLockFactory -Dtestmethod=testNativeFSLockFactoryPrefix -Dtests.directory=FSDirectory the test fails. Since FSDirectory is an abstract class it can not be instantiated so our code falls back to FSDirector.open. yet we miss to set the given lockFactory though. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 7621 - Still Failing
Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/7621/ 1 tests failed. FAILED: org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch Error Message: Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. - org.apache.solr.common.cloud.ZooKeeperException: at org.apache.solr.core.CoreContainer.register(CoreContainer.java:523) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:407) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:291) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:240) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:93) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1104) at org.mortbay.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1140) at org.mortbay.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:940) at org.mortbay.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:895) at org.mortbay.jetty.servlet.Context.addFilter(Context.java:207) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:98) at org.mortbay.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:140) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:52) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:123) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:118) at org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:245) at org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:236) at org.apache.solr.cloud.AbstractDistributedZkTestCase.createServers(AbstractDistributedZkTestCase.java:64) at org.apache.solr.BaseDistributedSearch Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. - org.apache.solr.common.cloud.ZooKeeperException: at org.apache.solr.core.CoreContainer.register(CoreContainer.java:523) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:407) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:291) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:240) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:93) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1104) at org.mortbay.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1140) at org.mortbay.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:940) at org.mortbay.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:895) at org.mortbay.jetty.servlet.Context.addFilter(Context.java:207) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:98) at org.mortbay.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:140) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:52) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:123) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:118) at org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:245) at org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:236) at org.apache.solr.cloud.AbstractDistributedZkTestCase.createServers(AbstractDistributedZkTestCase.java:64) at org.apache.solr.BaseDistributedSearch request: http://localhost:53042/solr/update?wt=javabinversion=2 Stack Trace: request: http://localhost:53042/solr/update?wt=javabinversion=2 at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105) at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:111) at
[jira] [Resolved] (LUCENE-3023) Land DWPT on trunk
[ https://issues.apache.org/jira/browse/LUCENE-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-3023. --- Resolution: Fixed Committed merged branch to trunk revision: 1098427 Moved branch away as tag in revision: 1098428 Land DWPT on trunk -- Key: LUCENE-3023 URL: https://issues.apache.org/jira/browse/LUCENE-3023 Project: Lucene - Java Issue Type: Task Affects Versions: CSF branch, 4.0 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.0 Attachments: LUCENE-3023-svn-diff.patch, LUCENE-3023-ws-changes.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_CHANGES.patch, LUCENE-3023_iw_iwc_jdoc.patch, LUCENE-3023_simonw_review.patch, LUCENE-3023_svndiff.patch, LUCENE-3023_svndiff.patch, diffMccand.py, diffSources.patch, diffSources.patch, realtime-TestAddIndexes-3.txt, realtime-TestAddIndexes-5.txt, realtime-TestIndexWriterExceptions-assert-6.txt, realtime-TestIndexWriterExceptions-npe-1.txt, realtime-TestIndexWriterExceptions-npe-2.txt, realtime-TestIndexWriterExceptions-npe-4.txt, realtime-TestOmitTf-corrupt-0.txt With LUCENE-2956 we have resolved the last remaining issue for LUCENE-2324 so we can proceed landing the DWPT development on trunk soon. I think one of the bigger issues here is to make sure that all JavaDocs for IW etc. are still correct though. I will start going through that first. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Re: Solr Config XML DTD's
My first post too - but if I can offer a suggestion - there are more
modern XML validation technologies available than DTD. I would heartily
recommend RelaxNG/Compact notation (see
http://relaxng.org/compact-tutorial-20030326.html) - you can generate
Relax from a DTD, but it is more expressive, while still being easy on
the eyes (uses curly-brace syntax), and much simpler than XML schema.
In particular it lets you express wildcard constraints like:
start = anyElement
anyElement =
element * {
(attribute * { text }
| text
| anyElement)*
}
which matches absolutely anything.
I'm not sure what kinds of constraints can actually be applied to solr's
configuration in practice?
But using a formal constraint language will give decent error reporting out of
the box.
Java-based tools for Relax validation and conversion are available here:
http://code.google.com/p/jing-trang/
-Mike S
On 2:59 PM, Michael McCandless wrote:
If not a DTD, can we put some more customized form of validation for
Solr's configuration?
In general, I think servers should be anal on startup, refusing to
start if there's anything off in their configuration.
(Of course, along with this, the error messaging has to be *excellent*
so you know precisely where the problem is, what's wrong, how to fix
it).
If you take the lenient/forgiving approach then you wind up with Solr
instances in unknown states -- the app developer thinks they turned X
on, everything starts fine, but then, silently, inexplicably, it's not
working. This then leads to frustration, thinking Solr is buggy, not
using this feature, blogging about problems, etc.
Mike
http://blog.mikemccandless.com
On Tue, Mar 29, 2011 at 7:15 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
: Hi, this is my first post to the mailing list. I'm working on a commercial
Welcome!
: My DTD works for our internal version of queryElevation.xml, but since the
: ATTRIB name of thedoc/ tag could be anything, I'm not sure how to write a
: DTD that would validate any valid query elevation file.
right .. this is one of the reasons why we've never tried to publish a DTD
for the solrconfig.xml or schema.xml files either. there are lots of
cases where plugins can define arbitrary attributes on the XML nodes.
If i had the chance to do it all over again, and i better understood xml
back when yonik first showed me what the configs would look like, i would
have suggested using xml namespaces .. but that ship kind of sailed a
while ago.
we're getting a little better -- moving towards using the same type of
NamedList backed XML for the initialization anytime new plugins are
added, but i don't see it being feasible to have a config DTD anytime
soon.
-Hoss
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-trunk - Build # 1548 - Still Failing
Build: https://builds.apache.org/hudson/job/Lucene-trunk/1548/ No tests ran. Build Log (for compile errors): [...truncated 9474 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password protected files
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027544#comment-13027544 ] Shinichiro Abe commented on SOLR-2480: -- There is a same issue. https://issues.apache.org/jira/browse/SOLR-445 If it be able to applied by that same policy, this issue is duplicate. Text extraction of password protected files --- Key: SOLR-2480 URL: https://issues.apache.org/jira/browse/SOLR-2480 Project: Solr Issue Type: Improvement Components: contrib - Solr Cell (Tika extraction) Affects Versions: 3.1 Reporter: Shinichiro Abe Priority: Minor Proposal: There are password-protected files. PDF, Office documents in 2007 format/97 format. These files are posted using SolrCell. We do not have to read these files if we do not know the reading password of files. So, these files may not be extracted text. My requirement is that these files should be processed normally without extracting text, and without throwing exception. This background: Now, when you post a password-protected file, solr returns 500 server error. Solr catches the error in ExtractingDocumentLoader and throws TikException. I use ManifoldCF. If the solr server responds 500, ManifoldCF judge is that this document should be retried because I have absolutely no idea what happened. And it attempts to retry posting many times without getting the password. In the other case, my customer posts the files with embedded images. Sometimes it seems that solr throws TikaException of unknown cause. He wants to post just metadata without extracting text, but makes him stop posting by the exception. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-445) Update Handlers abort with bad documents
[ https://issues.apache.org/jira/browse/SOLR-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027545#comment-13027545 ] Shinichiro Abe commented on SOLR-445: - In Solr Cell, There is the same problem. It aborts mid during posting the protected files(SOLR-2480). I hope that update handlers should be fixed by applying that model. Update Handlers abort with bad documents Key: SOLR-445 URL: https://issues.apache.org/jira/browse/SOLR-445 Project: Solr Issue Type: Bug Components: update Affects Versions: 1.3 Reporter: Will Johnson Assignee: Grant Ingersoll Fix For: Next Attachments: SOLR-445-3_x.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445_3x.patch, solr-445.xml Has anyone run into the problem of handling bad documents / failures mid batch. Ie: add doc field name=id1/field /doc doc field name=id2/field field name=myDateFieldI_AM_A_BAD_DATE/field /doc doc field name=id3/field /doc /add Right now solr adds the first doc and then aborts. It would seem like it should either fail the entire batch or log a message/return a code and then continue on to add doc 3. Option 1 would seem to be much harder to accomplish and possibly require more memory while Option 2 would require more information to come back from the API. I'm about to dig into this but I thought I'd ask to see if anyone had any suggestions, thoughts or comments. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password protected files
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027547#comment-13027547 ] Koji Sekiguchi commented on SOLR-2480: -- Though I've not yet read entire comment SOLR-445, I don't think your requirement is same. According to description of SOLR-445, the reporter wants Solr to skip the error doc/ and continue adding the rest of doc/ in add.../add. But I think you want Solr to skip the content *field* because tika cannot extract it for some reasons but add meta data fields, right? Text extraction of password protected files --- Key: SOLR-2480 URL: https://issues.apache.org/jira/browse/SOLR-2480 Project: Solr Issue Type: Improvement Components: contrib - Solr Cell (Tika extraction) Affects Versions: 3.1 Reporter: Shinichiro Abe Priority: Minor Proposal: There are password-protected files. PDF, Office documents in 2007 format/97 format. These files are posted using SolrCell. We do not have to read these files if we do not know the reading password of files. So, these files may not be extracted text. My requirement is that these files should be processed normally without extracting text, and without throwing exception. This background: Now, when you post a password-protected file, solr returns 500 server error. Solr catches the error in ExtractingDocumentLoader and throws TikException. I use ManifoldCF. If the solr server responds 500, ManifoldCF judge is that this document should be retried because I have absolutely no idea what happened. And it attempts to retry posting many times without getting the password. In the other case, my customer posts the files with embedded images. Sometimes it seems that solr throws TikaException of unknown cause. He wants to post just metadata without extracting text, but makes him stop posting by the exception. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password protected files
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027553#comment-13027553 ] Koji Sekiguchi commented on SOLR-2480: -- BTW, I have a similar issue when using UIMA update processor, as sometimes UIMA annotators fail to extract meta data for some reason (eg Alchemy Web services stop). I'll open a separate ticket for it. Text extraction of password protected files --- Key: SOLR-2480 URL: https://issues.apache.org/jira/browse/SOLR-2480 Project: Solr Issue Type: Improvement Components: contrib - Solr Cell (Tika extraction) Affects Versions: 3.1 Reporter: Shinichiro Abe Priority: Minor Proposal: There are password-protected files. PDF, Office documents in 2007 format/97 format. These files are posted using SolrCell. We do not have to read these files if we do not know the reading password of files. So, these files may not be extracted text. My requirement is that these files should be processed normally without extracting text, and without throwing exception. This background: Now, when you post a password-protected file, solr returns 500 server error. Solr catches the error in ExtractingDocumentLoader and throws TikException. I use ManifoldCF. If the solr server responds 500, ManifoldCF judge is that this document should be retried because I have absolutely no idea what happened. And it attempts to retry posting many times without getting the password. In the other case, my customer posts the files with embedded images. Sometimes it seems that solr throws TikaException of unknown cause. He wants to post just metadata without extracting text, but makes him stop posting by the exception. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
