[jira] Created: (SOLR-139) Support updateable/modifiable documents
Support updateable/modifiable documents --- Key: SOLR-139 URL: https://issues.apache.org/jira/browse/SOLR-139 Project: Solr Issue Type: Improvement Components: update Reporter: Ryan McKinley It would be nice to be able to update some fields on a document without having to insert the entire document. Given the way lucene is structured, (for now) one can only modify stored fields. While we are at it, we can support incrementing an existing value - I think this only makes sense for numbers. for background, see: http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-140) single quote in search query will break any sorts
single quote in search query will break any sorts - Key: SOLR-140 URL: https://issues.apache.org/jira/browse/SOLR-140 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.1.0 Environment: SOLR nightly build 2007-02-01, but old builds as well. Ubuntu 6.06 LTS server. Tomcat 5.5.20 Reporter: John Kleven Query strings that contain a single quote, properly escaped, break *as soon as you add a sort field*. For example, If I want to search for the word jam's including the single quote, I can. But as soon as I add a sort criteria to the URL, it breaks. Note: %27 is a single quote encoded for URI's Here's the examples http://10.0.101.11:8080/forsale/select/?q=jam%27s-- this works (i get ~20 hits back) http://10.0.101.11:8080/forsale/select/?q=jam%27s;score%20desc --- this doesn't (get 0 hits back) Seems that if the first query works, it shouldn't be breakable just by telling it explicitly to sort on score?? All other sorts break in this scenario as well ( i.e., price, etc). email rsp from Yonik: Hmmm, Solr tries to be smart about splitting the query from the sort specification. If it sees a semicolon inside quotes (single or double), or backslash escaped, then it will treat it as part of the query. An immediate workaround would be to query for "jam's" or jam\'s It would be helpful if you could open a JIRA issue for this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-139: --- Attachment: SOLR-139-IndexDocumentCommand.patch > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[
https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470035
]
Ryan McKinley commented on SOLR-139:
SOLR-139-IndexDocumentCommand.patch adds a new command to UpdateHandler and
deprecates 'AddUpdateCommand'
This patch is only concerned with adding updateability to the UpdateHandler, it
does not deal with how request handlers specify what should happen with each
field.
I added:
public class IndexDocumentCommand
{
public enum MODE {
OVERWRITE, // overwrite existing values with the new one. (the default
behavior)
APPEND,// add the new value to existing value
DISTINCT, // same as APPEND, but make sure each value is distinct
INCREMENT, // increment existing value. Must be a number!
REMOVE // remove the previous value.
};
public boolean overwrite = true;
public SolrDocument doc;
public Map mode; // What to do for each field. null is the
default
public int commitMaxTime = -1; // make sure the document is commited within
this much time
}
RequestHandlers will need to fill up the 'mode' map if they want to support
updateability. Setting the mode.put( null, APPEND ) sets the default mode.
> Support updateable/modifiable documents
> ---
>
> Key: SOLR-139
> URL: https://issues.apache.org/jira/browse/SOLR-139
> Project: Solr
> Issue Type: Improvement
> Components: update
>Reporter: Ryan McKinley
> Attachments: SOLR-139-IndexDocumentCommand.patch
>
>
> It would be nice to be able to update some fields on a document without
> having to insert the entire document.
> Given the way lucene is structured, (for now) one can only modify stored
> fields.
> While we are at it, we can support incrementing an existing value - I think
> this only makes sense for numbers.
> for background, see:
> http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: Connecting custom RequestHandler
On Sat, 2007-02-03 at 18:41 -0800, Ryan McKinley wrote: > The 'new' request parser reads the content type header to see if it > should parse the body as params or not. If the content type is > "application/x-www-form-urlencoded" it parses them as params, if it is > "multipart/*" it parses them as multipart. if it is *anything* else, > it reads the body as a stream. > > curl defaults the content type to "application/x-www-form-urlencoded" > > so you will need to run curl with -H "Content-Type: text/xml > > - - - - - - - Hmm, I patched the post.sh only regarding the URL. FILES=$* URL=http://localhost:8983/solr/update/stax for f in $FILES; do echo Posting file $f to $URL curl $URL --data-binary @$f -H 'Content-type:text/xml; charset=utf-8' echo done Meaning what you describe is already in there. I now played around again with it and surprise it works now. Thanks Ryan. > > I should add this to the wiki. > > Can we change the post.sh to use -H "Content-Type: text/xml"? This > will not affect old updater and will work for new UpdateHandlers. It is see above. Not sure why it did not work the yesterday. Thanks for the feedback. salu2 > ryan > > > On 2/3/07, Thorsten Scherler <[EMAIL PROTECTED]> wrote: > > On Sat, 2007-02-03 at 18:14 +0100, Thorsten Scherler wrote: > > > Hi all, > > > > > > I am working on SOLR-133 and I have wrapped up a first version of the > > > XmlUpdateRequestHandlerStax.java. Now I am trying to connect it in the > > > example but I have some problems. > > > > > > I am trying: > > > > > class="solr.XmlUpdateRequestHandlerStax" /> > > > > > > Trying to curl to URL=http://localhost:8983/solr/update/stax > > > > Debugging this I used http://localhost:8983/solr/update/xml and I get > > the same error. > > > > So I figured that we still use the SolrUpdateServlet @Deprecated in the > > post.sh. > > > > For now I can test the StAX changing > > XmlUpdateRequestHandlerStax legacyUpdateHandler; > > and implementing a doLegacyUpdate method in the handler. > > > > salu2 > > > > > I get: > > > Posting file solr.xml to http://localhost:8983/solr/update/stax > > > > > > > > > 0 > > name="QTime">67 > > > > > > > > > > > > > > > Error 400 missing content stream stax > > > > > > > > > HTTP ERROR: 400missing content stream stax > > > RequestURI=/solr/update/stax > > > ... > > > > > > What did I forget that the content stream is not passed to the method? > > > > > > Any tip, hint or shoot in the dark welcome. > > > > > > > > > salu2 > > -- > > Thorsten Scherler thorsten.at.apache.org > > Open Source Java & XMLconsulting, training and solutions > > > > -- Thorsten Scherler thorsten.at.apache.org Open Source Java & XMLconsulting, training and solutions
[jira] Updated: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-139: --- Attachment: SOLR-139-IndexDocumentCommand.patch > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-139: --- Attachment: SOLR-139-XmlUpdater.patch > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470043 ] Ryan McKinley commented on SOLR-139: I just attached a modified XmlUpdateRequestHandler that uses the new IndexDocumentCommand . I left this out originally because I think the discussion around syntax and functionality should be separated... BUT without some example, it is tough to get a sense how this would work, so i added this example. Check the new file: monitor-modifier.xml It starts with: 3007WFP ... If you run ./post.sh monitor-modifier.xml multiple times and check: http://localhost:8983/solr/select?q=id:3007WFP you should notice 1) the price increments by 5 each time 2) there is an additional 'feature' line each time 3) the categories are distinct even if the input is not sku=REMOVE is required because sku is a stored field that is written to with copyField. Although I think this syntax is reasonable, this is just an example intended to spark discussion. Other things to consider: * rather then 'field=mode,' we could do 'field:mode,' this may look less like HTTP request parameter syntax * The update handler could skip any stored field that is the target of a 'copyField' automatically. This is the most normal case, so it may be the most reasonable thing to do. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470061 ] Yonik Seeley commented on SOLR-139: --- Haven't had a chance to check out any code, but a few quick comments: If the field modes were parameters, they could be reused for other update handlers like SQL or CSV Perhaps something like: /update/xml?modify=true&f.price.mode=increment,f.features.mode=append > sku=REMOVE is required because sku is a stored field that is written to with > copyField. I'm not sure I quite grok what REMOVE means yet, and how it fixes the copyField problem. Another way to work around copyField is to only collect stored fields that aren't copyField targets. Then you run the copyField logic while indexing the document again (as you should anyway). I think I'll have a tagging usecase that requires removing a specific field value from a multivalued field. Removing based on a regex might be nice too. though. f.tag.mode=remove f.tag.mode=removeMatching or removeRegex > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-124) use NewIndexModifier, LUCENE-565
[ https://issues.apache.org/jira/browse/SOLR-124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-124: -- Summary: use NewIndexModifier, LUCENE-565 (was: use LUCENE-565) LUCEUE-565 has been committed, but Solr doesn't have that version yet. Lucene 2.1 is right around the corner though. > use NewIndexModifier, LUCENE-565 > > > Key: SOLR-124 > URL: https://issues.apache.org/jira/browse/SOLR-124 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Yonik Seeley > > LUCENE-565 adds extension points to the IndexWriter, and adds delete-by-term > functionality. > We should probably take advantage of this (when available) in our > UpdateHandler (a new one, or modify DU2?) > and perhaps implement a more efficient deleteByQuery. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470064 ] Yonik Seeley commented on SOLR-139: --- I browsed the code really quick, looking for the tricky part... It's here: + openSearcher(); + Term t = new Term( uniqueKey.getName(), uniqueKey.getType().toInternal( id.toString() ) ); + int docID = searcher.getFirstMatch( t ); When you overwrite a document, it is really just adds another instance... so the index contains multiple copies. When we "commit", deletes of the older versions are performed. So you really want the *last* doc matching a term, not the first. Also, to need to make sure that the searcher you are using can actually "see" the last document (once a searcher is opened, it sees documents that were added since the last IndexWriter close(). So a quick fix would be to do a commit() first that would close the writer, and then delete any old copies of docments. Opening and closing readers and writers is *very* expensive though. You can get slightly more sophisticated by checking the pset (set of pending documents), and skip the commit() if the doc you are updating isn't in there (so you know an older searcher will still have the freshest doc for that id). We might be able to get more efficient yet in the future by leveraging NewIndexModifier: SOLR-124 > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470065 ] Yonik Seeley commented on SOLR-139: --- Oh, so we obviously need some tests that modify the same document multiple times w/o a commit in between. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470076 ] Ryan McKinley commented on SOLR-139: > > If the field modes were parameters, they could be reused for other update > handlers like SQL or CSV > Perhaps something like: > /update/xml?modify=true&f.price.mode=increment,f.features.mode=append > Yes. I think we should decide a standard 'modify modes' syntax that can be used across handlers. In this example, I am using the string: mode=cat=DISTINCT,features=APPEND,price=INCREMENT,sku=REMOVE,OVERWRITE and passing it to 'parseFieldModes' in XmlUpdateRequestHandler. Personally, I think all the modes should be specified in a single param rather then a list of them. I vote for a syntax like: cat=DISTINCT,features=APPEND,price=INCREMENT,sku=REMOVE,OVERWRITE or: cat:DISTINCT,features:APPEND,price:INCREMENT,sku:REMOVE,OVERWRITE rather then: DISTINCT APPEND INCREMENT REMOVE OVERWRITE >> sku=REMOVE is required because sku is a stored field that is written to with >> copyField. > I'm not sure I quite grok what REMOVE means yet, and how it fixes the > copyField problem. > I'm using 'REMOVE' to say "remove the previous value of this field before doing anything." Essentially, this makes sure you new document does not start with a value for 'sku'. > Another way to work around copyField is to only collect stored fields that > aren't copyField targets. I just implemented this. It is the most normal case, so it should be the default. It can be overridden by setting the mode for a copyField explicitly. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470080 ] Yonik Seeley commented on SOLR-139: --- If modes are in a single param, a ":" syntax might be nicer because you could cut-n-paste it into a URL w/o escaping. > I'm using 'REMOVE' to say "remove the previous value of this field before > doing anything." Essentially, this makes sure you new > document does not start with a value for 'sku'. That wouldn't work for multi-valued fields though, right? If we keep this option, perhaps we should find a better name... to me "remove" suggests that the given value should be removed. Think adding / removing tag values from a document. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[
https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470081
]
Ryan McKinley commented on SOLR-139:
Is this what you are suggesting?
I added a second searcher to DirectUpdatehandler2 that is only closed when you
call commit();
// Check if the document has not been commited yet
Integer cnt = pset.get( id.toString() );
if( cnt != null && cnt > 0 ) {
commit( new CommitUpdateCommand(false) );
}
if( committedSearcher == null ) {
committedSearcher = core.newSearcher("DirectUpdateHandler2.committed");
}
Term t = new Term( uniqueKey.getName(), uniqueKey.getType().toInternal(
id.toString() ) );
int docID = committedSearcher.getFirstMatch( t );
if( docID >= 0 ) {
Document luceneDoc = committedSearcher.doc( docID );
// set the new doc as the existingDoc + our changes
DocumentBuilder builder = new DocumentBuilder( schema );
add.doc = builder.bulid(
this.modifyDocument( luceneDoc, cmd.doc, cmd.mode ) );
}
- - - - - - -
This passes a new test that adds the same doc multiple times. BUT it does
commit each time.
Another alternative would be to keep a Map of the pending
documents in memory. Then we would not have to commit each time something has
changed.
> Support updateable/modifiable documents
> ---
>
> Key: SOLR-139
> URL: https://issues.apache.org/jira/browse/SOLR-139
> Project: Solr
> Issue Type: Improvement
> Components: update
>Reporter: Ryan McKinley
> Attachments: SOLR-139-IndexDocumentCommand.patch,
> SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch
>
>
> It would be nice to be able to update some fields on a document without
> having to insert the entire document.
> Given the way lucene is structured, (for now) one can only modify stored
> fields.
> While we are at it, we can support incrementing an existing value - I think
> this only makes sense for numbers.
> for background, see:
> http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470082 ] Ryan McKinley commented on SOLR-139: > > That wouldn't work for multi-valued fields though, right? 'REMOVE' on a mult-valued field would clear the old fields before adding the new ones. It is essentially the same as OVERWRITE, but you may or may not pass in a new value on top of the old one. > If we keep this option, perhaps we should find a better name... how about 'IGNORE' or 'CLEAR' It is awkward because it refers to what was in in the document before, not what you are passing in. The more i think about it, I think we should drop the 'REMOVE' option. You can get the same effect using 'OVERWRITE' and passing in a null value. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
SOLR-1.1 Performance Tuning
Hello, Can I send COMMIT and OPTIMIZE concurrently to ADD/DELETE? I temporary send it via synchronized method call. About 100 adds/deletes per second, commit each 1000 adds/deletes, and optimize each 1. Thanks, Fuad
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470084 ] Yonik Seeley commented on SOLR-139: --- > I added a second searcher to DirectUpdatehandler2 that is only closed when > you call commit(); That's OK for right now, but longer term we should re-use the other searcher. Previously, the searcher was only used for deletions, hence we always had to close it before we opened a writer. If it's going to be used for doc retrieval now, we should change closeSearcher() to flushDeletes() and only really close the searcher if there had been deletes pending (from our current deleteByQuery implementation). > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Float Values shown as 1.2E7
Hello, http://www.tokenizer.org shows price (in CAD$) 1.2E7 instead of 1199.99 (that is real!) I use "sfloat" for a price field... If you see different page, click 'refresh' F5; HTTP Caching... Thanks, Fuad
Re: SOLR-1.1 Performance Tuning
On 2/4/07, Fuad Efendi <[EMAIL PROTECTED]> wrote: Hello, Can I send COMMIT and OPTIMIZE concurrently to ADD/DELETE? Yes, but adds will block while a commit or optimize is in progress. I temporary send it via synchronized method call. About 100 adds/deletes per second, commit each 1000 adds/deletes, and optimize each 1. You don't need to commit that often really, and you don't need to optimize until you are all done (assuming an index build and not incremental updating). -Yonik
Re: Float Values shown as 1.2E7
On 2/4/07, Fuad Efendi <[EMAIL PROTECTED]> wrote: Hello, http://www.tokenizer.org shows price (in CAD$) 1.2E7 instead of 1199.99 (that is real!) I use "sfloat" for a price field... An IEEE float only has 24 mantissa bits (7 decimal digits) of precision. 24*log(2)/log(10) = 7.22 Try an sdouble if you need exact prices that large. -Yonik
RE: Float Values shown as 1.2E7
>Try an sdouble if you need exact prices that large. Thanks, Will it broke my indexes if I modify schema.xml and restart Tomcat?
Re: Float Values shown as 1.2E7
On 2/4/07, Fuad Efendi <[EMAIL PROTECTED]> wrote: >Try an sdouble if you need exact prices that large. Thanks, Will it broke my indexes if I modify schema.xml and restart Tomcat? Yes, float isn't compatible with double since they are essentially stored in a modified IEEE format. You will need to re-index. -Yonik
Re: finer granularity of configuration
On 2/3/07, Erik Hatcher <[EMAIL PROTECTED]> wrote:
I'd like to be able to have a common schema.xml and solrconfig.xml
but be able to fire up Solr instances pointed to different data
directories. I realize we have SOLR-79 in JIRA. Is that the
approach we want long term?
Here's an off-the-cuff idea... what if we hook Config.get() to look
for system properties that would override configuration values.
SolrCore does this:
dataDir = SolrConfig.config.get("dataDir",Config.getInstanceDir()
+"data");
If it looked for a system property (perhaps with a "solr." prefix)
you could override anything Config serves up. Thoughts?
Seems like a good idea... and as long as the env vars don't clash,
this doesn't conflict with SOLR-79.
Perhaps the full path with a leading solr.
So -Dsolr.config.dataDir=/path
Speaking of which, is this incorrect in SolrCore.java?
public String getDataDir() { return index_path; } // shouldn't
this return dataDir?
public String getIndexDir() { return index_path; }
Fixed. getDataDir() wasn't used anywhere outside SolrCore, and I
changed uses of getDataDir to getIndexDir.
-Yonik
[jira] Commented: (SOLR-113) Some example + post.sh in docs in client/solrb
[ https://issues.apache.org/jira/browse/SOLR-113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470093 ] Antonio Eggberg commented on SOLR-113: -- Erik: Cool features you added this weekend. We can close this issue as you have already added sample data i.e. delicious examples under example dir. (probably we should add a line in the README file i.e how to load the sample data) .. Very cool work.. I like the new flare logo :-) .. I am just curious what else do you have in mind for next weekend? :-) or for flare .. Regards. > Some example + post.sh in docs in client/solrb > -- > > Key: SOLR-113 > URL: https://issues.apache.org/jira/browse/SOLR-113 > Project: Solr > Issue Type: Wish > Components: clients - ruby - flare > Environment: OSX 10.4 >Reporter: Antonio Eggberg >Priority: Trivial > > I tried flare today really nice :=) > It would be nice to add some example docs like current Solr distro for the > Ruby/Flare client.. If I understand correctly the exampledocs in Solr i.e > /example/exampledocs is not compatible with solrb.. Maybe I am doing > something wrong? if so please clarify and delete the issue. The issue is not > so important but good for the folks that are impetiant. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
AutoCommitTest failing
With the latest changes to AutoCommitTest, now I'm getting failures: [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 5.593 sec [junit] Test org.apache.solr.update.AutoCommitTest FAILED -Yonik
Re: AutoCommitTest failing
On 2/4/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: With the latest changes to AutoCommitTest, now I'm getting failures: [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 5.593 sec [junit] Test org.apache.solr.update.AutoCommitTest FAILED Stack trace: INFO: Opening [EMAIL PROTECTED] main Feb 4, 2007 5:19:41 PM org.apache.solr.core.SolrException log SEVERE: java.util.concurrent.RejectedExecutionException at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1477) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:384) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:856) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:45) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:606) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:486) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:514) at org.apache.solr.update.DirectUpdateHandler2$CommitTracker.run(DirectUpdateHandler2.java:625) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:65) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:168) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595) -Yonik
Re: AutoCommitTest failing
is this on your machine or on the apache machine? I have never had a problem running on my machine. The previous error made more sense to me.. it looked like commit did not actually complete within the given time period. This one seems more like a Problem. Do we need to explicitly handle the ThreadPoolExecutor.AbortPolicy? http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.AbortPolicy.html ryan On 2/4/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 2/4/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: > With the latest changes to AutoCommitTest, now I'm getting failures: > > [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 5.593 sec > [junit] Test org.apache.solr.update.AutoCommitTest FAILED
Re: AutoCommitTest failing
On 2/4/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: is this on your machine or on the apache machine? I have never had a problem running on my machine. My local machine happens every time now. I think Erik said it was failing for him also. -Yonik
[jira] Updated: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-139: --- Attachment: SOLR-139-IndexDocumentCommand.patch > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, > SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: AutoCommitTest failing
Can you tell specifically what part triggers the bad behavior? or is it whenever autocommit gets triggered? I wish I could be more useful, but from my system (win XP, jdk1.5.0_07) things look ok... On 2/4/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 2/4/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: > is this on your machine or on the apache machine? I have never had a > problem running on my machine. My local machine happens every time now. I think Erik said it was failing for him also. -Yonik
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470109 ] Ryan McKinley commented on SOLR-139: I added a new version of SOLR-139-IndexDocumentCommand.patch that: * gets rid of 'REMOVE' option * uses a separate searcher to search for existing docs * includes the XmlUpdateRequestHandler * moves general code from XmlUpdateRequestHandler to SolrPluginUtils * adds a few more tests Can someone with a better lucene understanding look into re-useint the existing searcher as Yonik suggests above - I don't quite understand the other DUH2 implications. I moved the part that parses (and validates) field mode parsing into SolrPluginUtils. This could be used by other RequestHandlers to parse the mode map. The XmlUpdateRequestHandler in this patch should support all legacy calls *except* cases where overwritePending != overwriteCommitted. There are no existing tests with this case, so it is not a problem from the testing standpoint. I don't know if anyone is using this (mis?) feature. > Support updateable/modifiable documents > --- > > Key: SOLR-139 > URL: https://issues.apache.org/jira/browse/SOLR-139 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Ryan McKinley > Attachments: SOLR-139-IndexDocumentCommand.patch, > SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, > SOLR-139-XmlUpdater.patch > > > It would be nice to be able to update some fields on a document without > having to insert the entire document. > Given the way lucene is structured, (for now) one can only modify stored > fields. > While we are at it, we can support incrementing an existing value - I think > this only makes sense for numbers. > for background, see: > http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-60) Remove overwritePending, overwriteCommitted flags?
[ https://issues.apache.org/jira/browse/SOLR-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470110 ] Ryan McKinley commented on SOLR-60: --- As a note, there are no tests where overwritePending != overwriteCommitted In SOLR-139, the XmlUpdateRequestHandler maps all varitions of allowDups, overwriteCommitted and overwritePending to a simpler 'overwrite' -- it throws an exception if overwritePending != overwriteCommitted. > Remove overwritePending, overwriteCommitted flags? > -- > > Key: SOLR-60 > URL: https://issues.apache.org/jira/browse/SOLR-60 > Project: Solr > Issue Type: Improvement > Components: update >Reporter: Yonik Seeley >Priority: Minor > > The overwritePending, overwriteCommitted, allowDups flags seem needlessly > complex and don't add much value. Do people need/use separate control over > pending vs committed documents? > Perhaps all most people need is overwrite=true/false? > overwritePending, overwriteCommitted were originally added because it was a > (mis)feature that another internal search tool had. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: AutoCommitTest failing
On 2/4/07, Ryan McKinley <[EMAIL PROTECTED]> wrote:
Can you tell specifically what part triggers the bad behavior? or is
it whenever autocommit gets triggered?
I wish I could be more useful, but from my system (win XP,
jdk1.5.0_07) things look ok...
I'm on WinXP, and it fails on both Java5 (_09), and Java6.
I'm running off a laptop drive connected via firewire though... so
that might slow it down compared to what you are running.
Here's all the output from when I run it in my IDE (fails there too):
Feb 4, 2007 5:19:37 PM org.apache.solr.core.Config getInstanceDir
INFO: Solr home defaulted to 'solr/' (system property solr.solr.home not set)
Feb 4, 2007 5:19:37 PM org.apache.solr.core.SolrConfig initConfig
INFO: Loaded SolrConfig: solrconfig.xml
Feb 4, 2007 5:19:37 PM org.apache.solr.core.SolrConfig initConfig
INFO: Loaded SolrConfig: solrconfig.xml
Feb 4, 2007 5:19:38 PM org.apache.solr.schema.IndexSchema readConfig
INFO: Reading Solr Schema
Feb 4, 2007 5:19:38 PM org.apache.solr.schema.IndexSchema readConfig
INFO: Schema name=test
Feb 4, 2007 5:19:39 PM org.apache.solr.schema.IndexSchema readConfig
INFO: default search field is text
Feb 4, 2007 5:19:39 PM org.apache.solr.schema.IndexSchema readConfig
INFO: unique key field: id
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler standard=solr.StandardRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler indexinfo=solr.IndexInfoRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler dismaxOldStyleDefaults=solr.DisMaxRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler dismax=solr.DisMaxRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler old=solr.tst.OldRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.tst.OldRequestHandler init
INFO: Unused request handler
arguments:{myparam=1000,ratio=1.4142135,myarr=[1, 2],null=foo}
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler oldagain=solr.tst.OldRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.tst.OldRequestHandler init
INFO: Unused request handler
arguments:{lst1={op=sqrt,val=2},lst2={op=log,val=10.0}}
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler test=solr.tst.TestRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.tst.TestRequestHandler init
INFO: Unused request handler arguments:{}
Feb 4, 2007 5:19:39 PM org.apache.solr.core.RequestHandlers
INFO: adding requestHandler defaults=solr.StandardRequestHandler
Feb 4, 2007 5:19:39 PM org.apache.solr.core.SolrCore
INFO: Opening new SolrCore at solr/,
dataDir=C:\DOCUME~1\Yonik\LOCALS~1\Temp\org.apache.solr.update.AutoCommitTest-testMaxDocs-1170627577515
Feb 4, 2007 5:19:39 PM org.apache.solr.core.SolrCore parseListener
INFO: Searching for listeners: //[EMAIL PROTECTED]"firstSearcher"]
Feb 4, 2007 5:19:39 PM org.apache.solr.core.SolrCore parseListener
INFO: Searching for listeners: //[EMAIL PROTECTED]"newSearcher"]
Feb 4, 2007 5:19:39 PM org.apache.solr.core.SolrCore initIndex
WARNING: Solr index directory
'C:\DOCUME~1\Yonik\LOCALS~1\Temp\org.apache.solr.update.AutoCommitTest-testMaxDocs-1170627577515\index'
doesn't exist. Creating new index...
Feb 4, 2007 5:19:39 PM org.apache.solr.search.SolrIndexSearcher
INFO: Opening [EMAIL PROTECTED] main
Feb 4, 2007 5:19:39 PM org.apache.solr.core.SolrCore registerSearcher
INFO: Registered new searcher [EMAIL PROTECTED] main
Feb 4, 2007 5:19:39 PM
org.apache.solr.update.DirectUpdateHandler2$CommitTracker
INFO: CommitTracker:
[EMAIL PROTECTED]
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A0} in 234ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A1} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A2} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A3} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A4} in 16ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A5} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A6} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A7} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A8} in 16ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A9} in 0ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A10} in 31ms
Feb 4, 2007 5:19:40 PM org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={A11} in 0ms
Feb 4, 2007
Re: AutoCommitTest failing
still trying (in vain) to make this fail on my setup... The one line that looks suspicious from your stack trace is: WARNING: best effort to remove C:\DOCUME~1\Yonik\LOCALS~1\Temp\org.apache.solr.update.AutoCommitTest-testMaxDocs-1170627577515 FAILED ! can you delete the directory where solr stores the testing index manually?
Re: AutoCommitTest failing
On Feb 4, 2007, at 8:34 PM, Yonik Seeley wrote: On 2/4/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: is this on your machine or on the apache machine? I have never had a problem running on my machine. My local machine happens every time now. I think Erik said it was failing for him also. Currently all is well (svn revision 503532): BUILD SUCCESSFUL Total time: 1 minute 16 seconds I ran it twice for good measure, and passed both times. Erik
Re: AutoCommitTest failing
Removing the directory did not help. I just tried a fresh checkout on my C: drive, still no dice. I'll try a reboot, and a remote login to my machine at work next. -Yonik http://lucene.apache.org/java/docs/api/";> http://java.sun.com/";> http://java.sun.com/cgi-bin/bugreport.cgi";> http://junit.sourceforge.net/javadoc/";> http://java.sun.com/j2se/1.5.0/docs/api/";> junit.framework.AssertionFailedError: should find one query failed XPath: //[EMAIL PROTECTED] xml response was:at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:157) at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:98) 0 0
Re: AutoCommitTest failing
OK, the wildcard seems to be norton antivirus. Antivirus on: BUILD FAILED Total time: 2 minutes 22 seconds Antivirus off: BUILD SUCCESSFUL Total time: 1 minute 24 seconds Not yet sure why that would make the difference... and only on this test. -Yonik
Re: AutoCommitTest failing
On 2/4/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: OK, the wildcard seems to be norton antivirus. Antivirus on: BUILD FAILED Total time: 2 minutes 22 seconds Antivirus off: BUILD SUCCESSFUL Total time: 1 minute 24 seconds Not yet sure why that would make the difference... and only on this test. Ah wait.. it still is timing/performance related. With NAV disabled, and one other shell doing "while true; do true; done", the test succeeds in 2m2s, but with two shells doing it, the test fails in 9m1s So I guess my antivirus is slowing my 3GHz P4 down enough to fail. It probably has more to do with the disk access checking than the CPU. [junit] Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 10.687 sec [junit] Test org.apache.solr.update.AutoCommitTest FAILED http://lucene.apache.org/java/docs/api/";> http://java.sun.com/";> http://java.sun.com/cgi-bin/bugreport.cgi";> http://junit.sourceforge.net/javadoc/";> http://java.sun.com/j2se/1.5.0/docs/api/";> junit.framework.AssertionFailedError: should find one query failed XPath: //[EMAIL PROTECTED] xml response was:at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:157) at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:98) junit.framework.AssertionFailedError: should find one query failed XPath: //[EMAIL PROTECTED] xml response was: 0 0 at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:157) at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:150) -Yonik 0 0
Re: AutoCommitTest failing
hymmm. what do you think the best option is? Should we increase the time it waits to check if stuff autocommited? It currently waits 1 second for something that *should* start an autocommit within 1/2 sec. Is the RejectedExecutionException only thrown because JUnit failed? or is this an exception that would be thrown in the wild?
[jira] Created: (SOLR-141) Errors/Exceptions should be formated by ResponseWriter
Errors/Exceptions should be formated by ResponseWriter -- Key: SOLR-141 URL: https://issues.apache.org/jira/browse/SOLR-141 Project: Solr Issue Type: Wish Reporter: Hoss Man Whenever possible, the Solr Dispatcher should to let the ResponseWriter format Exceptions using the format the user expects -- this should in no way change the fact that Exceptions currently generate non 200 HTTP status codes, nor should it prevent the Dispatcher from using the exception message as the HTTP status message -- but clients that want the full details of the error should be able to parse them in the format they expected based on the request. this would also give RequestHandlers the oportunity to return "partial" results - by adding both whatever results they have to the Response as well as the Exception they encountered. situations where this can't happen are obviously: * Exception thrown by ResponseWriter * Exception thrown so early in the request thta the DIspather doesn't know which ResposneWriter the client wanted. ...in those cases, plain text is a wise choice. thing that would probably need to be done to make this work: 1) if the Dispatcher catches an exception, it should call SolrQueryResponse.setException, set the HTTP status code/message as it currently does, but then hand off to the ResponseWriter. 2) Dispatcher needs to check SolrQueryResponse.getException and set the HTTP status code/message just like if it caught the exception itself. 3) all of the ResponseWriters should start looking at SolrQueryResponse.getException if they aren't already, and formatting it in a usefull way. 4) if the ResponseWriter throws an exception, Dispatcher needs to return a nice plain text error page extension to this idea... add a new method to ResponseWriter to generate a generic error message in the appropriate format that Dispatcher can use if the ResponseWriter throws an exception (as a backup before resorting to plain text) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: finer granularity of configuration
: Here's an off-the-cuff idea... what if we hook Config.get() to look : for system properties that would override configuration values. : SolrCore does this: the number of different system properties could get very messy ... i'd much rather see a good patch for SOLR-79 (one thought i had rereading the issue is that the property replacement could happen when the config was parsed into the DOM tree at stratup -- then all of the various methods used to get config values wouldn't need to be changed -- jus hte parsing). another way to make this more customizable, would be to make sure we support Xinclude when parsing xml config files... http://www.w3.org/TR/2004/PR-xinclude-20040930/ -Hoss
