[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12983155#action_12983155 ] Michael McCandless commented on LUCENE-2856: The ReaderEvent is never generated? Is that still work-in-progress? When would this be invoked? Only if IW is pooling readers? Maybe we should hold off on that for a separate issue? Why were the added checks needed in SegmentInfo? Oh I see, it's because you compute the sizeInBytes of the merged segment before the merge completes... hmm. I think I'd prefer that this SegmentInfo not be published until the Type == COMPLETE. How come merge is not also final in MergeEvent? I agree we should change the name. IndexEventListener? I don't think we need CompositeSegmentListener? Why not an API to just add/remove listeners? Also: are we sure this belongs in IWC? This is analogous to infoStream, which is on IW. It's not a config parameter that affects indexing. Should we also track segment flushed/aborted events? Can you add some jdocs and mark the API as experimental? Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch, LUCENE-2856.patch, LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980626#action_12980626 ] Shai Erera commented on LUCENE-2856: I see. I'm ok with both then. Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980803#action_12980803 ] Jason Rutherglen commented on LUCENE-2856: -- I'll added events for flush, open, clone, close and the CompositeSegmentsListener. Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981019#action_12981019 ] Jason Rutherglen commented on LUCENE-2856: -- I separated out a ReaderListener because it's tied to the ReaderPool which eventually will exist external to IW. Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch, LUCENE-2856.patch, LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980390#action_12980390 ] Earwin Burrfoot commented on LUCENE-2856: - A CompositeSegmentListener niftily removes the need for collection. Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980436#action_12980436 ] Jason Rutherglen commented on LUCENE-2856: -- bq. A CompositeSegmentListener niftily removes the need for collection How would it look? Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980448#action_12980448 ] Earwin Burrfoot commented on LUCENE-2856: - A SegmentListener that has a number of children SLs and delegates eventHappened() calls to them. Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[
https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980547#action_12980547
]
Shai Erera commented on LUCENE-2856:
I think start/endTime can be long and not Long?
Maybe instead of adding init/start/endTime to OneMerge, you can pass a 'time'
parameter to MergeEvent. So its signature will be MergeEvent(Type type, long
time, OneMerge merge). The 'time' parameter can then be interpreted according
to Type.
Even better, I think you can remove that parameter entirely, and have
MergeEvent call System.currentTimeMillis() and set its internal member
according to Type?
About this:
{code}
public long getMergeSegmentSize(boolean includeDocStores) throws IOException {
// nocommit: cache this?
return mergeInfo.sizeInBytes(includeDocStores);
}
{code}
SegmentInfo caches sizeInBytes, so I think the 'nocommit' can go away.
However, I did notice that SegmentInfo's cache is potentially buggy -- it
doesn't take into account 'includeDocStores'. I.e., if you call it once w/
'false' and then with 'true' (or vice versa), you won't get the right
sizeInBytes. I'll open a separate issue to fix this.
Create IndexWriter event listener, specifically for merges
--
Key: LUCENE-2856
URL: https://issues.apache.org/jira/browse/LUCENE-2856
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 4.0
Reporter: Jason Rutherglen
Attachments: LUCENE-2856.patch
The issue will allow users to monitor merges occurring within IndexWriter
using a callback notifier event listener. This can be used by external
applications such as Solr to monitor large segment merges.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980558#action_12980558 ] Shai Erera commented on LUCENE-2856: bq. Long's used because it allows null. I see. We could go with a -1 setting also - I see no reason to allocate an object, and use auto-boxing as we do that :). bq. I think it's generally useful to keep track of the time(s) in the OneMerge object. I'm not going to argue too much about that point - was just thinking that OneMerge is already filled with members and it'd be nice if we can add more to it. As for the logging scenario, I think that with the CompositeSegmentsListener an application can easily plug in its SL for logging purposes. But as I said, I don't mind about it too much. BTW, I think SegmentListener is not the proper name? I.e., it does not listen on Segments, right? Maybe a SegmentMergeListener, or simple a MergeListener? It anyway accepts only a MergeEvent Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2856) Create IndexWriter event listener, specifically for merges
[ https://issues.apache.org/jira/browse/LUCENE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980565#action_12980565 ] Jason Rutherglen commented on LUCENE-2856: -- bq. We could go with a -1 setting also - I see no reason to allocate an object, and use auto-boxing as we do that That's a style thing. I prefer null as it's conclusive and doesn't lead to incorrect calculations, just NPEs. bq. I think SegmentListener is not the proper name? I.e., it does not listen on Segments, right? Maybe a SegmentMergeListener, or simple a MergeListener? I had MergeListener, then changed it to be more generic. I think we should add other segment events such as flush, open, clone, close, etc? Create IndexWriter event listener, specifically for merges -- Key: LUCENE-2856 URL: https://issues.apache.org/jira/browse/LUCENE-2856 Project: Lucene - Java Issue Type: Improvement Components: Index Affects Versions: 4.0 Reporter: Jason Rutherglen Attachments: LUCENE-2856.patch The issue will allow users to monitor merges occurring within IndexWriter using a callback notifier event listener. This can be used by external applications such as Solr to monitor large segment merges. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
