[GitHub] [lucene] jpountz commented on a diff in pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
jpountz commented on code in PR #780: URL: https://github.com/apache/lucene/pull/780#discussion_r869932548 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -269,11 +276,23 @@ public PointValues.Relation compare(byte[] minPackedValue, by

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534702#comment-17534702 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534699#comment-17534699 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[GitHub] [lucene] jpountz merged pull request #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
jpountz merged PR #878: URL: https://github.com/apache/lucene/pull/878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[GitHub] [lucene] Yuti-G commented on pull request #806: LUCENE-10488: Optimize Facets#getTopDims in FloatTaxonomyFacets

2022-05-10 Thread GitBox
Yuti-G commented on PR #806: URL: https://github.com/apache/lucene/pull/806#issuecomment-1123167787 Thanks! Please see the latest commit for the update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [lucene] wormday opened a new pull request, #879: fix errors in DocStats comments

2022-05-10 Thread GitBox
wormday opened a new pull request, #879: URL: https://github.com/apache/lucene/pull/879 Comments for two fields of DocStats are reversed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [lucene] Yuti-G commented on a diff in pull request #779: LUCENE-10488: Optimize Facets#getTopDims in IntTaxonomyFacets

2022-05-10 Thread GitBox
Yuti-G commented on code in PR #779: URL: https://github.com/apache/lucene/pull/779#discussion_r869834570 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/IntTaxonomyFacets.java: ## @@ -169,18 +176,54 @@ public FacetResult getTopChildren(int topN, String dim, String...

[GitHub] [lucene] wjp719 commented on pull request #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
wjp719 commented on PR #878: URL: https://github.com/apache/lucene/pull/878#issuecomment-1123114722 @jpountz I see your commit [8476ac1f6ac5258052f93e85278f70770fe3dfd5](https://github.com/apache/lucene/commit/8476ac1f6ac5258052f93e85278f70770fe3dfd5) to solve tests error, maybe is cause b

[GitHub] [lucene] wjp719 opened a new pull request, #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
wjp719 opened a new pull request, #878: URL: https://github.com/apache/lucene/pull/878 #864 judges if `NumericComparator#iteratorCost` is -1 or not to determine whether to reset `NumericComparator#iteratorCost` in `NumericComparator#setScorer`, but `NumericComparator#iteratorCost` is set a

[GitHub] [lucene] wjp719 commented on pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
wjp719 commented on PR #780: URL: https://github.com/apache/lucene/pull/780#issuecomment-1123104829 @jpountz Hi, I modify the logic. The skip interval is only changed when `updateCounter` is bigger than 256, and the speed of double skip interval is 3 times slower than the speed of divide s

[jira] [Created] (LUCENE-10566) Create an @Concurrent test group for concurrency related tests

2022-05-10 Thread Vigya Sharma (Jira)
Vigya Sharma created LUCENE-10566: - Summary: Create an @Concurrent test group for concurrency related tests Key: LUCENE-10566 URL: https://issues.apache.org/jira/browse/LUCENE-10566 Project: Lucene -

[GitHub] [lucene] vigyasharma commented on a diff in pull request #633: LUCENE-10216: Use MergeScheduler and MergePolicy to run addIndexes(CodecReader[]) merges.

2022-05-10 Thread GitBox
vigyasharma commented on code in PR #633: URL: https://github.com/apache/lucene/pull/633#discussion_r869769905 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/MockRandomMergePolicy.java: ## @@ -86,6 +86,20 @@ public MergeSpecification findMerges( return merg

[GitHub] [lucene] jtibshirani commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
jtibshirani commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r869686535 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level gener

[GitHub] [lucene] gsmiller commented on pull request #843: LUCENE-10538: TopN is not being used in getTopChildren in RangeFacetCounts

2022-05-10 Thread GitBox
gsmiller commented on PR #843: URL: https://github.com/apache/lucene/pull/843#issuecomment-1122824451 Thanks @Yuti-G ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [lucene] LuXugang commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
LuXugang commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122770417 Thanks @mayya-sharipova , let's move to https://github.com/apache/lucene/pull/877 to continue this change. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [lucene] LuXugang opened a new pull request, #877: LUCENE-10502: Use IndexedDISI to store docIds and DirectMonotonicWriter/Reader to handle ordToDoc

2022-05-10 Thread GitBox
LuXugang opened a new pull request, #877: URL: https://github.com/apache/lucene/pull/877 follow up of https://github.com/apache/lucene/pull/792 and https://github.com/apache/lucene/pull/870 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (LUCENE-10502) Use IndexedDISI to store docIds and DirectMonotonicWriter/Reader to handle ordToDoc

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534529#comment-17534529 ] ASF subversion and git services commented on LUCENE-10502: -- Co

[GitHub] [lucene] mayya-sharipova commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122737290 @LuXugang Please feel free to open a new PR to merge vectors-disi-direct feature branch to the main branch when you have time. -- This is an automated message from the Apache Git S

[GitHub] [lucene] mayya-sharipova commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122736714 @msokolov In the interest of time, I will be merging this PR to the feature branch `vectors-disi-direct`. There will an opportunity to provide more feedback once @LuXugang opens a ne

[GitHub] [lucene] mayya-sharipova merged pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova merged PR #870: URL: https://github.com/apache/lucene/pull/870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene

[GitHub] [lucene] rmuir commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
rmuir commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122687335 Thanks both of you for the explanation, sorry for the confusion! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [lucene] Yuti-G commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
Yuti-G commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122671065 Hi @rmuir and @gsmiller, I haven't added `getTopDims` to benchmarks yet, but I plan to add it once all the PRs in LUCENE-10488 are merged. Thanks! -- This is an automated message from the A

[GitHub] [lucene] gsmiller commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
gsmiller commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122652678 > just an observation, this is a large amount of code changes for performance change that may be in the noise? I'm a bit confused. I don't think we actually have benchmarks for this f

[GitHub] [lucene] rmuir commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
rmuir commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869459393 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more ef

[GitHub] [lucene] jpountz commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869429459 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869397640 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] jpountz commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869377332 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869363854 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] rmuir commented on pull request #874: LUCENE-10471 Increse max dims for vectors to 2048

2022-05-10 Thread GitBox
rmuir commented on PR #874: URL: https://github.com/apache/lucene/pull/874#issuecomment-1122395334 My concerns are on the JIRA issue, I don't want them to be forgotten. https://issues.apache.org/jira/browse/LUCENE-10471 I don't know how we can say "we will not recommend further increa

[GitHub] [lucene] mayya-sharipova commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mayya-sharipova commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r869226440 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level g

[GitHub] [lucene] rmuir commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
rmuir commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869220064 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more ef

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869191445 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] jpountz commented on pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on PR #875: URL: https://github.com/apache/lucene/pull/875#issuecomment-1122324472 As a data point, I ran the current version of the benchmark at https://github.com/mikemccand/luceneutil/pull/170 and got the following numbers: ``` Before: id: 12556.30082 msec

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534311#comment-17534311 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[jira] [Created] (LUCENE-10565) Can we "warm" SSDV ordinal maps on index reopen?

2022-05-10 Thread Greg Miller (Jira)
Greg Miller created LUCENE-10565: Summary: Can we "warm" SSDV ordinal maps on index reopen? Key: LUCENE-10565 URL: https://issues.apache.org/jira/browse/LUCENE-10565 Project: Lucene - Core Is

[GitHub] [lucene] gsmiller commented on a diff in pull request #779: LUCENE-10488: Optimize Facets#getTopDims in IntTaxonomyFacets

2022-05-10 Thread GitBox
gsmiller commented on code in PR #779: URL: https://github.com/apache/lucene/pull/779#discussion_r868497732 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/TaxonomyFacets.java: ## @@ -109,7 +109,23 @@ public boolean childrenLoaded() { * @lucene.experimental */

[GitHub] [lucene] rmuir commented on pull request #876: LUCENE-9356: Change test to detect mismatched checksums instead of byte flips.

2022-05-10 Thread GitBox
rmuir commented on PR #876: URL: https://github.com/apache/lucene/pull/876#issuecomment-1122257235 do we really need the linefiledocs? If the test has rare failures, we'll want to reproduce them. practically, linefiledocs just prevents reproducibility (uses a different file in CI th

[GitHub] [lucene] jpountz merged pull request #863: LUCENE-10554: fix bkd test case logic error and java doc error

2022-05-10 Thread GitBox
jpountz merged PR #863: URL: https://github.com/apache/lucene/pull/863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534290#comment-17534290 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[GitHub] [lucene] jpountz merged pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
jpountz merged PR #864: URL: https://github.com/apache/lucene/pull/864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[GitHub] [lucene] wjp719 commented on pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
wjp719 commented on PR #864: URL: https://github.com/apache/lucene/pull/864#issuecomment-1122089465 @jpountz Hi, I have add the change entry. please review again, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [lucene] wjp719 commented on a diff in pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
wjp719 commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868964284 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[jira] [Commented] (LUCENE-9356) Add tests for corruptions caused by byte flips

2022-05-10 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-9356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534216#comment-17534216 ] Adrien Grand commented on LUCENE-9356: -- This test does try to account for CRC32 col

[GitHub] [lucene] jpountz opened a new pull request, #876: LUCENE-9356: Change test to detect mismatched checksums instead of byte flips.

2022-05-10 Thread GitBox
jpountz opened a new pull request, #876: URL: https://github.com/apache/lucene/pull/876 This makes the test more robust and gives a good sense of whether file formats are implementing `checkIntegrity` correctly. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868929217 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868929217 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868923104 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] jpountz commented on a diff in pull request #864: LUCENE-10555: avoid repeated NumericLeafComparator#setScorer calls

2022-05-10 Thread GitBox
jpountz commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868918283 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868923104 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] jpountz commented on a diff in pull request #864: LUCENE-10555: avoid repeated NumericLeafComparator#setScorer calls

2022-05-10 Thread GitBox
jpountz commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868918283 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[GitHub] [lucene] jpountz commented on a diff in pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
jpountz commented on code in PR #780: URL: https://github.com/apache/lucene/pull/780#discussion_r868900109 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -42,6 +42,8 @@ * but in this case you must override both of these methods. */