[GitHub] [lucene] jpountz commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jpountz commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r783701570 ## File path: lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java ## @@ -49,7 +43,7 @@ public abstract void writeField(FieldInfo fieldInfo,

[GitHub] [lucene] jpountz commented on pull request #592: LUCENE-10366: Reduce the number of valid checks for ByteBufferIndexInput#readVInt

2022-01-13 Thread GitBox
jpountz commented on pull request #592: URL: https://github.com/apache/lucene/pull/592#issuecomment-1011911664 Interesting, so it looks like rolling the loop back actually helps a bit. Can you open a new JIRA/PR for this change specifically @gf2121 ? -- This is an automated message from

[jira] [Created] (LUCENE-10376) Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread Feng Guo (Jira)
Feng Guo created LUCENE-10376: - Summary: Roll up the loop in vint/vlong in DataInput Key: LUCENE-10376 URL: https://issues.apache.org/jira/browse/LUCENE-10376 Project: Lucene - Core Issue Type: I

[GitHub] [lucene] jpountz commented on a change in pull request #541: LUCENE-10315: Speed up BKD leaf block ids codec by a 512 ints ForUtil

2022-01-13 Thread GitBox
jpountz commented on a change in pull request #541: URL: https://github.com/apache/lucene/pull/541#discussion_r783744532 ## File path: lucene/core/src/java/org/apache/lucene/util/bkd/BKDForUtil.java ## @@ -0,0 +1,88 @@ +// This file has been automatically generated, DO NOT EDIT

[jira] [Commented] (LUCENE-10376) Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475181#comment-17475181 ] Adrien Grand commented on LUCENE-10376: --- +1 > Roll up the loop in vint/vlong in

[GitHub] [lucene] gf2121 opened a new pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 opened a new pull request #602: URL: https://github.com/apache/lucene/pull/602 https://issues.apache.org/jira/browse/LUCENE-10376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [lucene] gf2121 commented on pull request #592: LUCENE-10366: Reduce the number of valid checks for ByteBufferIndexInput#readVInt

2022-01-13 Thread GitBox
gf2121 commented on pull request #592: URL: https://github.com/apache/lucene/pull/592#issuecomment-1011957978 Thanks @jpountz . I opened https://issues.apache.org/jira/browse/LUCENE-10376 (https://github.com/apache/lucene/pull/602) It can be seen that `IntNRQ` is getting a 10% speed

[GitHub] [lucene] gf2121 edited a comment on pull request #592: LUCENE-10366: Reduce the number of valid checks for ByteBufferIndexInput#readVInt

2022-01-13 Thread GitBox
gf2121 edited a comment on pull request #592: URL: https://github.com/apache/lucene/pull/592#issuecomment-1011957978 Thanks @jpountz . I opened https://issues.apache.org/jira/browse/LUCENE-10376 (https://github.com/apache/lucene/pull/602) It can be seen that `IntNRQ` is getting a 10

[GitHub] [lucene] rmuir commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
rmuir commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1011980920 There are some more copies of this stuff in BufferedIndexInput and BlockPackedReaderIterator -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [lucene] iverase commented on pull request #541: LUCENE-10315: Speed up BKD leaf block ids codec by a 512 ints ForUtil

2022-01-13 Thread GitBox
iverase commented on pull request #541: URL: https://github.com/apache/lucene/pull/541#issuecomment-1012044387 I don't like that the optimisation only works for a specific number of points, it feels very tricky. If we are doing something like that we should remove the possibility of users

[GitHub] [lucene] iverase edited a comment on pull request #541: LUCENE-10315: Speed up BKD leaf block ids codec by a 512 ints ForUtil

2022-01-13 Thread GitBox
iverase edited a comment on pull request #541: URL: https://github.com/apache/lucene/pull/541#issuecomment-1012044387 I don't like that the optimisation only works for a specific number of points, it feels trappy. If we are doing something like that we should remove the possibility of user

[jira] [Created] (LUCENE-10377) Replace 'sortPos' parameter in SortField.getComparator()

2022-01-13 Thread Alan Woodward (Jira)
Alan Woodward created LUCENE-10377: -- Summary: Replace 'sortPos' parameter in SortField.getComparator() Key: LUCENE-10377 URL: https://issues.apache.org/jira/browse/LUCENE-10377 Project: Lucene - Core

[jira] [Created] (LUCENE-10378) Implement Weight#count on PointRangeQuery

2022-01-13 Thread Adrien Grand (Jira)
Adrien Grand created LUCENE-10378: - Summary: Implement Weight#count on PointRangeQuery Key: LUCENE-10378 URL: https://issues.apache.org/jira/browse/LUCENE-10378 Project: Lucene - Core Issue T

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: (was: LUCENE-10367.patch) > Use WANDScorer in CoveringQuery Can accelerate scorer

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: LUCENE-10367.patch > Use WANDScorer in CoveringQuery Can accelerate scorer time > ---

[GitHub] [lucene] romseygeek commented on pull request #603: LUCENE-10377: Replace 'sortPos' with 'enableSkipping' in SortField.getComparator()

2022-01-13 Thread GitBox
romseygeek commented on pull request #603: URL: https://github.com/apache/lucene/pull/603#issuecomment-1012140425 This might make it possible to remove the `getPointsValues()` override from NumericLeafComparator as well as a follow-up, cc @mayya-sharipova -- This is an automated message

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: TestCoveringQueryBench.java > Use WANDScorer in CoveringQuery Can accelerate scorer t

[GitHub] [lucene] jpountz commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
jpountz commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012141279 There is ByteArrayDataInput too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475364#comment-17475364 ] Adrien Grand commented on LUCENE-10367: --- Your patch is only rewriting to a Boolea

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475365#comment-17475365 ] LuYunCheng commented on LUCENE-10367: - [~jpountz]  Yes, It's better to handled this

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475370#comment-17475370 ] LuYunCheng commented on LUCENE-10367: - >  Your patch is only rewriting to a Boolean

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: LUCENE-10367-1.patch > Use WANDScorer in CoveringQuery Can accelerate scorer time > -

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475379#comment-17475379 ] LuYunCheng commented on LUCENE-10367: - i think when the minimumNumberMatch is Const

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Robert Muir (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475381#comment-17475381 ] Robert Muir commented on LUCENE-10374: -- Looks like that did the trick! http://peo

[GitHub] [lucene] gf2121 commented on pull request #541: LUCENE-10315: Speed up BKD leaf block ids codec by a 512 ints ForUtil

2022-01-13 Thread GitBox
gf2121 commented on pull request #541: URL: https://github.com/apache/lucene/pull/541#issuecomment-1012163438 Thanks @iverase ! > I remember in the approach I tried I was batching the docIds by a number (128 or 256) but in general tI didn't see much better performance comparing to t

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784021962 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(F

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784021962 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(F

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784021962 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(F

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784021962 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(F

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784037293 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(F

[jira] [Commented] (LUCENE-10288) Are 1-dimensional kd trees in pre-86 indices always unbalanced trees?

2022-01-13 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475423#comment-17475423 ] Adrien Grand commented on LUCENE-10288: --- This problem would also happen if you us

[GitHub] [lucene] gsmiller commented on pull request #597: LUCENE-10372: Performance of TaxoFacets in Nightly benchmark decreased, revert LUCENE-10350

2022-01-13 Thread GitBox
gsmiller commented on pull request #597: URL: https://github.com/apache/lucene/pull/597#issuecomment-1012217406 Interesting. It appears LUCENE-10372 was the cause of the regression in the nightly benchmarks (see most recent runs of the tasks mentioned [above](https://github.com/apache/luce

[GitHub] [lucene] dweiss commented on pull request #597: LUCENE-10372: Performance of TaxoFacets in Nightly benchmark decreased, revert LUCENE-10350

2022-01-13 Thread GitBox
dweiss commented on pull request #597: URL: https://github.com/apache/lucene/pull/597#issuecomment-1012219388 > LUCENE-10372 when run on our machines while the nightly run would show a regression. I don't think it's a tractable problem in general, given hardware architecture and rac

[GitHub] [lucene] gsmiller edited a comment on pull request #597: LUCENE-10372: Performance of TaxoFacets in Nightly benchmark decreased, revert LUCENE-10350

2022-01-13 Thread GitBox
gsmiller edited a comment on pull request #597: URL: https://github.com/apache/lucene/pull/597#issuecomment-1012217406 Interesting. It appears LUCENE-10372 was the cause of the regression in the nightly benchmarks (see most recent runs of the tasks mentioned [above](https://github.com/apac

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475432#comment-17475432 ] Greg Miller commented on LUCENE-10374: -- Indeed [~rcmuir]. Now I'm even more confus

[GitHub] [lucene] mayya-sharipova commented on pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
mayya-sharipova commented on pull request #601: URL: https://github.com/apache/lucene/pull/601#issuecomment-1012221563 @jtibshirani That's a great change. Agree with Adrien, great to see these speedups with an additional benefit of simplified code. -- This is an automated message from th

[GitHub] [lucene] gsmiller edited a comment on pull request #597: LUCENE-10372: Performance of TaxoFacets in Nightly benchmark decreased, revert LUCENE-10350

2022-01-13 Thread GitBox
gsmiller edited a comment on pull request #597: URL: https://github.com/apache/lucene/pull/597#issuecomment-1012217406 Interesting. It appears LUCENE-10350 was the cause of the regression in the nightly benchmarks (see most recent runs of the tasks mentioned [above](https://github.com/apac

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Robert Muir (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475435#comment-17475435 ] Robert Muir commented on LUCENE-10374: -- [~gsmiller] I agree, looking at the commit

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475438#comment-17475438 ] Greg Miller commented on LUCENE-10374: -- It appears that LUCENE-10350 was responsib

[jira] [Commented] (LUCENE-10375) Speed up HNSW merge by writing combined vector data

2022-01-13 Thread Michael Sokolov (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475446#comment-17475446 ] Michael Sokolov commented on LUCENE-10375: -- Ooh, exciting. That code was compl

[GitHub] [lucene] msokolov commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
msokolov commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784071103 ## File path: lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java ## @@ -271,52 +207,5 @@ public long cost() { public int dimension() {

[GitHub] [lucene] gsmiller opened a new pull request #604: revert LUCENE-10350

2022-01-13 Thread GitBox
gsmiller opened a new pull request #604: URL: https://github.com/apache/lucene/pull/604 This reverts LUCENE-10350 on 9.x as well given that it appears to be the cause of a performance regression (see #597) -- This is an automated message from the Apache Git Service. To respond to the mes

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475507#comment-17475507 ] Adrien Grand commented on LUCENE-10367: --- Thanks for running benchmarks. It looks

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475529#comment-17475529 ] Greg Miller commented on LUCENE-10374: -- OK, here's a thought: LUCENE-10350 actuall

[GitHub] [lucene] gf2121 commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012314786 Thanks f! I've fixed these places and will do a benchmark based on the newest code tonight. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [lucene] gf2121 edited a comment on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 edited a comment on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012314786 Thanks ! I've fixed these places and will do a benchmark based on the newest code tonight. -- This is an automated message from the Apache Git Service. To respond to the m

[jira] [Created] (LUCENE-10379) Count directly into the values array in FastTaxonomyFacetCounts#countAl

2022-01-13 Thread Greg Miller (Jira)
Greg Miller created LUCENE-10379: Summary: Count directly into the values array in FastTaxonomyFacetCounts#countAl Key: LUCENE-10379 URL: https://issues.apache.org/jira/browse/LUCENE-10379 Project: Lu

[GitHub] [lucene] gsmiller opened a new pull request #605: LUCENE-10379: Count directly into the dense values array in FastTaxonomyFacetCounts#countAll

2022-01-13 Thread GitBox
gsmiller opened a new pull request #605: URL: https://github.com/apache/lucene/pull/605 This change just pulls out part of the optimization done (and subsequently reverted) in LUCENE-10350. The idea is to push only part of the optimization to see if we can isolate the performance regressio

[GitHub] [lucene] gsmiller merged pull request #604: revert LUCENE-10350

2022-01-13 Thread GitBox
gsmiller merged pull request #604: URL: https://github.com/apache/lucene/pull/604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr.

[jira] [Commented] (LUCENE-10350) Avoid some null checking for FastTaxonomyFacetCounts#countAll()

2022-01-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475544#comment-17475544 ] ASF subversion and git services commented on LUCENE-10350: -- Co

[jira] [Commented] (LUCENE-10372) Performance of TaxoFacets in Nightly benchmark decreased

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475545#comment-17475545 ] Greg Miller commented on LUCENE-10372: -- This is now reverted on both {{main}} and

[jira] [Resolved] (LUCENE-10372) Performance of TaxoFacets in Nightly benchmark decreased

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Miller resolved LUCENE-10372. -- Fix Version/s: 9.1 10.0 (main) Resolution: Fixed > Performance of T

[GitHub] [lucene] uschindler commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
uschindler commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012341843 Great that the original code was still there in a comment. 🤗 This makes easy to review. I don't fully understand the variant that supports negative values. Do we still n

[GitHub] [lucene] gsmiller commented on pull request #605: LUCENE-10379: Count directly into the dense values array in FastTaxonomyFacetCounts#countAll

2022-01-13 Thread GitBox
gsmiller commented on pull request #605: URL: https://github.com/apache/lucene/pull/605#issuecomment-1012342391 I'm seeing the nice performance improvement locally that @gf2121 and I both observed with LUCENE-10350, so I'm going to go ahead and merge this, which is just a portion of LUCENE

[GitHub] [lucene] gsmiller merged pull request #605: LUCENE-10379: Count directly into the dense values array in FastTaxonomyFacetCounts#countAll

2022-01-13 Thread GitBox
gsmiller merged pull request #605: URL: https://github.com/apache/lucene/pull/605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr.

[jira] [Commented] (LUCENE-10379) Count directly into the values array in FastTaxonomyFacetCounts#countAl

2022-01-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475547#comment-17475547 ] ASF subversion and git services commented on LUCENE-10379: -- Co

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475548#comment-17475548 ] Greg Miller commented on LUCENE-10374: -- Just merged a change for just part 1 above

[jira] [Commented] (LUCENE-10379) Count directly into the values array in FastTaxonomyFacetCounts#countAl

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475550#comment-17475550 ] Greg Miller commented on LUCENE-10379: -- Merged this and will watch the next nightl

[jira] [Commented] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475553#comment-17475553 ] Greg Miller commented on LUCENE-10374: -- {quote}[~gsmiller] I agree, looking at the

[GitHub] [lucene] gf2121 commented on pull request #605: LUCENE-10379: Count directly into the dense values array in FastTaxonomyFacetCounts#countAll

2022-01-13 Thread GitBox
gf2121 commented on pull request #605: URL: https://github.com/apache/lucene/pull/605#issuecomment-1012355358 This is great! Thanks @gsmiller ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784189861 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[GitHub] [lucene] gf2121 commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012371355 Thanks @uschindler . > I don't fully understand the variant that supports negative values. Do we still need it as the default one again supports negative values. The code loo

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784199491 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784199665 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[jira] [Assigned] (LUCENE-10374) Track down the "browse" taxonomy faceting qps regression

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Miller reassigned LUCENE-10374: Assignee: Greg Miller > Track down the "browse" taxonomy faceting qps regression > -

[jira] [Assigned] (LUCENE-10379) Count directly into the values array in FastTaxonomyFacetCounts#countAl

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Miller reassigned LUCENE-10379: Assignee: Greg Miller > Count directly into the values array in FastTaxonomyFacetCounts#

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784214402 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -155,6 +207,37 @@ public void writeField(Field

[GitHub] [lucene-solr] gerlowskija commented on pull request #2634: SOLR-15501: GCSBackupRepository operations without credentials - Branch 8 11

2022-01-13 Thread GitBox
gerlowskija commented on pull request #2634: URL: https://github.com/apache/lucene-solr/pull/2634#issuecomment-1012399441 I ended up targeting 9.0 with these changes, as it _looks_ like 8.x is about wrapped up except for urgent security fixes. With that in mind I'm going to close thi

[GitHub] [lucene-solr] gerlowskija closed pull request #2634: SOLR-15501: GCSBackupRepository operations without credentials - Branch 8 11

2022-01-13 Thread GitBox
gerlowskija closed pull request #2634: URL: https://github.com/apache/lucene-solr/pull/2634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues

[jira] [Assigned] (LUCENE-9952) FacetResult#value can be inaccurate in SortedSetDocValueFacetCounts

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Miller reassigned LUCENE-9952: --- Assignee: Greg Miller > FacetResult#value can be inaccurate in SortedSetDocValueFacetCounts

[jira] [Commented] (LUCENE-9952) FacetResult#value can be inaccurate in SortedSetDocValueFacetCounts

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475634#comment-17475634 ] Greg Miller commented on LUCENE-9952: - I think we're finally setup to address this n

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784251362 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784270583 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[jira] [Commented] (LUCENE-9952) FacetResult#value can be inaccurate in SortedSetDocValueFacetCounts

2022-01-13 Thread Greg Miller (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475682#comment-17475682 ] Greg Miller commented on LUCENE-9952: - OK, I've got a fix for this working but am go

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #416: LUCENE-10054 Make HnswGraph hierarchical

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #416: URL: https://github.com/apache/lucene/pull/416#discussion_r784284382 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraph.java ## @@ -56,75 +57,124 @@ public final class HnswGraph extends KnnGraphV

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #416: LUCENE-10054 Make HnswGraph hierarchical

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #416: URL: https://github.com/apache/lucene/pull/416#discussion_r784284582 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraph.java ## @@ -56,75 +57,124 @@ public final class HnswGraph extends KnnGraphV

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #416: LUCENE-10054 Make HnswGraph hierarchical

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #416: URL: https://github.com/apache/lucene/pull/416#discussion_r784284757 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraph.java ## @@ -40,10 +41,10 @@ * Hyperparameters * * - * numSeed is th

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #416: LUCENE-10054 Make HnswGraph hierarchical

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #416: URL: https://github.com/apache/lucene/pull/416#discussion_r784284891 ## File path: lucene/core/src/test/org/apache/lucene/util/hnsw/TestHnswGraph.java ## @@ -116,33 +116,59 @@ public KnnVectorsFormat getKnnVectorsForm

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #416: LUCENE-10054 Make HnswGraph hierarchical

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #416: URL: https://github.com/apache/lucene/pull/416#discussion_r784285235 ## File path: lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java ## @@ -153,21 +161,56 @@ public void testMergeProducesSameGraph() throws

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784334763 ## File path: lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java ## @@ -49,7 +43,7 @@ public abstract void writeField(FieldInfo fieldInf

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784338686 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -145,6 +139,64 @@ public void writeField(Field

[GitHub] [lucene] jtibshirani commented on pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on pull request #601: URL: https://github.com/apache/lucene/pull/601#issuecomment-1012533036 Thanks for the reviews. For each comment I either pushed a commit or asked a follow-up question. -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [lucene] uschindler commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
uschindler commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784344280 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -155,6 +207,37 @@ public void writeField(FieldI

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784346369 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -138,32 +129,145 @@ public void writeField(Fie

[GitHub] [lucene] jtibshirani commented on pull request #588: LUCENE-10236: Update field-weight used in CombinedFieldQuery scoring calculation (9.1.0 Backporting)

2022-01-13 Thread GitBox
jtibshirani commented on pull request #588: URL: https://github.com/apache/lucene/pull/588#issuecomment-1012582171 > Maybe the best strategy here would be to keep the change entry for whichever version that goes out first (I think it will be 9.0.1? ), and for the other one we can skip the

[jira] [Commented] (LUCENE-10375) Speed up HNSW merge by writing combined vector data

2022-01-13 Thread Julie Tibshirani (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475818#comment-17475818 ] Julie Tibshirani commented on LUCENE-10375: --- I was happily surprised it helpe

[GitHub] [lucene] uschindler commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
uschindler commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784389033 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -138,32 +129,145 @@ public void writeField(Fiel

[GitHub] [lucene] mayya-sharipova commented on a change in pull request #603: LUCENE-10377: Replace 'sortPos' with 'enableSkipping' in SortField.getComparator()

2022-01-13 Thread GitBox
mayya-sharipova commented on a change in pull request #603: URL: https://github.com/apache/lucene/pull/603#discussion_r784378305 ## File path: lucene/core/src/java/org/apache/lucene/search/SortedNumericSortField.java ## @@ -242,12 +241,21 @@ public void setMissingValue(Object

[GitHub] [lucene] mayya-sharipova commented on pull request #603: LUCENE-10377: Replace 'sortPos' with 'enableSkipping' in SortField.getComparator()

2022-01-13 Thread GitBox
mayya-sharipova commented on pull request #603: URL: https://github.com/apache/lucene/pull/603#issuecomment-1012600488 > This might make it possible to remove the getPointsValues() override from NumericLeafComparator as well as a follow-up, cc @mayya-sharipova @romseygeek I think you

[GitHub] [lucene] jtibshirani commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-13 Thread GitBox
jtibshirani commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r784393289 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -138,32 +129,145 @@ public void writeField(Fie

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Status: Patch Available (was: Open) > Use WANDScorer in CoveringQuery Can accelerate scorer time

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: (was: LUCENE-10367-1.patch) > Use WANDScorer in CoveringQuery Can accelerate scor

[GitHub] [lucene] gf2121 commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012797333 Here is the report (30 JVM * 200 repeat): ``` TaskQPS baseline StdDevQPS my_modified_version StdDevPct diff p-value

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: (was: LUCENE-10367.patch) > Use WANDScorer in CoveringQuery Can accelerate scorer

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: LUCENE-10367.patch > Use WANDScorer in CoveringQuery Can accelerate scorer time > ---

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: (was: LUCENE-10367.patch) > Use WANDScorer in CoveringQuery Can accelerate scorer

[jira] [Updated] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuYunCheng updated LUCENE-10367: Attachment: LUCENE-10367.patch > Use WANDScorer in CoveringQuery Can accelerate scorer time > ---

[jira] [Commented] (LUCENE-10367) Use WANDScorer in CoveringQuery Can accelerate scorer time

2022-01-13 Thread LuYunCheng (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475977#comment-17475977 ] LuYunCheng commented on LUCENE-10367: - [~jpountz]  Thanks for reviewing this patch.

[GitHub] [lucene] gf2121 commented on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 commented on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012836713 FYI here are some of my thoughts based on all these benchmark reports: * Rolling up loops for vint/vlong seems not bring a significant speed up (neither significant regressio

[GitHub] [lucene] gf2121 edited a comment on pull request #602: LUCENE-10376: Roll up the loop in vint/vlong in DataInput

2022-01-13 Thread GitBox
gf2121 edited a comment on pull request #602: URL: https://github.com/apache/lucene/pull/602#issuecomment-1012836713 FYI here are some of my thoughts based on all these benchmark reports: * Rolling up loops for vint/vlong seems not bring a significant speed up (neither significant re

  1   2   >