[jira] [Created] (LUCENE-9250) LatLonShape Circle queries do not detect crossings over dateline
Ignacio Vera created LUCENE-9250: Summary: LatLonShape Circle queries do not detect crossings over dateline Key: LUCENE-9250 URL: https://issues.apache.org/jira/browse/LUCENE-9250 Project: Lucene - Core Issue Type: Bug Reporter: Ignacio Vera This was actually an oversight in LUCENE-8707. The original PR did support crossing over the dateline added in b59b24d but it was lost during a refactor of the classes. It should be added again. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase opened a new pull request #1289: LUCENE-9250: Add support for Circle2d#intersectsLine around the dateline.
iverase opened a new pull request #1289: LUCENE-9250: Add support for Circle2d#intersectsLine around the dateline. URL: https://github.com/apache/lucene-solr/pull/1289 This was actually an oversight in LUCENE-8707. The original PR did support crossing over the dateline added in b59b24d but it was lost during a refactor of the classes. This PR adds the capability again. It adds more documentation to the class as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant commented on a change in pull request #1270: LUCENE-9237: Faster UniformSplit IntersectBlockReader.
bruno-roustant commented on a change in pull request #1270: LUCENE-9237: Faster
UniformSplit IntersectBlockReader.
URL: https://github.com/apache/lucene-solr/pull/1270#discussion_r383748557
##
File path:
lucene/codecs/src/java/org/apache/lucene/codecs/uniformsplit/IntersectBlockReader.java
##
@@ -18,260 +18,337 @@
package org.apache.lucene.codecs.uniformsplit;
import java.io.IOException;
-import java.util.Objects;
+import java.util.Arrays;
import org.apache.lucene.codecs.PostingsReaderBase;
import org.apache.lucene.index.TermState;
import org.apache.lucene.index.TermsEnum;
import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.util.ArrayUtil;
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.BytesRefBuilder;
import org.apache.lucene.util.IntsRefBuilder;
-import org.apache.lucene.util.StringHelper;
import org.apache.lucene.util.automaton.Automaton;
import org.apache.lucene.util.automaton.ByteRunAutomaton;
import org.apache.lucene.util.automaton.CompiledAutomaton;
-import org.apache.lucene.util.automaton.Operations;
import org.apache.lucene.util.automaton.Transition;
/**
* The "intersect" {@link TermsEnum} response to {@link
UniformSplitTerms#intersect(CompiledAutomaton, BytesRef)},
* intersecting the terms with an automaton.
+ *
+ * By design of the UniformSplit block keys, it is less efficient than
+ * {@code org.apache.lucene.codecs.blocktree.IntersectTermsEnum} for {@link
org.apache.lucene.search.FuzzyQuery} (-37%).
+ * It is slightly slower for {@link org.apache.lucene.search.WildcardQuery}
(-5%) and slightly faster for
+ * {@link org.apache.lucene.search.PrefixQuery} (+5%).
+ *
+ * @lucene.experimental
*/
public class IntersectBlockReader extends BlockReader {
- protected final AutomatonNextTermCalculator nextStringCalculator;
- protected final ByteRunAutomaton runAutomaton;
- protected final BytesRef commonSuffixRef; // maybe null
- protected final BytesRef commonPrefixRef;
- protected final BytesRef startTerm; // maybe null
+ /**
+ * Block iteration order. Whether to move next block, jump to a block away,
or end the iteration.
+ */
+ protected enum BlockIteration {NEXT, SEEK, END}
- /** Set this when our current mode is seeking to this term. Set to null
after. */
- protected BytesRef seekTerm;
+ /**
+ * Threshold that controls when to attempt to jump to a block away.
+ *
+ * This counter is 0 when entering a block. It is incremented each time a
term is rejected by the automaton.
+ * When the counter is greater than or equal to this threshold, then we
compute the next term accepted by
+ * the automaton, with {@link AutomatonNextTermCalculator}, and we jump to a
block away if the next term
+ * accepted is greater than the immediate next term in the block.
+ *
+ * A low value, for example 1, improves the performance of automatons
requiring many jumps, for example
+ * {@link org.apache.lucene.search.FuzzyQuery} and most {@link
org.apache.lucene.search.WildcardQuery}.
+ * A higher value improves the performance of automatons with less or no
jump, for example
+ * {@link org.apache.lucene.search.PrefixQuery}.
+ * A threshold of 4 seems to be a good balance.
+ */
+ protected final int NUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD = 4;
- protected int blockPrefixRunAutomatonState;
- protected int blockPrefixLen;
+ protected final Automaton automaton;
+ protected final ByteRunAutomaton runAutomaton;
+ protected final boolean finite;
+ protected final BytesRef commonSuffix; // maybe null
+ protected final int minTermLength;
+ protected final AutomatonNextTermCalculator nextStringCalculator;
/**
- * Number of bytes accepted by the last call to {@link
#runAutomatonForState}.
+ * Set this when our current mode is seeking to this term. Set to null
after.
+ */
+ protected BytesRef seekTerm;
+ /**
+ * Number of bytes accepted by the automaton when validating the current
term.
+ */
+ protected int numMatchedBytes;
+ /**
+ * Automaton states reached when validating the current term, from 0 to
{@link #numMatchedBytes} - 1.
+ */
+ protected int[] states;
+ /**
+ * Block iteration order determined when scanning the terms in the current
block.
*/
- protected int numBytesAccepted;
+ protected BlockIteration blockIteration;
/**
- * Whether the current term is beyond the automaton common prefix.
- * If true this means the enumeration should stop immediately.
+ * Counter of the number of consecutively rejected terms.
+ * Depending on {@link #NUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD}, this
may trigger a jump to a block away.
*/
- protected boolean beyondCommonPrefix;
+ protected int numConsecutivelyRejectedTerms;
- public IntersectBlockReader(CompiledAutomaton compiled, BytesRef startTerm,
- IndexDictionary.BrowserSupplier
dictionaryBrowserSupplier, IndexInput blockInput, PostingsReade
[GitHub] [lucene-solr] bruno-roustant edited a comment on issue #1270: LUCENE-9237: Faster UniformSplit IntersectBlockReader.
bruno-roustant edited a comment on issue #1270: LUCENE-9237: Faster UniformSplit IntersectBlockReader. URL: https://github.com/apache/lucene-solr/pull/1270#issuecomment-590780144 When I debugged thoroughly to understand what was the limitation of the approach we had (to compute the common prefix between two consecutive block keys in the FST), I saw that actually for all FuzzyQuery the common prefix matched so we entered all blocks. I realized that the FuzzyQuery automaton accepts many variations for the prefix, and the common prefix was not long enough to allow us to filter correctly. I looked at what VarGapFixedInterval did. It jumped all the time after each term to find the next target term accepted by the automaton. And this was sufficiently efficient thanks to a vital optimization that compared the target term to the immediate following term, to actually not jump most of the time. So I applied the same idea to compute the next accepted term and jump, but now with a first condition based on the number of consecutively rejected terms, and by anticipating the comparison of the accepted term with the immediate next term. This is the main factor of the improvement. We leverage also optimizations that speed up the automaton validation of terms in the block. For the proposal of the block prefix in the BlockHeader, does that mean that we have to open the block to get the prefix? Because the speed for FuzzyQuery highly depends on how many blocks we *don't* open. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045265#comment-17045265 ] Bruno Roustant commented on LUCENE-9237: So here is the benchmark for Lucene84 with FST on heap and UniformSplit with block of 26 terms (FST on heap). In this case UniformSplit takes 25.2 MB for the term dictionary (+6.1 MB for facets). Actually the effect of smaller blocks on fuzzy queries is not good. It slows down. I believe it is because we have a jump-to-next-accepted-term policy which suffers from having more blocks (as opposed to a browse-automaton-transitions policy which benefits from more smaller blocks to skip). TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Respell 47.21 (3.7%) 24.78 (1.6%) -47.5% ( -50% - -43%) Fuzzy1 55.44 (5.8%) 29.32 (2.0%) -47.1% ( -51% - -41%) Fuzzy2 52.63 (10.0%) 28.17 (4.0%) -46.5% ( -54% - -36%) PKLookup 179.58 (4.8%) 164.08 (6.3%) -8.6% ( -18% - 2%) Wildcard 14.96 (4.2%) 13.90 (5.8%) -7.0% ( -16% - 3%) OrHighNotHigh 588.73 (8.2%) 562.35 (8.0%) -4.5% ( -19% - 12%) BrowseDayOfYearSSDVFacets 3.64 (3.0%) 3.52 (4.9%) -3.4% ( -11% - 4%) HighSpanNear 4.03 (3.6%) 3.91 (5.0%) -3.0% ( -11% - 5%) LowSpanNear 12.70 (3.9%) 12.34 (4.9%) -2.8% ( -11% - 6%) BrowseMonthSSDVFacets 4.06 (3.1%) 3.95 (4.9%) -2.7% ( -10% - 5%) LowSloppyPhrase 16.87 (5.6%) 16.43 (6.0%) -2.6% ( -13% - 9%) MedSpanNear 12.14 (3.7%) 11.83 (4.6%) -2.6% ( -10% - 5%) OrHighMed 35.49 (4.0%) 34.62 (5.7%) -2.4% ( -11% - 7%) AndHighMed 131.58 (5.0%) 128.48 (6.4%) -2.4% ( -13% - 9%) AndHighHigh 33.98 (4.3%) 33.21 (6.5%) -2.3% ( -12% - 8%) MedSloppyPhrase 14.38 (4.6%) 14.06 (5.4%) -2.2% ( -11% - 8%) OrHighHigh 11.66 (3.6%) 11.41 (5.2%) -2.1% ( -10% - 6%) HighSloppyPhrase 6.04 (5.4%) 5.93 (6.2%) -1.9% ( -12% - 10%) OrNotHighLow 524.51 (7.6%) 514.55 (7.2%) -1.9% ( -15% - 14%) HighIntervalsOrdered 2.75 (3.3%) 2.70 (4.6%) -1.9% ( -9% - 6%) HighTermDayOfYearSort 32.36 (7.1%) 31.77 (9.8%) -1.8% ( -17% - 16%) HighTermMonthSort 17.40 (17.5%) 17.10 (16.9%) -1.7% ( -30% - 39%) OrNotHighMed 551.94 (6.1%) 542.75 (7.6%) -1.7% ( -14% - 12%) LowPhrase 26.37 (3.9%) 26.02 (5.9%) -1.3% ( -10% - 8%) OrHighNotLow 594.32 (8.6%) 586.44 (8.7%) -1.3% ( -17% - 17%) OrHighLow 311.94 (5.2%) 311.00 (6.5%) -0.3% ( -11% - 12%) OrHighNotMed 593.04 (7.0%) 593.24 (8.7%) 0.0% ( -14% - 16%) HighPhrase 196.99 (5.2%) 198.14 (6.2%) 0.6% ( -10% - 12%) AndHighLow 341.56 (6.0%) 344.35 (8.2%) 0.8% ( -12% - 15%) IntNRQ 96.54 (5.4%) 97.37 (6.6%) 0.9% ( -10% - 13%) OrNotHighHigh 605.77 (8.2%) 611.14 (9.5%) 0.9% ( -15% - 20%) LowTerm 1408.69 (7.5%) 1428.02 (7.9%) 1.4% ( -13% - 18%) MedPhrase 154.81 (7.0%) 157.66 (6.2%) 1.8% ( -10% - 16%) Prefix3 91.52 (6.5%) 94.01 (8.8%) 2.7% ( -11% - 19%) HighTerm 992.54 (7.2%) 1022.11 (8.3%) 3.0% ( -11% - 19%) MedTerm 1048.57 (8.0%) 1100.60 (8.3%) 5.0% ( -10% - 23%) BrowseDayOfYearTaxoFacets 0.92 (3.1%) 1.44 (8.2%) 55.9% ( 43% - 69%) BrowseDateTaxoFacets 0.93 (3.1%) 1.45 (7.9%) 56.7% ( 44% - 69%) BrowseMonthTaxoFacets 1.00 (2.8%) 1.67 (7.9%) 67.4% ( 55% - 80%) > Faster TermsEnum intersect for UniformSplit > --- > > Key: LUCENE-9237 > URL: https://issues.apache.org/jira/browse/LUCENE-9237 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > New version of TermsEnum intersect for UniformSplit. It is 75% more efficient > than the previous version for FuzzyQuery. > Compared to BlockTree IntersectTermsEnum: > - It is still slower for FuzzyQuery (-37%) but it is faster than the > previous version (which was -65%). > - It is slightly slower for WildcardQuery (-5%). > - It is slightly faster for PrefixQuery (+5%). Sometimes benchmarks show > more improvement (I've seen up to +17% a fourth of the time). > > When I debugged thoroughly to understand what was the limitation of the > approach we had (to compute the common prefix between two consecutive block > keys in the FST), I saw that actually for all FuzzyQuery the common prefix > matched so we entered all blocks. > I realized that the FuzzyQuery automaton accepts many variations for the > prefix, and the common prefix was not long enough to allow us to filter > correctly. > I looked at what VarGapFixedInterval did. It jumped all the time after each > term to find the next target term accepted by the automaton. And this was > sufficiently efficient thanks to a vital optimization that compared the > target term to the immediate following term, to actually not jump most of the > time. > So I applied the same idea to compute the next accepted term and jump, but > now with a first condition based on the number of consecutively rejected > terms, and by ant
[jira] [Comment Edited] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045274#comment-17045274 ] Bruno Roustant edited comment on LUCENE-9237 at 2/26/20 9:05 AM: - Other wikimediumall benchmark with Lucene84 FST-on-heap and UniformSplit block of 32 (default) FST-on-heap. FST term dictionary is more compact (-20% size) but actually the speed for fuzzy query is better. So my conclusion is that we should not attempt to reduce the block size for UniformSplit. With blocks of 32 terms we have a good balance between lookup speed and fuzzy query speed. I notice that with Lucene84 FST-on-heap, the relative speed of fuzzy queries for UniformSplit is -44%, wildcard is on par, prefix query is +9%. I'll update this Jira entry description. TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Fuzzy2 56.74 (8.1%) 29.96 (3.1%) -47.2% ( -53% - -39%) Fuzzy1 77.44 (9.9%) 43.79 (3.4%) -43.5% ( -51% - -33%) Respell 51.63 (3.3%) 30.85 (2.4%) -40.2% ( -44% - -35%) PKLookup 178.85 (3.0%) 165.93 (3.3%) -7.2% ( -13% - 0%) HighTermMonthSort 53.25 (10.6%) 51.44 (13.0%) -3.4% ( -24% - 22%) BrowseDayOfYearSSDVFacets 3.64 (2.9%) 3.59 (2.1%) -1.3% ( -6% - 3%) HighSpanNear 3.60 (3.2%) 3.55 (2.4%) -1.3% ( -6% - 4%) LowSpanNear 14.52 (3.5%) 14.33 (2.7%) -1.3% ( -7% - 5%) BrowseMonthSSDVFacets 4.06 (2.6%) 4.01 (2.1%) -1.2% ( -5% - 3%) MedSpanNear 4.53 (2.8%) 4.48 (2.0%) -1.0% ( -5% - 3%) LowSloppyPhrase 18.44 (5.4%) 18.26 (4.0%) -1.0% ( -9% - 8%) HighIntervalsOrdered 2.57 (3.4%) 2.55 (2.9%) -1.0% ( -7% - 5%) HighTermDayOfYearSort 38.25 (7.3%) 37.89 (5.9%) -0.9% ( -13% - 13%) IntNRQ 20.81 (16.1%) 20.62 (15.9%) -0.9% ( -28% - 37%) OrHighMed 60.92 (3.9%) 60.54 (4.1%) -0.6% ( -8% - 7%) MedSloppyPhrase 10.53 (7.1%) 10.46 (5.7%) -0.6% ( -12% - 13%) OrHighHigh 10.99 (3.2%) 10.94 (3.4%) -0.5% ( -6% - 6%) HighSloppyPhrase 7.99 (8.1%) 7.95 (7.1%) -0.5% ( -14% - 15%) Wildcard 49.06 (10.8%) 49.00 (10.0%) -0.1% ( -18% - 23%) OrHighLow 51.28 (3.9%) 51.38 (3.7%) 0.2% ( -7% - 8%) MedPhrase 31.21 (3.9%) 31.44 (4.1%) 0.7% ( -7% - 9%) AndHighHigh 22.94 (5.0%) 23.13 (4.5%) 0.9% ( -8% - 10%) AndHighMed 52.33 (4.0%) 52.79 (4.6%) 0.9% ( -7% - 9%) OrNotHighMed 452.70 (5.5%) 462.21 (6.7%) 2.1% ( -9% - 15%) OrNotHighHigh 503.99 (5.9%) 518.75 (6.4%) 2.9% ( -8% - 16%) OrHighNotLow 575.63 (7.8%) 594.89 (6.8%) 3.3% ( -10% - 19%) LowPhrase 94.01 (4.9%) 97.29 (5.3%) 3.5% ( -6% - 14%) OrHighNotHigh 428.99 (5.9%) 444.99 (7.8%) 3.7% ( -9% - 18%) HighPhrase 69.13 (6.8%) 72.63 (5.7%) 5.1% ( -6% - 18%) OrNotHighLow 610.85 (5.7%) 644.82 (8.3%) 5.6% ( -8% - 20%) AndHighLow 414.81 (5.9%) 438.07 (4.7%) 5.6% ( -4% - 17%) MedTerm 1181.99 (6.3%) 1249.96 (5.4%) 5.8% ( -5% - 18%) OrHighNotMed 592.99 (6.2%) 637.89 (7.4%) 7.6% ( -5% - 22%) HighTerm 930.29 (5.8%) 1010.67 (7.2%) 8.6% ( -4% - 22%) Prefix3 125.93 (8.3%) 137.78 (10.1%) 9.4% ( -8% - 30%) LowTerm 1387.71 (7.7%) 1543.23 (8.1%) 11.2% ( -4% - 29%) BrowseDayOfYearTaxoFacets 0.92 (3.7%) 1.44 (6.6%) 56.2% ( 44% - 69%) BrowseDateTaxoFacets 0.93 (3.8%) 1.45 (6.4%) 57.1% ( 45% - 69%) BrowseMonthTaxoFacets 1.00 (4.0%) 1.68 (5.9%) 68.2% ( 56% - 81%) was (Author: broustant): Other benchmark with Lucene84 FST-on-heap and UniformSplit block of 32 (default) FST-on-heap. FST term dictionary is more compact (-20% size) but actually the speed for fuzzy query is better. So my conclusion is that we should not attempt to reduce the block size for UniformSplit. With blocks of 32 terms we have a good balance between lookup speed and fuzzy query speed. I notice that with Lucene84 FST-on-heap, the relative speed of fuzzy queries for UniformSplit is -44%, wildcard is on par, prefix query is +9%. I'll update this Jira entry description. TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Fuzzy2 56.74 (8.1%) 29.96 (3.1%) -47.2% ( -53% - -39%) Fuzzy1 77.44 (9.9%) 43.79 (3.4%) -43.5% ( -51% - -33%) Respell 51.63 (3.3%) 30.85 (2.4%) -40.2% ( -44% - -35%) PKLookup 178.85 (3.0%) 165.93 (3.3%) -7.2% ( -13% - 0%) HighTermMonthSort 53.25 (10.6%) 51.44 (13.0%) -3.4% ( -24% - 22%) BrowseDayOfYearSSDVFacets 3.64 (2.9%) 3.59 (2.1%) -1.3% ( -6% - 3%) HighSpanNear 3.60 (3.2%) 3.55 (2.4%) -1.3% ( -6% - 4%) LowSpanNear 14.52 (3.5%) 14.33 (2.7%) -1.3% ( -7% - 5%) BrowseMonthSSDVFacets 4.06 (2.6%) 4.01 (2.1%) -1.2% ( -5% - 3%) MedSpanNear 4.53 (2.8%) 4.48 (2.0%) -1.0% ( -5% - 3%) LowSloppyPhrase 18.44 (5.4%) 18.26 (4.0%) -1.0% ( -9% - 8%) HighIntervalsOrdered 2.57 (3.4%) 2.55 (2.9%) -1.0% ( -7% - 5%) HighTermDayOfYearSort 38.25 (7.3%) 37.89 (5.9%) -0.9% ( -13% - 13%) IntNRQ 20.81 (16.1%) 20.62 (15.9%) -0.9% ( -28% - 37%) OrHighMed 60.92 (3.9%) 60.54 (4.1%) -0.6% ( -8% - 7%) MedSloppyPhrase 10.53 (7.1%) 10.46 (5.7%) -0.6% ( -12% - 13%) OrHighHigh 10.99 (3.2%) 10.94 (3.4%) -0.5% ( -6% - 6%) HighSloppyPhrase 7.99 (8.1%) 7.95 (7.1%) -0.5% ( -14% - 15%
[jira] [Commented] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045274#comment-17045274 ] Bruno Roustant commented on LUCENE-9237: Other benchmark with Lucene84 FST-on-heap and UniformSplit block of 32 (default) FST-on-heap. FST term dictionary is more compact (-20% size) but actually the speed for fuzzy query is better. So my conclusion is that we should not attempt to reduce the block size for UniformSplit. With blocks of 32 terms we have a good balance between lookup speed and fuzzy query speed. I notice that with Lucene84 FST-on-heap, the relative speed of fuzzy queries for UniformSplit is -44%, wildcard is on par, prefix query is +9%. I'll update this Jira entry description. TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Fuzzy2 56.74 (8.1%) 29.96 (3.1%) -47.2% ( -53% - -39%) Fuzzy1 77.44 (9.9%) 43.79 (3.4%) -43.5% ( -51% - -33%) Respell 51.63 (3.3%) 30.85 (2.4%) -40.2% ( -44% - -35%) PKLookup 178.85 (3.0%) 165.93 (3.3%) -7.2% ( -13% - 0%) HighTermMonthSort 53.25 (10.6%) 51.44 (13.0%) -3.4% ( -24% - 22%) BrowseDayOfYearSSDVFacets 3.64 (2.9%) 3.59 (2.1%) -1.3% ( -6% - 3%) HighSpanNear 3.60 (3.2%) 3.55 (2.4%) -1.3% ( -6% - 4%) LowSpanNear 14.52 (3.5%) 14.33 (2.7%) -1.3% ( -7% - 5%) BrowseMonthSSDVFacets 4.06 (2.6%) 4.01 (2.1%) -1.2% ( -5% - 3%) MedSpanNear 4.53 (2.8%) 4.48 (2.0%) -1.0% ( -5% - 3%) LowSloppyPhrase 18.44 (5.4%) 18.26 (4.0%) -1.0% ( -9% - 8%) HighIntervalsOrdered 2.57 (3.4%) 2.55 (2.9%) -1.0% ( -7% - 5%) HighTermDayOfYearSort 38.25 (7.3%) 37.89 (5.9%) -0.9% ( -13% - 13%) IntNRQ 20.81 (16.1%) 20.62 (15.9%) -0.9% ( -28% - 37%) OrHighMed 60.92 (3.9%) 60.54 (4.1%) -0.6% ( -8% - 7%) MedSloppyPhrase 10.53 (7.1%) 10.46 (5.7%) -0.6% ( -12% - 13%) OrHighHigh 10.99 (3.2%) 10.94 (3.4%) -0.5% ( -6% - 6%) HighSloppyPhrase 7.99 (8.1%) 7.95 (7.1%) -0.5% ( -14% - 15%) Wildcard 49.06 (10.8%) 49.00 (10.0%) -0.1% ( -18% - 23%) OrHighLow 51.28 (3.9%) 51.38 (3.7%) 0.2% ( -7% - 8%) MedPhrase 31.21 (3.9%) 31.44 (4.1%) 0.7% ( -7% - 9%) AndHighHigh 22.94 (5.0%) 23.13 (4.5%) 0.9% ( -8% - 10%) AndHighMed 52.33 (4.0%) 52.79 (4.6%) 0.9% ( -7% - 9%) OrNotHighMed 452.70 (5.5%) 462.21 (6.7%) 2.1% ( -9% - 15%) OrNotHighHigh 503.99 (5.9%) 518.75 (6.4%) 2.9% ( -8% - 16%) OrHighNotLow 575.63 (7.8%) 594.89 (6.8%) 3.3% ( -10% - 19%) LowPhrase 94.01 (4.9%) 97.29 (5.3%) 3.5% ( -6% - 14%) OrHighNotHigh 428.99 (5.9%) 444.99 (7.8%) 3.7% ( -9% - 18%) HighPhrase 69.13 (6.8%) 72.63 (5.7%) 5.1% ( -6% - 18%) OrNotHighLow 610.85 (5.7%) 644.82 (8.3%) 5.6% ( -8% - 20%) AndHighLow 414.81 (5.9%) 438.07 (4.7%) 5.6% ( -4% - 17%) MedTerm 1181.99 (6.3%) 1249.96 (5.4%) 5.8% ( -5% - 18%) OrHighNotMed 592.99 (6.2%) 637.89 (7.4%) 7.6% ( -5% - 22%) HighTerm 930.29 (5.8%) 1010.67 (7.2%) 8.6% ( -4% - 22%) Prefix3 125.93 (8.3%) 137.78 (10.1%) 9.4% ( -8% - 30%) LowTerm 1387.71 (7.7%) 1543.23 (8.1%) 11.2% ( -4% - 29%) BrowseDayOfYearTaxoFacets 0.92 (3.7%) 1.44 (6.6%) 56.2% ( 44% - 69%) BrowseDateTaxoFacets 0.93 (3.8%) 1.45 (6.4%) 57.1% ( 45% - 69%) BrowseMonthTaxoFacets 1.00 (4.0%) 1.68 (5.9%) 68.2% ( 56% - 81%) > Faster TermsEnum intersect for UniformSplit > --- > > Key: LUCENE-9237 > URL: https://issues.apache.org/jira/browse/LUCENE-9237 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > New version of TermsEnum intersect for UniformSplit. It is 75% more efficient > than the previous version for FuzzyQuery. > Compared to BlockTree IntersectTermsEnum: > - It is still slower for FuzzyQuery (-37%) but it is faster than the > previous version (which was -65%). > - It is slightly slower for WildcardQuery (-5%). > - It is slightly faster for PrefixQuery (+5%). Sometimes benchmarks show > more improvement (I've seen up to +17% a fourth of the time). > > When I debugged thoroughly to understand what was the limitation of the > approach we had (to compute the common prefix between two consecutive block > keys in the FST), I saw that actually for all FuzzyQuery the common prefix > matched so we entered all blocks. > I realized that the FuzzyQuery automaton accepts many variations for the > prefix, and the common prefix was not long enough to allow us to filter > correctly. > I looked at what VarGapFixedInterval did. It jumped all the time after each > term to find the next target term accepted by the automaton. And this was > sufficiently efficient thanks to a vital optimization that compared the > target term to the immediate following term, to actually not jump most of the > time. > So I applied the same idea to compute the next accepted term and jump, but > now with a first condition based on the number of cons
[jira] [Comment Edited] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045265#comment-17045265 ] Bruno Roustant edited comment on LUCENE-9237 at 2/26/20 9:05 AM: - So here is the wikimediumall benchmark for Lucene84 with FST on heap and UniformSplit with block of 26 terms (FST on heap). In this case UniformSplit takes 25.2 MB for the term dictionary (+6.1 MB for facets). Actually the effect of smaller blocks on fuzzy queries is not good. It slows down. I believe it is because we have a jump-to-next-accepted-term policy which suffers from having more blocks (as opposed to a browse-automaton-transitions policy which benefits from more smaller blocks to skip). TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Respell 47.21 (3.7%) 24.78 (1.6%) -47.5% ( -50% - -43%) Fuzzy1 55.44 (5.8%) 29.32 (2.0%) -47.1% ( -51% - -41%) Fuzzy2 52.63 (10.0%) 28.17 (4.0%) -46.5% ( -54% - -36%) PKLookup 179.58 (4.8%) 164.08 (6.3%) -8.6% ( -18% - 2%) Wildcard 14.96 (4.2%) 13.90 (5.8%) -7.0% ( -16% - 3%) OrHighNotHigh 588.73 (8.2%) 562.35 (8.0%) -4.5% ( -19% - 12%) BrowseDayOfYearSSDVFacets 3.64 (3.0%) 3.52 (4.9%) -3.4% ( -11% - 4%) HighSpanNear 4.03 (3.6%) 3.91 (5.0%) -3.0% ( -11% - 5%) LowSpanNear 12.70 (3.9%) 12.34 (4.9%) -2.8% ( -11% - 6%) BrowseMonthSSDVFacets 4.06 (3.1%) 3.95 (4.9%) -2.7% ( -10% - 5%) LowSloppyPhrase 16.87 (5.6%) 16.43 (6.0%) -2.6% ( -13% - 9%) MedSpanNear 12.14 (3.7%) 11.83 (4.6%) -2.6% ( -10% - 5%) OrHighMed 35.49 (4.0%) 34.62 (5.7%) -2.4% ( -11% - 7%) AndHighMed 131.58 (5.0%) 128.48 (6.4%) -2.4% ( -13% - 9%) AndHighHigh 33.98 (4.3%) 33.21 (6.5%) -2.3% ( -12% - 8%) MedSloppyPhrase 14.38 (4.6%) 14.06 (5.4%) -2.2% ( -11% - 8%) OrHighHigh 11.66 (3.6%) 11.41 (5.2%) -2.1% ( -10% - 6%) HighSloppyPhrase 6.04 (5.4%) 5.93 (6.2%) -1.9% ( -12% - 10%) OrNotHighLow 524.51 (7.6%) 514.55 (7.2%) -1.9% ( -15% - 14%) HighIntervalsOrdered 2.75 (3.3%) 2.70 (4.6%) -1.9% ( -9% - 6%) HighTermDayOfYearSort 32.36 (7.1%) 31.77 (9.8%) -1.8% ( -17% - 16%) HighTermMonthSort 17.40 (17.5%) 17.10 (16.9%) -1.7% ( -30% - 39%) OrNotHighMed 551.94 (6.1%) 542.75 (7.6%) -1.7% ( -14% - 12%) LowPhrase 26.37 (3.9%) 26.02 (5.9%) -1.3% ( -10% - 8%) OrHighNotLow 594.32 (8.6%) 586.44 (8.7%) -1.3% ( -17% - 17%) OrHighLow 311.94 (5.2%) 311.00 (6.5%) -0.3% ( -11% - 12%) OrHighNotMed 593.04 (7.0%) 593.24 (8.7%) 0.0% ( -14% - 16%) HighPhrase 196.99 (5.2%) 198.14 (6.2%) 0.6% ( -10% - 12%) AndHighLow 341.56 (6.0%) 344.35 (8.2%) 0.8% ( -12% - 15%) IntNRQ 96.54 (5.4%) 97.37 (6.6%) 0.9% ( -10% - 13%) OrNotHighHigh 605.77 (8.2%) 611.14 (9.5%) 0.9% ( -15% - 20%) LowTerm 1408.69 (7.5%) 1428.02 (7.9%) 1.4% ( -13% - 18%) MedPhrase 154.81 (7.0%) 157.66 (6.2%) 1.8% ( -10% - 16%) Prefix3 91.52 (6.5%) 94.01 (8.8%) 2.7% ( -11% - 19%) HighTerm 992.54 (7.2%) 1022.11 (8.3%) 3.0% ( -11% - 19%) MedTerm 1048.57 (8.0%) 1100.60 (8.3%) 5.0% ( -10% - 23%) BrowseDayOfYearTaxoFacets 0.92 (3.1%) 1.44 (8.2%) 55.9% ( 43% - 69%) BrowseDateTaxoFacets 0.93 (3.1%) 1.45 (7.9%) 56.7% ( 44% - 69%) BrowseMonthTaxoFacets 1.00 (2.8%) 1.67 (7.9%) 67.4% ( 55% - 80%) was (Author: broustant): So here is the benchmark for Lucene84 with FST on heap and UniformSplit with block of 26 terms (FST on heap). In this case UniformSplit takes 25.2 MB for the term dictionary (+6.1 MB for facets). Actually the effect of smaller blocks on fuzzy queries is not good. It slows down. I believe it is because we have a jump-to-next-accepted-term policy which suffers from having more blocks (as opposed to a browse-automaton-transitions policy which benefits from more smaller blocks to skip). TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Respell 47.21 (3.7%) 24.78 (1.6%) -47.5% ( -50% - -43%) Fuzzy1 55.44 (5.8%) 29.32 (2.0%) -47.1% ( -51% - -41%) Fuzzy2 52.63 (10.0%) 28.17 (4.0%) -46.5% ( -54% - -36%) PKLookup 179.58 (4.8%) 164.08 (6.3%) -8.6% ( -18% - 2%) Wildcard 14.96 (4.2%) 13.90 (5.8%) -7.0% ( -16% - 3%) OrHighNotHigh 588.73 (8.2%) 562.35 (8.0%) -4.5% ( -19% - 12%) BrowseDayOfYearSSDVFacets 3.64 (3.0%) 3.52 (4.9%) -3.4% ( -11% - 4%) HighSpanNear 4.03 (3.6%) 3.91 (5.0%) -3.0% ( -11% - 5%) LowSpanNear 12.70 (3.9%) 12.34 (4.9%) -2.8% ( -11% - 6%) BrowseMonthSSDVFacets 4.06 (3.1%) 3.95 (4.9%) -2.7% ( -10% - 5%) LowSloppyPhrase 16.87 (5.6%) 16.43 (6.0%) -2.6% ( -13% - 9%) MedSpanNear 12.14 (3.7%) 11.83 (4.6%) -2.6% ( -10% - 5%) OrHighMed 35.49 (4.0%) 34.62 (5.7%) -2.4% ( -11% - 7%) AndHighMed 131.58 (5.0%) 128.48 (6.4%) -2.4% ( -13% - 9%) AndHighHigh 33.98 (4.3%) 33.21 (6.5%) -2.3% ( -12% - 8%) MedSloppyPhrase 14.38 (4.6%) 14.06 (5.4%) -2.2% ( -11% - 8%) OrHighHigh 11.66 (3.6%) 11.41 (5.2%) -2.1% ( -10% - 6%) HighSloppyPhrase 6.04 (5.4%) 5.93 (6.2%) -1.9% ( -12% - 10%) OrNotHighLow 524.51 (7.6%) 514.55 (7.2%) -1.9% ( -15% - 14%) HighIntervalsOrdered 2.75 (3.3%) 2.70 (4.6%) -1.9% ( -9%
[jira] [Updated] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant updated LUCENE-9237: --- Description: New version of TermsEnum intersect for UniformSplit. It is 75% more efficient than the previous version for FuzzyQuery. Compared to BlockTree IntersectTermsEnum: - It is still slower for FuzzyQuery (between -37% and -44% in our benchmarks) but it is faster than the previous version (which was -65%). - It is on par or slightly slower for WildcardQuery (between -5% and 0%). - It is slightly faster for PrefixQuery (between +5% and +10%). When I debugged thoroughly to understand what was the limitation of the previous approach we had (to compute the common prefix between two consecutive block keys in the FST), I saw that actually for all FuzzyQuery the common prefix matched so we entered all blocks. I realized that the FuzzyQuery automaton accepts many variations for the prefix, and the common prefix was not long enough to allow us to filter correctly. I looked at what VarGapFixedInterval did. It jumped all the time after each term to find the next target term accepted by the automaton. And this was sufficiently efficient thanks to a vital optimization that compared the target term to the immediate following term, to actually not jump most of the time. So I applied the same idea to compute the next accepted term and jump, but now with a first condition based on the number of consecutively rejected terms, and by anticipating the comparison of the accepted term with the immediate next term. This is the main factor of the improvement. We leverage also other optimizations that speed up the automaton validation of each sequential term in the block. was: New version of TermsEnum intersect for UniformSplit. It is 75% more efficient than the previous version for FuzzyQuery. Compared to BlockTree IntersectTermsEnum: - It is still slower for FuzzyQuery (-37%) but it is faster than the previous version (which was -65%). - It is slightly slower for WildcardQuery (-5%). - It is slightly faster for PrefixQuery (+5%). Sometimes benchmarks show more improvement (I've seen up to +17% a fourth of the time). When I debugged thoroughly to understand what was the limitation of the approach we had (to compute the common prefix between two consecutive block keys in the FST), I saw that actually for all FuzzyQuery the common prefix matched so we entered all blocks. I realized that the FuzzyQuery automaton accepts many variations for the prefix, and the common prefix was not long enough to allow us to filter correctly. I looked at what VarGapFixedInterval did. It jumped all the time after each term to find the next target term accepted by the automaton. And this was sufficiently efficient thanks to a vital optimization that compared the target term to the immediate following term, to actually not jump most of the time. So I applied the same idea to compute the next accepted term and jump, but now with a first condition based on the number of consecutively rejected terms, and by anticipating the comparison of the accepted term with the immediate next term. This is the main factor of the improvement. We leverage also other optimizations that speed up the automaton validation of each sequential term in the block. > Faster TermsEnum intersect for UniformSplit > --- > > Key: LUCENE-9237 > URL: https://issues.apache.org/jira/browse/LUCENE-9237 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > New version of TermsEnum intersect for UniformSplit. It is 75% more efficient > than the previous version for FuzzyQuery. > Compared to BlockTree IntersectTermsEnum: > - It is still slower for FuzzyQuery (between -37% and -44% in our > benchmarks) but it is faster than the previous version (which was -65%). > - It is on par or slightly slower for WildcardQuery (between -5% and 0%). > - It is slightly faster for PrefixQuery (between +5% and +10%). > > When I debugged thoroughly to understand what was the limitation of the > previous approach we had (to compute the common prefix between two > consecutive block keys in the FST), I saw that actually for all FuzzyQuery > the common prefix matched so we entered all blocks. > I realized that the FuzzyQuery automaton accepts many variations for the > prefix, and the common prefix was not long enough to allow us to filter > correctly. > I looked at what VarGapFixedInterval did. It jumped all the time after each > term to find the next target term accepted by the automaton. And this was > sufficiently efficient thanks to a vital optimization that compared the > target term t
[jira] [Comment Edited] (LUCENE-9237) Faster TermsEnum intersect for UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045265#comment-17045265 ] Bruno Roustant edited comment on LUCENE-9237 at 2/26/20 9:10 AM: - So here is the wikimediumall benchmark for Lucene84 with FST on heap and UniformSplit with block of 26 terms (FST on heap). In this case UniformSplit takes 25.2 MB for the term dictionary (+6.1 MB for facets). Actually the effect of smaller blocks on fuzzy queries is not good. It slows down a little. I believe it is because we have a jump-to-next-accepted-term policy which suffers from having more blocks (as opposed to a browse-automaton-transitions policy which benefits from more smaller blocks to skip). TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Respell 47.21 (3.7%) 24.78 (1.6%) -47.5% ( -50% - -43%) Fuzzy1 55.44 (5.8%) 29.32 (2.0%) -47.1% ( -51% - -41%) Fuzzy2 52.63 (10.0%) 28.17 (4.0%) -46.5% ( -54% - -36%) PKLookup 179.58 (4.8%) 164.08 (6.3%) -8.6% ( -18% - 2%) Wildcard 14.96 (4.2%) 13.90 (5.8%) -7.0% ( -16% - 3%) OrHighNotHigh 588.73 (8.2%) 562.35 (8.0%) -4.5% ( -19% - 12%) BrowseDayOfYearSSDVFacets 3.64 (3.0%) 3.52 (4.9%) -3.4% ( -11% - 4%) HighSpanNear 4.03 (3.6%) 3.91 (5.0%) -3.0% ( -11% - 5%) LowSpanNear 12.70 (3.9%) 12.34 (4.9%) -2.8% ( -11% - 6%) BrowseMonthSSDVFacets 4.06 (3.1%) 3.95 (4.9%) -2.7% ( -10% - 5%) LowSloppyPhrase 16.87 (5.6%) 16.43 (6.0%) -2.6% ( -13% - 9%) MedSpanNear 12.14 (3.7%) 11.83 (4.6%) -2.6% ( -10% - 5%) OrHighMed 35.49 (4.0%) 34.62 (5.7%) -2.4% ( -11% - 7%) AndHighMed 131.58 (5.0%) 128.48 (6.4%) -2.4% ( -13% - 9%) AndHighHigh 33.98 (4.3%) 33.21 (6.5%) -2.3% ( -12% - 8%) MedSloppyPhrase 14.38 (4.6%) 14.06 (5.4%) -2.2% ( -11% - 8%) OrHighHigh 11.66 (3.6%) 11.41 (5.2%) -2.1% ( -10% - 6%) HighSloppyPhrase 6.04 (5.4%) 5.93 (6.2%) -1.9% ( -12% - 10%) OrNotHighLow 524.51 (7.6%) 514.55 (7.2%) -1.9% ( -15% - 14%) HighIntervalsOrdered 2.75 (3.3%) 2.70 (4.6%) -1.9% ( -9% - 6%) HighTermDayOfYearSort 32.36 (7.1%) 31.77 (9.8%) -1.8% ( -17% - 16%) HighTermMonthSort 17.40 (17.5%) 17.10 (16.9%) -1.7% ( -30% - 39%) OrNotHighMed 551.94 (6.1%) 542.75 (7.6%) -1.7% ( -14% - 12%) LowPhrase 26.37 (3.9%) 26.02 (5.9%) -1.3% ( -10% - 8%) OrHighNotLow 594.32 (8.6%) 586.44 (8.7%) -1.3% ( -17% - 17%) OrHighLow 311.94 (5.2%) 311.00 (6.5%) -0.3% ( -11% - 12%) OrHighNotMed 593.04 (7.0%) 593.24 (8.7%) 0.0% ( -14% - 16%) HighPhrase 196.99 (5.2%) 198.14 (6.2%) 0.6% ( -10% - 12%) AndHighLow 341.56 (6.0%) 344.35 (8.2%) 0.8% ( -12% - 15%) IntNRQ 96.54 (5.4%) 97.37 (6.6%) 0.9% ( -10% - 13%) OrNotHighHigh 605.77 (8.2%) 611.14 (9.5%) 0.9% ( -15% - 20%) LowTerm 1408.69 (7.5%) 1428.02 (7.9%) 1.4% ( -13% - 18%) MedPhrase 154.81 (7.0%) 157.66 (6.2%) 1.8% ( -10% - 16%) Prefix3 91.52 (6.5%) 94.01 (8.8%) 2.7% ( -11% - 19%) HighTerm 992.54 (7.2%) 1022.11 (8.3%) 3.0% ( -11% - 19%) MedTerm 1048.57 (8.0%) 1100.60 (8.3%) 5.0% ( -10% - 23%) BrowseDayOfYearTaxoFacets 0.92 (3.1%) 1.44 (8.2%) 55.9% ( 43% - 69%) BrowseDateTaxoFacets 0.93 (3.1%) 1.45 (7.9%) 56.7% ( 44% - 69%) BrowseMonthTaxoFacets 1.00 (2.8%) 1.67 (7.9%) 67.4% ( 55% - 80%) was (Author: broustant): So here is the wikimediumall benchmark for Lucene84 with FST on heap and UniformSplit with block of 26 terms (FST on heap). In this case UniformSplit takes 25.2 MB for the term dictionary (+6.1 MB for facets). Actually the effect of smaller blocks on fuzzy queries is not good. It slows down. I believe it is because we have a jump-to-next-accepted-term policy which suffers from having more blocks (as opposed to a browse-automaton-transitions policy which benefits from more smaller blocks to skip). TaskQPS Lucene84 StdDevQPS UniformSplit26 StdDev Pct diff Respell 47.21 (3.7%) 24.78 (1.6%) -47.5% ( -50% - -43%) Fuzzy1 55.44 (5.8%) 29.32 (2.0%) -47.1% ( -51% - -41%) Fuzzy2 52.63 (10.0%) 28.17 (4.0%) -46.5% ( -54% - -36%) PKLookup 179.58 (4.8%) 164.08 (6.3%) -8.6% ( -18% - 2%) Wildcard 14.96 (4.2%) 13.90 (5.8%) -7.0% ( -16% - 3%) OrHighNotHigh 588.73 (8.2%) 562.35 (8.0%) -4.5% ( -19% - 12%) BrowseDayOfYearSSDVFacets 3.64 (3.0%) 3.52 (4.9%) -3.4% ( -11% - 4%) HighSpanNear 4.03 (3.6%) 3.91 (5.0%) -3.0% ( -11% - 5%) LowSpanNear 12.70 (3.9%) 12.34 (4.9%) -2.8% ( -11% - 6%) BrowseMonthSSDVFacets 4.06 (3.1%) 3.95 (4.9%) -2.7% ( -10% - 5%) LowSloppyPhrase 16.87 (5.6%) 16.43 (6.0%) -2.6% ( -13% - 9%) MedSpanNear 12.14 (3.7%) 11.83 (4.6%) -2.6% ( -10% - 5%) OrHighMed 35.49 (4.0%) 34.62 (5.7%) -2.4% ( -11% - 7%) AndHighMed 131.58 (5.0%) 128.48 (6.4%) -2.4% ( -13% - 9%) AndHighHigh 33.98 (4.3%) 33.21 (6.5%) -2.3% ( -12% - 8%) MedSloppyPhrase 14.38 (4.6%) 14.06 (5.4%) -2.2% ( -11% - 8%) OrHighHigh 11.66 (3.6%) 11.41 (5.2%) -2.1% ( -10% - 6%) HighSloppyPhrase 6.04 (5.4%) 5.93 (6.2%) -1.9% ( -12% - 10%) OrNotHighLow 524.51 (7.6%) 514.55 (7.2%) -1.9% ( -15% - 14%) HighIntervalsOrdered 2.75 (3.3%)
[jira] [Commented] (LUCENE-7908) fix-or-suppress 3 'Resource leak' warnings in (ReadersAnd|FrozenBuffered)Updates.java
[
https://issues.apache.org/jira/browse/LUCENE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045291#comment-17045291
]
Andras Salamon commented on LUCENE-7908:
I think these warnings have been already fixed.
> fix-or-suppress 3 'Resource leak' warnings in
> (ReadersAnd|FrozenBuffered)Updates.java
> -
>
> Key: LUCENE-7908
> URL: https://issues.apache.org/jira/browse/LUCENE-7908
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Christine Poerschke
>Priority: Minor
>
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java#L846
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java#L186
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java#L144
> {code}
> [ecj-lint] 4. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java
> (at line 846)
> [ecj-lint] SegmentReader newReader = new SegmentReader(info, reader,
> liveDocs, info.info.maxDoc() - delCount);
> [ecj-lint] ^
> [ecj-lint] Resource leak: 'newReader' is never closed
> [ecj-lint] 2. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java
> (at line 144)
> [ecj-lint] RAMOutputStream out = new RAMOutputStream();
> [ecj-lint] ^^^
> [ecj-lint] Resource leak: 'out' is never closed
> [ecj-lint] 3. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java
> (at line 186)
> [ecj-lint] RAMOutputStream out = new RAMOutputStream();
> [ecj-lint] ^^^
> [ecj-lint] Resource leak: 'out' is never closed
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-site] janhoy opened a new pull request #15: Conditionally show security warning block only if news last 60 days.
janhoy opened a new pull request #15: Conditionally show security warning block only if news last 60 days. URL: https://github.com/apache/lucene-site/pull/15 SOLR-13910 followup This PR converts from static calculation in Pelican of age for last secuity news, to dynamic calculation in JS, so the yellow warning box goes away without re-publish. I also chose 60 days as "recent" instead of 90 days. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-site] janhoy commented on a change in pull request #15: Conditionally show security warning block only if news last 60 days.
janhoy commented on a change in pull request #15: Conditionally show security warning block only if news last 60 days. URL: https://github.com/apache/lucene-site/pull/15#discussion_r384401449 ## File path: .editorconfig ## @@ -6,13 +6,7 @@ end_of_line = lf insert_final_newline = true indent_style = space charset = utf-8 -indent_size = 4 +indent_size = 2 Review comment: I did this because all JS is also indent of 2, so it appears that this should be the default so we can remove the override sections further down. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13910) Create security news feed on website with RSS/Atom feed
[ https://issues.apache.org/jira/browse/SOLR-13910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045361#comment-17045361 ] Jan Høydahl commented on SOLR-13910: New PR which removes the yellow security warning dynamically after 60 days (in JS): https://github.com/apache/lucene-site/pull/15 > Create security news feed on website with RSS/Atom feed > --- > > Key: SOLR-13910 > URL: https://issues.apache.org/jira/browse/SOLR-13910 > Project: Solr > Issue Type: Task > Components: website >Reporter: Adam Walz >Assignee: Jan Høydahl >Priority: Minor > Attachments: recent-security-ann.png, security-page-with-table.png, > security-page-with-table.png, solr-security-page.png > > Time Spent: 20m > Remaining Estimate: 0h > > From [~janhoy] > We're in the process of migrating our web site to Git and in that same > process we also change CMS from an ASF one to Pelican. The new site has > built-in support for news posts as individual files and also RSS feeds of > those. So I propose to add [https://lucene.apache.org/solr/security.html] > to the site, including a list of newest CVEs and an RSS/Atom feed to go > along with it. This way users have ONE place to visit to check security > announcements and they can monitor RSS to be alerted once we post a new > announcement. > We could also add RSS feeds for Lucene-core news and Solr-news sections > of course. > At the same time I propose that the news on the front-page > [lucene.apache.org|http://lucene.apache.org/] > is replaced with widgets that show the title only of the last 3 announcements > from Lucene, Solr and PyLucene sub projects. That front page is waaay > too long :) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9251) Polygon tessellator fails to detect some collinear points
Ignacio Vera created LUCENE-9251:
Summary: Polygon tessellator fails to detect some collinear points
Key: LUCENE-9251
URL: https://issues.apache.org/jira/browse/LUCENE-9251
Project: Lucene - Core
Issue Type: Bug
Reporter: Ignacio Vera
A user of Elasticsearch [has reported|
https://discuss.elastic.co/t/unable-to-tessellate-shape-error-on-indexing-es-7-6/220867]
a tessellation error in a valid polygon. The reported polygon is quite complex
but after digging a bit, the problem is the tesselator fails to detect some
colinearities. In particular, in complex tessellation we can end up with two
equal edges with different flag in {{isEdgeFromPolygon}}. Still we should be
able to remove that co-linearity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase opened a new pull request #1290: LUCENE-9251: Filter equal edges with different value on isEdgeFromPolygon
iverase opened a new pull request #1290: LUCENE-9251: Filter equal edges with different value on isEdgeFromPolygon URL: https://github.com/apache/lucene-solr/pull/1290 See https://issues.apache.org/jira/browse/LUCENE-9251 Unfortunately, I haven't been able to produce a meaningful unit test as I can only reproduce the issue with the complex polygon provided by the user. Still I think the change make sense. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14282) /get handler doesn't return copied fields
Andrei Minin created SOLR-14282:
---
Summary: /get handler doesn't return copied fields
Key: SOLR-14282
URL: https://issues.apache.org/jira/browse/SOLR-14282
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: search, SolrJ
Affects Versions: 8.4
Environment: SOLR 8.4.0, SOLRJ, Oracle Java 8
Reporter: Andrei Minin
Attachments: copied_fields_test.zip, managed-schema.xml
We are using /get handler to retrieve documents by id in our Java application
(SolrJ)
I found that copied fields are missing in documents returned by /get handler
but same documents returned by query contain copied (by schema) fields.
Attached documents:
# Integration test project archive
# Managed schema file for SOLR
SOLR schema details:
# Unique field name "d_ida_s"
# Lowecase text type definition:
{code:java}
{code}
3. Copy field instruction sample:
{code:java}
{code}
ConcurrenceUserNamea_s is string type field and ConcurrenceUserNameu_lca_s is
lower case text type field.
Integration test uploads document to SOLR server and makes 2 requests: one
using /get rest point to fetch document by id and one using query :.
Document returned by /get rest, doesn't have copied fields while document
returned by query, contains copied fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated LUCENE-9252:
---
Status: Patch Available (was: Open)
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated LUCENE-9252:
---
Attachment: LUCENE-9252-01.patch
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
Andras Salamon created LUCENE-9252:
--
Summary: Fix or suppress 6 rawtypes/unchecked precommit warnings
in lucene/spatial
Key: LUCENE-9252
URL: https://issues.apache.org/jira/browse/LUCENE-9252
Project: Lucene - Core
Issue Type: Sub-task
Reporter: Andras Salamon
Attachments: LUCENE-9252-01.patch
There are 6 precommit warnings in org/apache/lucene/spatial:
{noformat}
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
warning: [rawtypes] found raw type: Geo3dShape
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
warning: [rawtypes] found raw type: ShapeCollection
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
warning: [rawtypes] found raw type: ShapeCollection
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
warning: [rawtypes] found raw type: Geo3dShape
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
warning: [rawtypes] found raw type: Geo3dShape
[javac]
/Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp
dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp URL: https://github.com/apache/lucene-solr/pull/1288#discussion_r384498162 ## File path: solr/CHANGES.txt ## @@ -110,6 +110,9 @@ Improvements * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production (janhoy) +* SOLR-14281: Make sharedLib configurable through SysProp and allow multiple comma separated paths. Review comment: SysProp should by "sys-prop" or system property. The way you capitalized it makes it seem like some class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp
dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib
configurable through SysProp
URL: https://github.com/apache/lucene-solr/pull/1288#discussion_r384499124
##
File path: solr/core/src/java/org/apache/solr/core/CoreContainer.java
##
@@ -607,19 +609,31 @@ public PackageStoreAPI getPackageStoreAPI() {
public void load() {
log.debug("Loading cores into CoreContainer [instanceDir={}]",
loader.getInstancePath());
+// Always add $SOLR_HOME/lib to the shared resource loader
+Set libDirs = new HashSet<>();
+libDirs.add("lib");
+
+if (!StringUtils.isBlank(cfg.getSharedLibDirectory())) {
+ List sharedLibs =
Arrays.asList(cfg.getSharedLibDirectory().split("\\s*,\\s*"));
Review comment:
I'm almost certain we've got a utility that does this so that you don't have
to define your regexp
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp
dsmiley commented on a change in pull request #1288: SOLR-14281: Make sharedLib
configurable through SysProp
URL: https://github.com/apache/lucene-solr/pull/1288#discussion_r384499506
##
File path: solr/core/src/java/org/apache/solr/core/CoreContainer.java
##
@@ -607,19 +609,31 @@ public PackageStoreAPI getPackageStoreAPI() {
public void load() {
log.debug("Loading cores into CoreContainer [instanceDir={}]",
loader.getInstancePath());
+// Always add $SOLR_HOME/lib to the shared resource loader
+Set libDirs = new HashSet<>();
Review comment:
Use LinkedHashSet
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045567#comment-17045567
]
Michael McCandless commented on LUCENE-9252:
Thanks [~asalamon74], patch looks great, I'll push shortly.
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14283) Fix NPE caused by SOLR-14217
Gus Heck created SOLR-14283:
---
Summary: Fix NPE caused by SOLR-14217
Key: SOLR-14283
URL: https://issues.apache.org/jira/browse/SOLR-14283
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Tests
Affects Versions: 8.5
Reporter: Gus Heck
Assignee: Gus Heck
Fix For: 8.5
Though it's goals were laudable, SOLR-14217 ran afoul of an untested
requirement that the SolrTestCaseJ4 class continue to function even if
ExternalPaths#SOURCE_HOME has been set to null by the logic in
org.apache.solr.util.ExternalPaths#determineSourceHome
The result is that in any almost all usages of the solr test framework outside
of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
SolrCloudTestCase the following exception would be thrown:
{code}
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
at
java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
at
java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
at java.base/java.util.Properties.put(Properties.java:1337)
at java.base/java.util.Properties.setProperty(Properties.java:225)
at java.base/java.lang.System.setProperty(System.java:895)
at
org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
This ticket will fix the issue and provides a test so that we don't break
nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9004) Approximate nearest vector search
[
https://issues.apache.org/jira/browse/LUCENE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045570#comment-17045570
]
Xin-Chun Zhang commented on LUCENE-9004:
As mentioned by [~tomoko], the static cache (for the graph structure) isn't a
good implementation. I have some ideas to share,
# Do not cache the whole graph structure on memory. We could cache the entry
points rather than the whole graph structure. When searching the nearest
neighbors, we only care about the neighbors of current point, and each point is
visited only once. Therefore, we could read its neighbors when the point is
actually visited. There's a simple implementation for HNSW (marked as
*_searchNeighborsV2_*) in my personal branch
([https://github.com/irvingzhang/lucene-solr/blob/jira/LUCENE-9136/lucene/core/src/java/org/apache/lucene/util/hnsw/HNSWGraphReader.java]).
The implementation could be further optimized, for example, we could store and
read neighbors by layers because we only need the neighbors in current search
layer.
# If higher search performance is preferred, we could keep the whole graph
structure in graph reader (similar to the implementation of
_Lucene90IvfFlatIndexReader_,
[https://github.com/irvingzhang/lucene-solr/blob/jira/LUCENE-9136/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90IvfFlatIndexReader.java]).
The on-heap cache would be released when the reader is closed.
It's likely not the best solution to the static cache, just for discussions.
> Approximate nearest vector search
> -
>
> Key: LUCENE-9004
> URL: https://issues.apache.org/jira/browse/LUCENE-9004
> Project: Lucene - Core
> Issue Type: New Feature
>Reporter: Michael Sokolov
>Priority: Major
> Attachments: hnsw_layered_graph.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> "Semantic" search based on machine-learned vector "embeddings" representing
> terms, queries and documents is becoming a must-have feature for a modern
> search engine. SOLR-12890 is exploring various approaches to this, including
> providing vector-based scoring functions. This is a spinoff issue from that.
> The idea here is to explore approximate nearest-neighbor search. Researchers
> have found an approach based on navigating a graph that partially encodes the
> nearest neighbor relation at multiple scales can provide accuracy > 95% (as
> compared to exact nearest neighbor calculations) at a reasonable cost. This
> issue will explore implementing HNSW (hierarchical navigable small-world)
> graphs for the purpose of approximate nearest vector search (often referred
> to as KNN or k-nearest-neighbor search).
> At a high level the way this algorithm works is this. First assume you have a
> graph that has a partial encoding of the nearest neighbor relation, with some
> short and some long-distance links. If this graph is built in the right way
> (has the hierarchical navigable small world property), then you can
> efficiently traverse it to find nearest neighbors (approximately) in log N
> time where N is the number of nodes in the graph. I believe this idea was
> pioneered in [1]. The great insight in that paper is that if you use the
> graph search algorithm to find the K nearest neighbors of a new document
> while indexing, and then link those neighbors (undirectedly, ie both ways) to
> the new document, then the graph that emerges will have the desired
> properties.
> The implementation I propose for Lucene is as follows. We need two new data
> structures to encode the vectors and the graph. We can encode vectors using a
> light wrapper around {{BinaryDocValues}} (we also want to encode the vector
> dimension and have efficient conversion from bytes to floats). For the graph
> we can use {{SortedNumericDocValues}} where the values we encode are the
> docids of the related documents. Encoding the interdocument relations using
> docids directly will make it relatively fast to traverse the graph since we
> won't need to lookup through an id-field indirection. This choice limits us
> to building a graph-per-segment since it would be impractical to maintain a
> global graph for the whole index in the face of segment merges. However
> graph-per-segment is a very natural at search time - we can traverse each
> segments' graph independently and merge results as we do today for term-based
> search.
> At index time, however, merging graphs is somewhat challenging. While
> indexing we build a graph incrementally, performing searches to construct
> links among neighbors. When merging segments we must construct a new graph
> containing elements of all the merged segments. Ideally we would somehow
> preserve the work done when building the initial graphs, but at least as a
> start I'd propose we co
[jira] [Updated] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Priority: Blocker (was: Major)
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Attachment: SOLR-14217.patch
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14217.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9227) Make page ready for pure HTTPS
[
https://issues.apache.org/jira/browse/LUCENE-9227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045577#comment-17045577
]
Michael McCandless commented on LUCENE-9227:
{quote}Is [~mikemccand]'s Lucene Search engine based on pure Lucene still
available and maintained (with HTTPS)?
{quote}
The only Lucene based search engine I'm currently personally running in
production is
[jirasearch.mikemccandless.com|http://jirasearch.mikemccandless.com/search.py?index=jira],
for searching all Lucene/Solr/Tika/Infra Jira issues ... it tries to
showcase/dog food a number of fun pure Lucene features ... [blog post
here|http://blog.mikemccandless.com/2016/10/jiraseseach-20-dog-food-using-lucene-to.html].
But, alas, it's still HTTP only :(
> Make page ready for pure HTTPS
> --
>
> Key: LUCENE-9227
> URL: https://issues.apache.org/jira/browse/LUCENE-9227
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Blocker
>
> The web page can currently be visited using HTTPS but this brings warning:
> - Both search providers create a form that passes USER ENTERED INPUT using no
> encryption. This is not allowed due to GDPR. We have to fix this asap. It
> looks like [~otis] search is working with HTTPS (if we change domain name),
> but the Lucidworks does not
> - There were some CSS files loaded with HTTP (fonts from Google - this was
> fixed)
> Once those 2 problems are fixed (I grepped for HTTP and still found many
> links with HTTP, but looks like no images or scripts or css anymore), I'd
> like to add a permanent redirect http://lucene.apache.org/ ->
> https://lucene.apache.org to the htaccess template file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] romseygeek merged pull request #1239: LUCENE-9207: Don't build span queries in QueryBuilder
romseygeek merged pull request #1239: LUCENE-9207: Don't build span queries in QueryBuilder URL: https://github.com/apache/lucene-solr/pull/1239 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9207) Don't build SpanQuery in QueryBuilder
[ https://issues.apache.org/jira/browse/LUCENE-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045579#comment-17045579 ] ASF subversion and git services commented on LUCENE-9207: - Commit 98dafe2e1047cdfe8ed8bbfb444fab2798193a9f in lucene-solr's branch refs/heads/master from Alan Woodward [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=98dafe2 ] LUCENE-9207: Don't build span queries in QueryBuilder (#1239) QueryBuilder currently has special logic for graph phrase queries with no slop, constructing a spanquery that attempts to follow all paths using a combination of OR and NEAR queries. However, this type of query has known bugs(LUCENE-7398). This commit removes this logic and just builds a disjunction of phrase queries, one phrase per path. > Don't build SpanQuery in QueryBuilder > - > > Key: LUCENE-9207 > URL: https://issues.apache.org/jira/browse/LUCENE-9207 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Alan Woodward >Assignee: Alan Woodward >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Subtask of LUCENE-9204. QueryBuilder currently has special logic for graph > phrase queries with no slop, constructing a spanquery that attempts to follow > all paths using a combination of OR and NEAR queries. Given the known bugs > in this type of query (LUCENE-7398) and that we would like to move span > queries out of core in any case, we should remove this logic and just build a > disjunction of phrase queries, one phrase per path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7398) Nested Span Queries are buggy
[ https://issues.apache.org/jira/browse/LUCENE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045580#comment-17045580 ] ASF subversion and git services commented on LUCENE-7398: - Commit 98dafe2e1047cdfe8ed8bbfb444fab2798193a9f in lucene-solr's branch refs/heads/master from Alan Woodward [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=98dafe2 ] LUCENE-9207: Don't build span queries in QueryBuilder (#1239) QueryBuilder currently has special logic for graph phrase queries with no slop, constructing a spanquery that attempts to follow all paths using a combination of OR and NEAR queries. However, this type of query has known bugs(LUCENE-7398). This commit removes this logic and just builds a disjunction of phrase queries, one phrase per path. > Nested Span Queries are buggy > - > > Key: LUCENE-7398 > URL: https://issues.apache.org/jira/browse/LUCENE-7398 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Christoph Goller >Assignee: Alan Woodward >Priority: Critical > Attachments: LUCENE-7398-20160814.patch, LUCENE-7398-20160924.patch, > LUCENE-7398-20160925.patch, LUCENE-7398.patch, LUCENE-7398.patch, > LUCENE-7398.patch, TestSpanCollection.java > > > Example for a nested SpanQuery that is not working: > Document: Human Genome Organization , HUGO , is trying to coordinate gene > mapping research worldwide. > Query: spanNear([body:coordinate, spanOr([spanNear([body:gene, body:mapping], > 0, true), body:gene]), body:research], 0, true) > The query should match "coordinate gene mapping research" as well as > "coordinate gene research". It does not match "coordinate gene mapping > research" with Lucene 5.5 or 6.1, it did however match with Lucene 4.10.4. It > probably stopped working with the changes on SpanQueries in 5.3. I will > attach a unit test that shows the problem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045581#comment-17045581
]
Gus Heck commented on SOLR-14283:
-
Given the severity of this, and with an 8.5 release coming up soon, I'm going
to commit this as soon as I run the full tests, but if there are tweaks or
suggestions we can certainly adjust.
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14217.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9207) Don't build SpanQuery in QueryBuilder
[ https://issues.apache.org/jira/browse/LUCENE-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Woodward resolved LUCENE-9207. --- Fix Version/s: master (9.0) Resolution: Fixed > Don't build SpanQuery in QueryBuilder > - > > Key: LUCENE-9207 > URL: https://issues.apache.org/jira/browse/LUCENE-9207 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Alan Woodward >Assignee: Alan Woodward >Priority: Major > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > > Subtask of LUCENE-9204. QueryBuilder currently has special logic for graph > phrase queries with no slop, constructing a spanquery that attempts to follow > all paths using a combination of OR and NEAR queries. Given the known bugs > in this type of query (LUCENE-7398) and that we would like to move span > queries out of core in any case, we should remove this logic and just build a > disjunction of phrase queries, one phrase per path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045582#comment-17045582
]
ASF subversion and git services commented on LUCENE-9252:
-
Commit 61e0b4cd87b9d4f3563de6b6a960fd26ae448f97 in lucene-solr's branch
refs/heads/master from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=61e0b4c ]
LUCENE-9252: fix javac linter warnings in spatial-extras (thanks Andras Salamon)
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045584#comment-17045584
]
ASF subversion and git services commented on LUCENE-9252:
-
Commit 570109f73de4ef8521e76339878534282196afed in lucene-solr's branch
refs/heads/branch_8x from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=570109f ]
LUCENE-9252: fix javac linter warnings in spatial-extras (thanks Andras Salamon)
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9252) Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
[
https://issues.apache.org/jira/browse/LUCENE-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-9252:
---
Fix Version/s: 8.5
master (9.0)
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~asalamon74]!
> Fix or suppress 6 rawtypes/unchecked precommit warnings in lucene/spatial
> --
>
> Key: LUCENE-9252
> URL: https://issues.apache.org/jira/browse/LUCENE-9252
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Andras Salamon
>Priority: Minor
> Fix For: master (9.0), 8.5
>
> Attachments: LUCENE-9252-01.patch
>
>
> There are 6 precommit warnings in org/apache/lucene/spatial:
> {noformat}
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:69:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:122:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dBinaryCodec.java:127:
> warning: [rawtypes] found raw type: ShapeCollection
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [rawtypes] found raw type: Geo3dShape
> [javac]
> /Users/andrassalamon/src/lucene-solr-upstream/lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/Geo3dShapeSphereModelRectRelationTest.java:73:
> warning: [unchecked] unchecked call to Geo3dShape(T,SpatialContext) as a
> member of the raw type Geo3dShape{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Attachment: (was: SOLR-14217.patch)
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Attachment: SOLR-14283.patch
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1233: LUCENE-9202: refactor leaf collectors in TopFieldCollector
mikemccand commented on a change in pull request #1233: LUCENE-9202: refactor
leaf collectors in TopFieldCollector
URL: https://github.com/apache/lucene-solr/pull/1233#discussion_r384540627
##
File path: lucene/core/src/java/org/apache/lucene/search/TopFieldCollector.java
##
@@ -69,6 +69,79 @@ public void setScorer(Scorable scorer) throws IOException {
}
}
+ private abstract class TopFieldLeafCollector extends
MultiComparatorLeafCollector {
Review comment:
OK let's do that in followon issue. Progress not perfection!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045622#comment-17045622
]
Gus Heck commented on SOLR-14283:
-
The patch unfortunately only works on 8.5, due to new java reflection rules.
I'm not finding powermock via any searches in the code base. We likely need to
bring that library in for 9.x and Java 11, unless there is an alternate
suggestion. This supposed workaround is not working for me:
https://stackoverflow.com/a/56043252/1172174 on
openjdk 11.0.6 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed
mode, sharing)
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other
[GitHub] [lucene-solr] janhoy opened a new pull request #1291: LUCENE-9016: RefGuide meta doc for how to publish website
janhoy opened a new pull request #1291: LUCENE-9016: RefGuide meta doc for how to publish website URL: https://github.com/apache/lucene-solr/pull/1291 Using new git site This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE caused by SOLR-14217
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045629#comment-17045629
]
David Smiley commented on SOLR-14283:
-
Please update the title to be more meaningful at a glance. A reader shouldn't
be forced to follow to SOLR-14238 to know what's going on. I suggest renaming
"SolrTestCaseJ4 no longer works on external projects" or something like that
(assuming I ascertained the issue correctly). Wow that's serious if true.
> Fix NPE caused by SOLR-14217
>
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apa
[jira] [Commented] (LUCENE-9016) Document how to update web site
[ https://issues.apache.org/jira/browse/LUCENE-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045631#comment-17045631 ] Jan Høydahl commented on LUCENE-9016: - [GitHub Pull Request #1291|https://github.com/apache/lucene-solr/pull/1291] for refGuide meta doc. Draft new page can be seen here [https://github.com/cominvent/lucene-solr/blob/lucene-9016-publish-refguide/solr/solr-ref-guide/src/meta-docs/publish.adoc] > Document how to update web site > --- > > Key: LUCENE-9016 > URL: https://issues.apache.org/jira/browse/LUCENE-9016 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Jan Høydahl >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Find all documentation across Wiki, RefGuide, scripts and website itself that > talks about how to update or publish the web site, and update accordingly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14283) Fix NPE in SolrTestCaseJ4
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Summary: Fix NPE in SolrTestCaseJ4 (was: Fix NPE caused by SOLR-14217)
> Fix NPE in SolrTestCaseJ4
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045632#comment-17045632
]
Gus Heck commented on SOLR-14283:
-
Good point, thx
> Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14283) Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck updated SOLR-14283:
Summary: Fix NPE in SolrTestCaseJ4 preventing it from working on external
projects (was: Fix NPE in SolrTestCaseJ4)
> Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045638#comment-17045638
]
Gus Heck commented on SOLR-14283:
-
Given that this needs to be fixed in 8.5 (a 2 line null check), the testing is
about 1000 times heavier than the fix, and the addition of a library to the
project is a rat-hole I don't have time for today, is it reasonable to commit
the above to branch_8x and then open a follow on ticket to solve this in 9.x
(also blocker) ?
> Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
--
[jira] [Resolved] (LUCENE-8954) Refactor Nori(Korean) Analyzer
[ https://issues.apache.org/jira/browse/LUCENE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namgyu Kim resolved LUCENE-8954. Resolution: Fixed > Refactor Nori(Korean) Analyzer > -- > > Key: LUCENE-8954 > URL: https://issues.apache.org/jira/browse/LUCENE-8954 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Namgyu Kim >Assignee: Namgyu Kim >Priority: Minor > Fix For: 8.x, master (9.0) > > Time Spent: 2h > Remaining Estimate: 0h > > There are many codes that can be refactored in the Nori analyzer. > (whitespace, wrong type casting, unnecessary throws, C-style array, ...) > I think it's good to proceed if we can. > It has nothing to do with the actual working of Nori. > I'll just remove unnecessary code and make the code simple. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8954) Refactor Nori(Korean) Analyzer
[ https://issues.apache.org/jira/browse/LUCENE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045654#comment-17045654 ] Namgyu Kim commented on LUCENE-8954: Jenkins builds were completed well. [https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/806/] [https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/2262/] [https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/676/] > Refactor Nori(Korean) Analyzer > -- > > Key: LUCENE-8954 > URL: https://issues.apache.org/jira/browse/LUCENE-8954 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Namgyu Kim >Assignee: Namgyu Kim >Priority: Minor > Fix For: 8.x, master (9.0) > > Time Spent: 2h > Remaining Estimate: 0h > > There are many codes that can be refactored in the Nori analyzer. > (whitespace, wrong type casting, unnecessary throws, C-style array, ...) > I think it's good to proceed if we can. > It has nothing to do with the actual working of Nori. > I'll just remove unnecessary code and make the code simple. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14283) Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
[
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045667#comment-17045667
]
Gus Heck commented on SOLR-14283:
-
Also, I'd guess that the present state with both gradle and ant files active is
not the optimal time to be adding a library to 9.x...
> Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested
> requirement that the SolrTestCaseJ4 class continue to function even if
> ExternalPaths#SOURCE_HOME has been set to null by the logic in
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at java.base/java.util.Properties.put(Properties.java:1337)
> at java.base/java.util.Properties.setProperty(Properties.java:225)
> at java.base/java.lang.System.setProperty(System.java:895)
> at
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break
> nearly every user of the test framework other than ourselves in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9227) Make page ready for pure HTTPS
[ https://issues.apache.org/jira/browse/LUCENE-9227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045671#comment-17045671 ] Uwe Schindler commented on LUCENE-9227: --- Thanks [~mikemccand]. I removed the dropdown on the Lucene website already, so we have now only one search engine (OSC) left over. I am fine with that, also looks better than before. > Make page ready for pure HTTPS > -- > > Key: LUCENE-9227 > URL: https://issues.apache.org/jira/browse/LUCENE-9227 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Uwe Schindler >Assignee: Uwe Schindler >Priority: Blocker > > The web page can currently be visited using HTTPS but this brings warning: > - Both search providers create a form that passes USER ENTERED INPUT using no > encryption. This is not allowed due to GDPR. We have to fix this asap. It > looks like [~otis] search is working with HTTPS (if we change domain name), > but the Lucidworks does not > - There were some CSS files loaded with HTTP (fonts from Google - this was > fixed) > Once those 2 problems are fixed (I grepped for HTTP and still found many > links with HTTP, but looks like no images or scripts or css anymore), I'd > like to add a permanent redirect http://lucene.apache.org/ -> > https://lucene.apache.org to the htaccess template file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8987) Move Lucene web site from svn to git
[
https://issues.apache.org/jira/browse/LUCENE-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045684#comment-17045684
]
Uwe Schindler commented on LUCENE-8987:
---
I tried to get hold (also together with INFRA) about some redirection issues
but it was not successful. We have to live with that: The way how the
Subversion stuff is included (internal rewrite) causes some strange redirects,
if user is missing the final slash "/" on a directory name:
https://lucene.apache.org/core/8_4_1/analyzers-nori (missing slash at end). The
webserver redirects this to to the "remapped" URL, causing the following URL to
be shown:
https://lucene.apache.org/__root/docs.lucene.apache.org/content/core/8_4_1/analyzers-nori/
(with slash, but also with the "__root/.../") in it.
This only happens with Javadocs and refguide. I only fixed it for the top-level
documentation folders, but with examples like shown before it leads to this
strange looking URL. The reason is the automatic redirect by Apache to the
folder with slash at end.
We should maybe only prevent Googlebot from crawling it, so we don't get those
wrong-looking URLs into the index.
> Move Lucene web site from svn to git
>
>
> Key: LUCENE-8987
> URL: https://issues.apache.org/jira/browse/LUCENE-8987
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-site-repo.png
>
>
> INFRA just enabled [a new way of configuring website
> build|https://s.apache.org/asfyaml] from a git branch, [see dev list
> email|https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E].
> It allows for automatic builds of both staging and production site, much
> like the old CMS. We can choose to auto publish the html content of an
> {{output/}} folder, or to have a bot build the site using
> [Pelican|https://github.com/getpelican/pelican] from a {{content/}} folder.
> The goal of this issue is to explore how this can be done for
> [http://lucene.apache.org|http://lucene.apache.org/] by, by creating a new
> git repo {{lucene-site}}, copy over the site from svn, see if it can be
> "Pelicanized" easily and then test staging. Benefits are that more people
> will be able to edit the web site and we can take PRs from the public (with
> GitHub preview of pages).
> Non-goals:
> * Create a new web site or a new graphic design
> * Change from Markdown to Asciidoc
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8987) Move Lucene web site from svn to git
[
https://issues.apache.org/jira/browse/LUCENE-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045687#comment-17045687
]
Uwe Schindler commented on LUCENE-8987:
---
One final solution to solve this is to add a redirect before the expansion is
taking place: If URL does not end with slash and the last part also has no dot
(".") in it, we redirect to same url with slash. This works because we have no
files without a dot in their name.
> Move Lucene web site from svn to git
>
>
> Key: LUCENE-8987
> URL: https://issues.apache.org/jira/browse/LUCENE-8987
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-site-repo.png
>
>
> INFRA just enabled [a new way of configuring website
> build|https://s.apache.org/asfyaml] from a git branch, [see dev list
> email|https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E].
> It allows for automatic builds of both staging and production site, much
> like the old CMS. We can choose to auto publish the html content of an
> {{output/}} folder, or to have a bot build the site using
> [Pelican|https://github.com/getpelican/pelican] from a {{content/}} folder.
> The goal of this issue is to explore how this can be done for
> [http://lucene.apache.org|http://lucene.apache.org/] by, by creating a new
> git repo {{lucene-site}}, copy over the site from svn, see if it can be
> "Pelicanized" easily and then test staging. Benefits are that more people
> will be able to edit the web site and we can take PRs from the public (with
> GitHub preview of pages).
> Non-goals:
> * Create a new web site or a new graphic design
> * Change from Markdown to Asciidoc
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13965) Adding new functions to GraphHandler should be same as Streamhandler
[ https://issues.apache.org/jira/browse/SOLR-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045762#comment-17045762 ] ASF subversion and git services commented on SOLR-13965: Commit b4f3efbf94e20cf0687c5a8f7b255eb48c1d8e78 in lucene-solr's branch refs/heads/master from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b4f3efb ] SOLR-13965: In GraphHandler, support configuration and deprecate configuration. (Eric Pugh via Christine Poerschke) Closes #1033 pull request. > Adding new functions to GraphHandler should be same as Streamhandler > > > Key: SOLR-13965 > URL: https://issues.apache.org/jira/browse/SOLR-13965 > Project: Solr > Issue Type: Improvement > Components: streaming expressions >Affects Versions: 8.3 >Reporter: David Eric Pugh >Priority: Minor > Attachments: SOLR-13965.01.patch, SOLR-13965.02.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently you add new functions to GraphHandler differently than you do in > StreamHandler. We should have one way of extending the handlers that support > streaming expressions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14267) complete solrconfig.xml removal
[ https://issues.apache.org/jira/browse/SOLR-14267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045764#comment-17045764 ] ASF subversion and git services commented on SOLR-14267: Commit 449a7ef7b5f2671d3456162605431e1d8847c46a in lucene-solr's branch refs/heads/branch_8x from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=449a7ef ] SOLR-14267: complete solrconfig.xml removal > complete solrconfig.xml removal > --- > > Key: SOLR-14267 > URL: https://issues.apache.org/jira/browse/SOLR-14267 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-14267.patch > > > * The code already warns about it being deprecated: > ** > [https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/core/src/java/org/apache/solr/core/SolrConfig.java#L217-L218] > – and the element attributes are read but not used. > * There are no apparent documentation references. > * A number of test solrconfig.xml files still configure the value. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13965) Adding new functions to GraphHandler should be same as Streamhandler
[ https://issues.apache.org/jira/browse/SOLR-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045763#comment-17045763 ] ASF subversion and git services commented on SOLR-13965: Commit a5e372369cc280addb1b2b1057fa022b3acbc460 in lucene-solr's branch refs/heads/branch_8x from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a5e3723 ] SOLR-13965: In GraphHandler, support configuration and deprecate configuration. (Eric Pugh via Christine Poerschke) Closes #1033 pull request. > Adding new functions to GraphHandler should be same as Streamhandler > > > Key: SOLR-13965 > URL: https://issues.apache.org/jira/browse/SOLR-13965 > Project: Solr > Issue Type: Improvement > Components: streaming expressions >Affects Versions: 8.3 >Reporter: David Eric Pugh >Priority: Minor > Attachments: SOLR-13965.01.patch, SOLR-13965.02.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently you add new functions to GraphHandler differently than you do in > StreamHandler. We should have one way of extending the handlers that support > streaming expressions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14278) data loss during live shard split if leader dies
[ https://issues.apache.org/jira/browse/SOLR-14278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045774#comment-17045774 ] Yonik Seeley commented on SOLR-14278: - Unfortunately (or fortunately, depending on how you look at it), the vast majority of failures was due to a test bug. The test did not account for update request failures that actually made it into the index. I've updated the pull request with this fix, and looped overnight. There were still 2 failures that I haven't looked deeply into yet. > data loss during live shard split if leader dies > > > Key: SOLR-14278 > URL: https://issues.apache.org/jira/browse/SOLR-14278 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Yonik Seeley >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > While trying to develop better tests for shared storage (SOLR-13101), I ran > across another failure for normal replica types as well (one of the first > things I do when a test fails for shared storage is to try and validate that > normal NRT replicas succeed.) The PR I'll open has a test adapted from the > one in SOLR-13813 for master. > Scenario: > - indexing is happening during shard split > - leader is killed shortly after (before the split has finished) and never > brought back up > - there are often some missing documents at the end > While it's possible that the simulated killing of the node in the unit test > is imperfect, I haven't reproduced a failure if I comment out the split > command and just kill the leader. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13965) Adding new functions to GraphHandler should be same as Streamhandler
[ https://issues.apache.org/jira/browse/SOLR-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-13965: --- Fix Version/s: 8.5 master (9.0) Resolution: Fixed Status: Resolved (was: Patch Available) > Adding new functions to GraphHandler should be same as Streamhandler > > > Key: SOLR-13965 > URL: https://issues.apache.org/jira/browse/SOLR-13965 > Project: Solr > Issue Type: Improvement > Components: streaming expressions >Affects Versions: 8.3 >Reporter: David Eric Pugh >Priority: Minor > Fix For: master (9.0), 8.5 > > Attachments: SOLR-13965.01.patch, SOLR-13965.02.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently you add new functions to GraphHandler differently than you do in > StreamHandler. We should have one way of extending the handlers that support > streaming expressions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14267) complete solrconfig.xml removal
[ https://issues.apache.org/jira/browse/SOLR-14267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke resolved SOLR-14267. Fix Version/s: 8.5 master (9.0) Resolution: Fixed > complete solrconfig.xml removal > --- > > Key: SOLR-14267 > URL: https://issues.apache.org/jira/browse/SOLR-14267 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Fix For: master (9.0), 8.5 > > Attachments: SOLR-14267.patch > > > * The code already warns about it being deprecated: > ** > [https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/core/src/java/org/apache/solr/core/SolrConfig.java#L217-L218] > – and the element attributes are read but not used. > * There are no apparent documentation references. > * A number of test solrconfig.xml files still configure the value. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14267) complete solrconfig.xml removal
[
https://issues.apache.org/jira/browse/SOLR-14267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045778#comment-17045778
]
Christine Poerschke commented on SOLR-14267:
{quote}+1 yeah just remove it immediately (8x).
{quote}
Thanks for the extra opinion on this!
Commit above is for branch_8x branch and
[https://github.com/apache/lucene-solr/commit/fc24fa95063c368cc446441da56712b464a3706a]
is for master branch (not sure why the bot did not add it here this time).
> complete solrconfig.xml removal
> ---
>
> Key: SOLR-14267
> URL: https://issues.apache.org/jira/browse/SOLR-14267
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-14267.patch
>
>
> * The code already warns about it being deprecated:
> **
> [https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/core/src/java/org/apache/solr/core/SolrConfig.java#L217-L218]
> – and the element attributes are read but not used.
> * There are no apparent documentation references.
> * A number of test solrconfig.xml files still configure the value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-7908) fix-or-suppress 3 'Resource leak' warnings in (ReadersAnd|FrozenBuffered)Updates.java
[
https://issues.apache.org/jira/browse/LUCENE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke resolved LUCENE-7908.
-
Resolution: Not A Problem
bq. I think these warnings have been already fixed.
You're correct, thanks [~asalamon74]!
> fix-or-suppress 3 'Resource leak' warnings in
> (ReadersAnd|FrozenBuffered)Updates.java
> -
>
> Key: LUCENE-7908
> URL: https://issues.apache.org/jira/browse/LUCENE-7908
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Christine Poerschke
>Priority: Minor
>
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java#L846
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java#L186
> http://www.github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java#L144
> {code}
> [ecj-lint] 4. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java
> (at line 846)
> [ecj-lint] SegmentReader newReader = new SegmentReader(info, reader,
> liveDocs, info.info.maxDoc() - delCount);
> [ecj-lint] ^
> [ecj-lint] Resource leak: 'newReader' is never closed
> [ecj-lint] 2. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java
> (at line 144)
> [ecj-lint] RAMOutputStream out = new RAMOutputStream();
> [ecj-lint] ^^^
> [ecj-lint] Resource leak: 'out' is never closed
> [ecj-lint] 3. WARNING in
> /Users/cpoerschke/lucene-solr/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java
> (at line 186)
> [ecj-lint] RAMOutputStream out = new RAMOutputStream();
> [ecj-lint] ^^^
> [ecj-lint] Resource leak: 'out' is never closed
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13878) Update commons-fileupload to 1.4 to fix potential resource leak issue
[ https://issues.apache.org/jira/browse/SOLR-13878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045800#comment-17045800 ] Christine Poerschke commented on SOLR-13878: (incomplete) code link: https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/lucene/ivy-versions.properties#L60 > Update commons-fileupload to 1.4 to fix potential resource leak issue > - > > Key: SOLR-13878 > URL: https://issues.apache.org/jira/browse/SOLR-13878 > Project: Solr > Issue Type: Task > Components: Build >Affects Versions: 8.2 >Reporter: Charles Dumont >Priority: Minor > > commons-fileupload version 1.3 has a potential resource leak issue as > described here: https://issues.apache.org/jira/browse/FILEUPLOAD-250. solr > currently has a dependency on 1.3 and should update to 1.4 to avoid being > potentially impacted by this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9236) Having a modular Doc Values format
[ https://issues.apache.org/jira/browse/LUCENE-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045803#comment-17045803 ] David Smiley commented on LUCENE-9236: -- I think it's weird that DocValuesFormat is a format of formats whereas the others are for their singular purpose. No? bq. ... you *must* implement the 5 different functions ... Wouldn't throwing UnsupportedOperationException work? > Having a modular Doc Values format > -- > > Key: LUCENE-9236 > URL: https://issues.apache.org/jira/browse/LUCENE-9236 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: juan camilo rodriguez duran >Priority: Minor > Labels: docValues > > Today DocValues Consumer/Producer require override 5 different methods, even > if you only want to use one and given that one given field can only support > one doc values type at same time. > > In the attached PR I’ve implemented a new modular version of those classes > (consumer/producer) each one having a single responsibility and writing in > the same unique file. > This is mainly a refactor of the existing format opening the possibility to > override or implement the sub-format you need. > > I’ll do in 3 steps: > # Create a CompositeDocValuesFormat and moving the code of > Lucene80DocValuesFormat in separate classes, without modifying the inner > code. At same time I created a Lucene85CompositeDocValuesFormat based on > these changes. > # I’ll introduce some basic components for writing doc values in general > such as: > ## DocumentIdSetIterator Serializer: used in each type of field based on an > IndexedDISI. > ## Document Ordinals Serializer: Used in Sorted and SortedSet for > deduplicate values using a dictionary. > ## Document Boundaries Serializer (optional used only for multivalued > fields: SortedNumeric and SortedSet) > ## TermsEnum Serializer: useful to write and read the terms dictionary for > sorted and sorted set doc values. > # I’ll create the new Sub-DocValues format using the previous components. > > PR: [https://github.com/apache/lucene-solr/pull/1282] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9077) Gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045838#comment-17045838
]
David Smiley commented on LUCENE-9077:
--
I found it useful to modify the "toDir" sync task in
{{solr/packaging/build.gradle}} with the following addition:
{code}
preserve {
include 'server/solr/**'
}
{code}
This retains my existing state of collections 'n such in-place. Then I can do
iterative development and re-assemble and debug etc. I use an IntelliJ "JAR
Application" type of run configuration that's configured to run the Jetty
start.jar. First it runs the gradle task for solr's assemble.
Is this addition cool with others? If so, should I commit directly with some
existing JIRA number (which?) or shall I make a PR for this?
> Gradle build
>
>
> Key: LUCENE-9077
> URL: https://issues.apache.org/jira/browse/LUCENE-9077
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9077-javadoc-locale-en-US.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> This task focuses on providing gradle-based build equivalent for Lucene and
> Solr (on master branch). See notes below on why this respin is needed.
> The code lives on *gradle-master* branch. It is kept with sync with *master*.
> Try running the following to see an overview of helper guides concerning
> typical workflow, testing and ant-migration helpers:
> gradlew :help
> A list of items that needs to be added or requires work. If you'd like to
> work on any of these, please add your name to the list. Once you have a
> patch/ pull request let me (dweiss) know - I'll try to coordinate the merges.
> * (/) Apply forbiddenAPIs
> * (/) Generate hardware-aware gradle defaults for parallelism (count of
> workers and test JVMs).
> * (/) Fail the build if --tests filter is applied and no tests execute
> during the entire build (this allows for an empty set of filtered tests at
> single project level).
> * (/) Port other settings and randomizations from common-build.xml
> * (/) Configure security policy/ sandboxing for tests.
> * (/) test's console output on -Ptests.verbose=true
> * (/) add a :helpDeps explanation to how the dependency system works
> (palantir plugin, lockfile) and how to retrieve structured information about
> current dependencies of a given module (in a tree-like output).
> * (/) jar checksums, jar checksum computation and validation. This should be
> done without intermediate folders (directly on dependency sets).
> * (/) verify min. JVM version and exact gradle version on build startup to
> minimize odd build side-effects
> * (/) Repro-line for failed tests/ runs.
> * (/) add a top-level README note about building with gradle (and the
> required JVM).
> * (/) add an equivalent of 'validate-source-patterns'
> (check-source-patterns.groovy) to precommit.
> * (/) add an equivalent of 'rat-sources' to precommit.
> * (/) add an equivalent of 'check-example-lucene-match-version' (solr only)
> to precommit.
> * (/) javadoc compilation
> Hard-to-implement stuff already investigated:
> * (/) (done) -*Printing console output of failed tests.* There doesn't seem
> to be any way to do this in a reasonably efficient way. There are onOutput
> listeners but they're slow to operate and solr tests emit *tons* of output so
> it's an overkill.-
> * (!) (LUCENE-9120) *Tests working with security-debug logs or other
> JVM-early log output*. Gradle's test runner works by redirecting Java's
> stdout/ syserr so this just won't work. Perhaps we can spin the ant-based
> test runner for such corner-cases.
> Of lesser importance:
> * Add an equivalent of 'documentation-lint" to precommit.
> * (/) Do not require files to be committed before running precommit. (staged
> files are fine).
> * (/) add rendering of javadocs (gradlew javadoc)
> * Attach javadocs to maven publications.
> * Add test 'beasting' (rerunning the same suite multiple times). I'm afraid
> it'll be difficult to run it sensibly because gradle doesn't offer cwd
> separation for the forked test runners.
> * if you diff solr packaged distribution against ant-created distribution
> there are minor differences in library versions and some JARs are excluded/
> moved around. I didn't try to force these as everything seems to work (tests,
> etc.) – perhaps these differences should be fixed in the ant build instead.
> * (/) identify and port various "regenerate" tasks from ant builds (javacc,
> precompiled automata, etc.)
> * Fill in POM details in gradle/defaults-maven.gradle so that they reflect
> the previous content better (dependencies aside).
> * Add any IDE integration layers that should be add
[jira] [Commented] (SOLR-13955) SHARED replicas can recover on clean disk
[
https://issues.apache.org/jira/browse/SOLR-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045849#comment-17045849
]
David Smiley commented on SOLR-13955:
-
Question: If a node comes up empty, then isn't it arbitrary what replicas get
assigned to it? Does the previous replica assignment to this node really
matter then? Also sometimes the node's identity may be different and thus it
will be impossible to know which replicas used to be there. I wonder if the
use-case addressed here could better be addressed by the auto-scaling framework
detecting a lack of replicas for a collection and recognizing an opportunity to
add them when the node is available (thus more capacity)?
Minor comment: The logging I see in the code added is using {{String.format(}}
style String interpolation. Solr mostly uses a nice SLF4J feature of `{}` and
substitution. Please use that in the future.
> SHARED replicas can recover on clean disk
> -
>
> Key: SOLR-13955
> URL: https://issues.apache.org/jira/browse/SOLR-13955
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Bilal Waheed
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> One of the benefits of SHARED replica is that it provides the ability to run
> SolrCloud with ephemeral disk(containers). Since the source of truth is in
> shared store, we can safely recover index from there. But currently there is
> no support to reason about the core descriptors that do not live locally on
> the disk. The purpose of this task is to discover missing core descriptors
> for SHARED replicas from ZK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-11508) Make coreRootDirectory configurable via an environment variable (SOLR_CORE_HOME)
[ https://issues.apache.org/jira/browse/SOLR-11508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-11508: Issue Type: Improvement (was: Bug) > Make coreRootDirectory configurable via an environment variable > (SOLR_CORE_HOME) > > > Key: SOLR-11508 > URL: https://issues.apache.org/jira/browse/SOLR-11508 > Project: Solr > Issue Type: Improvement >Reporter: Marc Morissette >Priority: Major > > (Heavily edited) > Since Solr 7, it is possible to store Solr cores in separate disk locations > using solr.data.home (see SOLR-6671). This is very useful when running Solr > in Docker where data must be stored in a directory which is independent from > the rest of the container. > While this works well in standalone mode, it doesn't in Cloud mode as the > core.properties automatically created by Solr are still stored in > coreRootDirectory and cores created that way disappear when the Solr Docker > container is redeployed. > The solution is to configure coreRootDirectory to an empty directory that can > be mounted outside the Docker container. > The incoming patch makes this easier to do by allowing coreRootDirectory to > be configured via a solr.core.home system property and SOLR_CORE_HOME > environment variable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13411) CompositeIdRouter calculates wrong route hash if atomic update is used for route.field
[ https://issues.apache.org/jira/browse/SOLR-13411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045857#comment-17045857 ] David Smiley commented on SOLR-13411: - +1 > CompositeIdRouter calculates wrong route hash if atomic update is used for > route.field > -- > > Key: SOLR-13411 > URL: https://issues.apache.org/jira/browse/SOLR-13411 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 7.5 >Reporter: Niko Himanen >Assignee: Mikhail Khludnev >Priority: Minor > Attachments: SOLR-13411.patch, SOLR-13411.patch > > > If collection is created with router.field -parameter to define some other > field than uniqueField as route field and document update comes containing > route field updated using atomic update syntax (for example set=123), hash > for document routing is calculated from "set=123" and not from 123 which is > the real value which may lead into routing document to wrong shard. > > This happens in CompositeIdRouter#sliceHash, where field value is used as is > for hash calculation. > > I think there are two possible solutions to fix this: > a) Allow use of atomic update also for route.field, but use real value > instead of atomic update syntax to route document into right shard. > b) Deny atomic update for route.field and throw exception. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14284) Document that you can add a new stream function via add-expressible
David Eric Pugh created SOLR-14284:
--
Summary: Document that you can add a new stream function via
add-expressible
Key: SOLR-14284
URL: https://issues.apache.org/jira/browse/SOLR-14284
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: documentation
Affects Versions: 8.5
Reporter: David Eric Pugh
I confirmed that in Solr 8.5 you will be able to dynamically add a Stream
function (assuming the Jar is in the path) via the configset api:
curl -X POST -H 'Content-type:application/json' -d '{
"add-expressible": {
"name": "dog",
"class": "org.apache.solr.handler.CatStream"
}
}' http://localhost:8983/solr/gettingstarted/config
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13996) Refactor HttpShardHandler#prepDistributed() into smaller pieces
[ https://issues.apache.org/jira/browse/SOLR-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045931#comment-17045931 ] David Smiley commented on SOLR-13996: - IMO we should use the word "Standalone" for standalone solr and not the word "Legacy", even if we imagine one day perhaps it might not be possible anymore. We use this word Standalone already for the distinction -- the "Standalone" subclass of ConfigSetService, for example, and SolrStandaloneScraper in the prometheus contrib. Of course "Legacy" appears in many class names but it merely means an old/deprecated way of doing something, not of Solr's standalone mode specifically. > Refactor HttpShardHandler#prepDistributed() into smaller pieces > --- > > Key: SOLR-13996 > URL: https://issues.apache.org/jira/browse/SOLR-13996 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Shalin Shekhar Mangar >Priority: Major > Attachments: SOLR-13996.patch, SOLR-13996.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, it is very hard to understand all the various things being done in > HttpShardHandler. I'm starting with refactoring the prepDistributed() method > to make it easier to grasp. It has standalone and cloud code intertwined, and > wanted to cleanly separate them out. Later, we can even have two separate > method (for standalone and cloud, each). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh opened a new pull request #1292: SOLR-14284 add expressible support to list, and add example of removing a component
epugh opened a new pull request #1292: SOLR-14284 add expressible support to list, and add example of removing a component URL: https://github.com/apache/lucene-solr/pull/1292 # Description Add expressible to the list of config supported methods. # Solution updated docs. # Tests Locally built ref guide. I manually tested the commands. # Checklist Please review the following and check all that apply: - [ X] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [ X] I have created a Jira issue and added the issue ID to my pull request title. - [ X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ X] I have developed this patch against the `master` branch. - [ ] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ X] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9077) Gradle build
[ https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045946#comment-17045946 ] Dawid Weiss commented on LUCENE-9077: - If you add such clauses then you run into the possibility of assembling and eventually releasing something that is local to your checkout... Just saying. I am not against if it makes life easier. > Gradle build > > > Key: LUCENE-9077 > URL: https://issues.apache.org/jira/browse/LUCENE-9077 > Project: Lucene - Core > Issue Type: Task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Major > Fix For: master (9.0) > > Attachments: LUCENE-9077-javadoc-locale-en-US.patch > > Time Spent: 2.5h > Remaining Estimate: 0h > > This task focuses on providing gradle-based build equivalent for Lucene and > Solr (on master branch). See notes below on why this respin is needed. > The code lives on *gradle-master* branch. It is kept with sync with *master*. > Try running the following to see an overview of helper guides concerning > typical workflow, testing and ant-migration helpers: > gradlew :help > A list of items that needs to be added or requires work. If you'd like to > work on any of these, please add your name to the list. Once you have a > patch/ pull request let me (dweiss) know - I'll try to coordinate the merges. > * (/) Apply forbiddenAPIs > * (/) Generate hardware-aware gradle defaults for parallelism (count of > workers and test JVMs). > * (/) Fail the build if --tests filter is applied and no tests execute > during the entire build (this allows for an empty set of filtered tests at > single project level). > * (/) Port other settings and randomizations from common-build.xml > * (/) Configure security policy/ sandboxing for tests. > * (/) test's console output on -Ptests.verbose=true > * (/) add a :helpDeps explanation to how the dependency system works > (palantir plugin, lockfile) and how to retrieve structured information about > current dependencies of a given module (in a tree-like output). > * (/) jar checksums, jar checksum computation and validation. This should be > done without intermediate folders (directly on dependency sets). > * (/) verify min. JVM version and exact gradle version on build startup to > minimize odd build side-effects > * (/) Repro-line for failed tests/ runs. > * (/) add a top-level README note about building with gradle (and the > required JVM). > * (/) add an equivalent of 'validate-source-patterns' > (check-source-patterns.groovy) to precommit. > * (/) add an equivalent of 'rat-sources' to precommit. > * (/) add an equivalent of 'check-example-lucene-match-version' (solr only) > to precommit. > * (/) javadoc compilation > Hard-to-implement stuff already investigated: > * (/) (done) -*Printing console output of failed tests.* There doesn't seem > to be any way to do this in a reasonably efficient way. There are onOutput > listeners but they're slow to operate and solr tests emit *tons* of output so > it's an overkill.- > * (!) (LUCENE-9120) *Tests working with security-debug logs or other > JVM-early log output*. Gradle's test runner works by redirecting Java's > stdout/ syserr so this just won't work. Perhaps we can spin the ant-based > test runner for such corner-cases. > Of lesser importance: > * Add an equivalent of 'documentation-lint" to precommit. > * (/) Do not require files to be committed before running precommit. (staged > files are fine). > * (/) add rendering of javadocs (gradlew javadoc) > * Attach javadocs to maven publications. > * Add test 'beasting' (rerunning the same suite multiple times). I'm afraid > it'll be difficult to run it sensibly because gradle doesn't offer cwd > separation for the forked test runners. > * if you diff solr packaged distribution against ant-created distribution > there are minor differences in library versions and some JARs are excluded/ > moved around. I didn't try to force these as everything seems to work (tests, > etc.) – perhaps these differences should be fixed in the ant build instead. > * (/) identify and port various "regenerate" tasks from ant builds (javacc, > precompiled automata, etc.) > * Fill in POM details in gradle/defaults-maven.gradle so that they reflect > the previous content better (dependencies aside). > * Add any IDE integration layers that should be added (I use IntelliJ and it > imports the project out of the box, without the need for any special tuning). > * Add Solr packaging for docs/* (see TODO in packaging/build.gradle; > currently XSLT...) > * I didn't bother adding Solr dist/test-framework to packaging (who'd use it > from a binary distribution? > * There is some python execution in check-broken-links and > check-missing-javadocs, not sure if it
[GitHub] [lucene-solr] janhoy commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp
janhoy commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp URL: https://github.com/apache/lucene-solr/pull/1288#discussion_r384806562 ## File path: solr/CHANGES.txt ## @@ -110,6 +110,9 @@ Improvements * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production (janhoy) +* SOLR-14281: Make sharedLib configurable through SysProp and allow multiple comma separated paths. Review comment: Re-phrased This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on a change in pull request #1288: SOLR-14281: Make sharedLib configurable through SysProp
janhoy commented on a change in pull request #1288: SOLR-14281: Make sharedLib
configurable through SysProp
URL: https://github.com/apache/lucene-solr/pull/1288#discussion_r384807112
##
File path: solr/core/src/java/org/apache/solr/core/CoreContainer.java
##
@@ -607,19 +609,31 @@ public PackageStoreAPI getPackageStoreAPI() {
public void load() {
log.debug("Loading cores into CoreContainer [instanceDir={}]",
loader.getInstancePath());
+// Always add $SOLR_HOME/lib to the shared resource loader
+Set libDirs = new HashSet<>();
+libDirs.add("lib");
+
+if (!StringUtils.isBlank(cfg.getSharedLibDirectory())) {
+ List sharedLibs =
Arrays.asList(cfg.getSharedLibDirectory().split("\\s*,\\s*"));
Review comment:
I tried `StrUtils.splitSmart()` but we don't need the smartness of not
splitting inside quotes etc, and it does not trim whitespace around comma. So
I'll keep this unless you know about a better split-util somewhere.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384809872
##
File path: solr/core/src/test/org/apache/solr/core/TestConfigSets.java
##
@@ -73,11 +73,11 @@ public void testConfigSetServiceFindsConfigSets() {
CoreContainer container = null;
try {
container = setupContainer(TEST_PATH().resolve("configsets").toString());
- Path testDirectory = container.getResourceLoader().getInstancePath();
+ Path solrHome = container.solrHome;
Review comment:
For consistency, prefer `getSolrHome()`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384806326
##
File path: solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
##
@@ -128,19 +129,23 @@ public static NodeConfig fromFile(SolrResourceLoader
loader, Path configFile, Pr
}
/** TEST-ONLY */
- public static NodeConfig fromString(SolrResourceLoader loader, String xml) {
-return fromInputStream(loader, new
ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)), new Properties());
+ public static NodeConfig fromString(Path solrHome, String xml) {
+return fromInputStream(
+solrHome,
+new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)),
+new Properties());
}
- public static NodeConfig fromInputStream(SolrResourceLoader loader,
InputStream is, Properties substituteProps) {
+ public static NodeConfig fromInputStream(Path solrHome, InputStream is,
Properties substituteProps) {
+SolrResourceLoader loader = new SolrResourceLoader(solrHome);
Review comment:
Does this need to be closed?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384802588
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -201,24 +198,19 @@ void addToClassLoader(List urls) {
.collect(Collectors.toList()));
}
- /**
- * Adds URLs to the ResourceLoader's internal classloader. This method
MUST
- * only be called prior to using this ResourceLoader to get any resources,
otherwise
- * its behavior will be non-deterministic. You also have to {link
@reloadLuceneSPI}
- * before using this ResourceLoader.
- *
- * @param urlsthe URLs of files to add
- */
- void addToClassLoader(URL... urls) {
-addToClassLoader(Arrays.asList(urls));
- }
-
/**
* Reloads all Lucene SPI implementations using the new classloader.
* This method must be called after {@link #addToClassLoader(List)}
* and before using this ResourceLoader.
*/
void reloadLuceneSPI() {
+// TODO improve to use a static Set to check when we need to
+if (!needToReloadLuceneSPI) {
Review comment:
is this thread safe? worst case we end up reloading more often than
necessary, which isn't a big deal, right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384812253
##
File path: solr/core/src/java/org/apache/solr/core/SolrPaths.java
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.core;
+
+import javax.naming.Context;
+import javax.naming.InitialContext;
+import javax.naming.NamingException;
+import javax.naming.NoInitialContextException;
+import java.io.File;
+import java.lang.invoke.MethodHandles;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Set;
+import java.util.concurrent.ConcurrentSkipListSet;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility methods about about paths in Solr.
+ */
+public class SolrPaths {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * Solr allows users to store arbitrary files in a special directory located
directly under SOLR_HOME.
+ *
+ * This directory is generally created by each node on startup. Files
located in this directory can then be
+ * manipulated using select Solr features (e.g. streaming expressions).
+ */
+ public static final String USER_FILES_DIRECTORY = "userfiles";
+ private static final Set loggedOnce = new ConcurrentSkipListSet<>();
+
+ private SolrPaths() {} // don't create this
+
+ /**
+ * Finds the solrhome based on looking up the value in one of three places:
+ *
+ * JNDI: via java:comp/env/solr/home
+ * The system property solr.solr.home
+ * Look in the current working directory for a solr/ directory
+ *
+ *
+ * The return value is normalized. Normalization essentially means it ends
in a trailing slash.
+ *
+ * @return A normalized solrhome
+ * @see #normalizeDir(String)
+ */
+ public static Path locateSolrHome() {
Review comment:
I'm assuming this is a verbatim copy from the old code, so I'm not looking
at it too closely. Let me know if that's not the case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384808059
##
File path: solr/core/src/test/org/apache/solr/core/TestSolrXml.java
##
@@ -118,16 +110,16 @@ public void testPropertySub() throws IOException {
System.setProperty("socketTimeout", "220");
System.setProperty("connTimeout", "200");
-File testSrcRoot = new File(SolrTestCaseJ4.TEST_HOME());
-FileUtils.copyFile(new File(testSrcRoot, "solr-50-all.xml"), new
File(solrHome, "solr.xml"));
+Path testSrcRoot = Paths.get(SolrTestCaseJ4.TEST_HOME());
Review comment:
Should this be TEST_PATH like above?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384803099
##
File path: solr/core/src/test/org/apache/solr/core/ResourceLoaderTest.java
##
@@ -169,12 +169,13 @@ public void testClassLoaderLibs() throws Exception {
}
SolrResourceLoader loader = new SolrResourceLoader(tmpRoot);
+loader.addToClassLoader(SolrResourceLoader.getURLs(lib));
-// ./lib is accessible by default
+// check "lib/aLibFile"
assertNotNull(loader.getClassLoader().getResource("aLibFile"));
// add inidividual jars from other paths
-loader.addToClassLoader(otherLib.resolve("jar2.jar").toUri().toURL());
+
loader.addToClassLoader(Arrays.asList(otherLib.resolve("jar2.jar").toUri().toURL()));
Review comment:
nit: Collections.singletonList is a tiny bit more readable IMO
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384810901 ## File path: solr/core/src/java/org/apache/solr/core/SolrPaths.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.solr.core; + +import javax.naming.Context; +import javax.naming.InitialContext; +import javax.naming.NamingException; +import javax.naming.NoInitialContextException; +import java.io.File; +import java.lang.invoke.MethodHandles; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Set; +import java.util.concurrent.ConcurrentSkipListSet; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility methods about about paths in Solr. Review comment: repeated word: "about about" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384811052
##
File path: solr/core/src/java/org/apache/solr/core/SolrPaths.java
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.core;
+
+import javax.naming.Context;
+import javax.naming.InitialContext;
+import javax.naming.NamingException;
+import javax.naming.NoInitialContextException;
+import java.io.File;
+import java.lang.invoke.MethodHandles;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Set;
+import java.util.concurrent.ConcurrentSkipListSet;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility methods about about paths in Solr.
+ */
+public class SolrPaths {
Review comment:
class can be final?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384804436
##
File path: solr/core/src/java/org/apache/solr/core/SolrCore.java
##
@@ -909,15 +907,9 @@ public CoreContainer getCoreContainer() {
/**
* Creates a new core and register it in the list of cores. If a core with
the
* same name already exists, it will be stopped and replaced by this one.
- *
- * @param dataDir the index directory
- * @param config a solr config instance
- * @param schema a solr schema instance
- * @since solr 1.3
*/
- public SolrCore(CoreContainer coreContainer, String name, String dataDir,
SolrConfig config,
- IndexSchema schema, NamedList configSetProperties,
- CoreDescriptor coreDescriptor, UpdateHandler updateHandler,
+ private SolrCore(CoreContainer coreContainer, String name, ConfigSet
configSet, CoreDescriptor coreDescriptor,
Review comment:
For this one and the other constructors that we changed, can we improve the
javadoc to note to the caller that configSet will be used to create the
resource loader? That might be too technical for a general audience, but it
also seems like something that might cause a debugging headache down the line.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on issue #1191: SOLR-14197 Reduce API of SolrResourceLoader
madrob commented on issue #1191: SOLR-14197 Reduce API of SolrResourceLoader URL: https://github.com/apache/lucene-solr/pull/1191#issuecomment-591684683 Also, general comment that the scope of each commit was just about perfect and I really appreciated that as a reviewer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] andyvuong opened a new pull request #1293: SOLR-14044: Delete collection bug fix by changing sharedShardName to use the same blob delimiter
andyvuong opened a new pull request #1293: SOLR-14044: Delete collection bug fix by changing sharedShardName to use the same blob delimiter URL: https://github.com/apache/lucene-solr/pull/1293 In shared storage, deletion from the blob store works by listing files beginning with a certain prefix and reading from a blob store requires knowing some or all of the file name. The sharedShardName, or the identifier prefixing all blob storage files per shard works by concatenating the collection name (uniqueness assumption in solr cloud overall) and the shard name with "_" joining the two. "_" is allowed in collection names which allows for a bug in listing files from other collections unintentionally. This change changes the delimiter to use "/" which is more consistent with how index files stored in blob are structured anyway /. Piggy-backed a minor unrelated test fix in DistributedZkUpdateProcessorTest.java that I missed earlier. Change Summary - Use "/" in sharedShardName - Update tests - Fix DistributedZkUpdateProcessorTest which tries to delete all collections after every test. In the same cleanup method the entire tmp directory we use as a local blob store gets deleted at the same time so the collection deletion errors out and leaks threads. This is a test setup issue vs a functional one. We spin up a new cluster in the different test cases anyway so we don't need to delete all collections. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14285) Trusted configset limitations should be relaxed in some circumstances
Cassandra Targett created SOLR-14285:
Summary: Trusted configset limitations should be relaxed in some
circumstances
Key: SOLR-14285
URL: https://issues.apache.org/jira/browse/SOLR-14285
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: configset-api
Affects Versions: 8.4.1, 8.4
Reporter: Cassandra Targett
The changes in SOLR-6736 mean that as of 8.4, a configset that includes
directives will only work if Solr has authentication configured.
I think we should be able to disable this (on by default) - in dev
environments, setting up authentication may be overly onerous and an obstacle
to getting started. These environments can already be well-protected by
internal firewalls if Solr does not yet need to be accessed.
Another way that change complicates things is on upgrades. Having your entire
Solr break only because you have not enabled authc (and maybe you don't intend
to) and used the default configset is a really poor user experience.
Similarly, the REINDEXCOLLECTION command copies an existing configset for the
new collection that it creates while reindexing the documents from a source
collection. When {{async=false}}, any errors are swallowed making it difficult
to know this restriction is the cause. I think it would be fair for a user to
assume that because the configset was already in use that copying it to a new
collection should be OK (more of an internal thing than a new upload).
Maybe some of this is just a problem for users trying to go from 8.3 to 8.4,
but it's creating a rather messy & painful upgrade experience. I still think it
would be worth being able to disable the check with a startup param (something
like {{-Ddisable.lib.restriction=true}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384851434
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -201,24 +198,19 @@ void addToClassLoader(List urls) {
.collect(Collectors.toList()));
}
- /**
- * Adds URLs to the ResourceLoader's internal classloader. This method
MUST
- * only be called prior to using this ResourceLoader to get any resources,
otherwise
- * its behavior will be non-deterministic. You also have to {link
@reloadLuceneSPI}
- * before using this ResourceLoader.
- *
- * @param urlsthe URLs of files to add
- */
- void addToClassLoader(URL... urls) {
-addToClassLoader(Arrays.asList(urls));
- }
-
/**
* Reloads all Lucene SPI implementations using the new classloader.
* This method must be called after {@link #addToClassLoader(List)}
* and before using this ResourceLoader.
*/
void reloadLuceneSPI() {
+// TODO improve to use a static Set to check when we need to
+if (!needToReloadLuceneSPI) {
Review comment:
Hey good point; needToReloadLuceneSPI should be volatile and
reloadLuceneSPI() should be synchronized. In practice it'd only happen if
someone had some strange Solr hacks / plugins because the only callers of the
couple methods that touch this boolean do so once and then never touch it
again. But still, lets be thorough.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384869359
##
File path: solr/core/src/java/org/apache/solr/core/SolrCore.java
##
@@ -909,15 +907,9 @@ public CoreContainer getCoreContainer() {
/**
* Creates a new core and register it in the list of cores. If a core with
the
* same name already exists, it will be stopped and replaced by this one.
- *
- * @param dataDir the index directory
- * @param config a solr config instance
- * @param schema a solr schema instance
- * @since solr 1.3
*/
- public SolrCore(CoreContainer coreContainer, String name, String dataDir,
SolrConfig config,
- IndexSchema schema, NamedList configSetProperties,
- CoreDescriptor coreDescriptor, UpdateHandler updateHandler,
+ private SolrCore(CoreContainer coreContainer, String name, ConfigSet
configSet, CoreDescriptor coreDescriptor,
Review comment:
I dunno... it's just one detail of a great many details that might be said.
And when you document something you run the risk of reality changing from what
you documented and so need to keep in sync. _Shrug._
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384911169
##
File path: solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
##
@@ -128,19 +129,23 @@ public static NodeConfig fromFile(SolrResourceLoader
loader, Path configFile, Pr
}
/** TEST-ONLY */
- public static NodeConfig fromString(SolrResourceLoader loader, String xml) {
-return fromInputStream(loader, new
ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)), new Properties());
+ public static NodeConfig fromString(Path solrHome, String xml) {
+return fromInputStream(
+solrHome,
+new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)),
+new Properties());
}
- public static NodeConfig fromInputStream(SolrResourceLoader loader,
InputStream is, Properties substituteProps) {
+ public static NodeConfig fromInputStream(Path solrHome, InputStream is,
Properties substituteProps) {
+SolrResourceLoader loader = new SolrResourceLoader(solrHome);
Review comment:
I looked into this a bit tonight; I didn't consider the close-ability of
SRLs before. Apparently it's because of the embedded URLClassLoader which
_ought to_ be closed. This was discussed when it was made closeable in
https://issues.apache.org/jira/browse/SOLR-4791 and it seems to only affect
Windows practically speaking. The CoreContainer.shutdown method will close
it's resourceLoader which is fetched from this SolrXmlConfig. That's the
particular instance you comment on here. But I don't see that a Core's SRL
will get closed. Shrug; I dunno if I should try to address that in this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on a change in pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on a change in pull request #1214: LUCENE-9074: Slice
Allocation Control Plane For Concurrent Searches
URL: https://github.com/apache/lucene-solr/pull/1214#discussion_r384911678
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -210,12 +216,27 @@ public IndexSearcher(IndexReader r, Executor executor) {
public IndexSearcher(IndexReaderContext context, Executor executor) {
assert context.isTopLevel: "IndexSearcher's ReaderContext must be topLevel
for reader" + context.reader();
reader = context.reader();
-this.executor = executor;
+this.sliceExecutionControlPlane = getSliceExecutionControlPlane(executor);
this.readerContext = context;
leafContexts = context.leaves();
this.leafSlices = executor == null ? null : slices(leafContexts);
}
+ /**
+ * We do this elaborate dance as to have only one constructor with a
nullable second parameter
+ * See the next constructor for more clarification
+ * Only for testing
+ */
+ IndexSearcher(IndexReaderContext context, Executor executor,
Review comment:
It is needed since executor is a final member of IndexSearcher hence needs
to be initialized? We could potentially always set it to null since it anyways
wont be used but that seemed a bit counter intuitive. WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on a change in pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on a change in pull request #1214: LUCENE-9074: Slice
Allocation Control Plane For Concurrent Searches
URL: https://github.com/apache/lucene-solr/pull/1214#discussion_r384911779
##
File path:
lucene/core/src/java/org/apache/lucene/search/SliceExecutionControlPlane.java
##
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.search;
+
+import java.util.Collection;
+
+
+/**
+ * Execution control plane which is responsible
+ * for execution of slices based on the current status
+ * of the system and current system load
+ */
+public interface SliceExecutionControlPlane {
+ /**
+ * Invoke all slices that are allocated for the query
+ */
+ C invokeAll(Collection tasks);
+}
Review comment:
Not sure about that. What would you advice it look like?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of SolrResourceLoader
dsmiley commented on a change in pull request #1191: SOLR-14197 Reduce API of
SolrResourceLoader
URL: https://github.com/apache/lucene-solr/pull/1191#discussion_r384912246
##
File path: solr/core/src/test/org/apache/solr/core/TestSolrXml.java
##
@@ -118,16 +110,16 @@ public void testPropertySub() throws IOException {
System.setProperty("socketTimeout", "220");
System.setProperty("connTimeout", "200");
-File testSrcRoot = new File(SolrTestCaseJ4.TEST_HOME());
-FileUtils.copyFile(new File(testSrcRoot, "solr-50-all.xml"), new
File(solrHome, "solr.xml"));
+Path testSrcRoot = Paths.get(SolrTestCaseJ4.TEST_HOME());
Review comment:
Indeed; a nice consistency improvement.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris opened a new pull request #1294: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris opened a new pull request #1294: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches URL: https://github.com/apache/lucene-solr/pull/1294 This commit introduces a mechanism to control allocation of threads to slices planned for a query. The default implementation uses the size of backlog queue of the executor to determine if a slice should be allocated a new thread Supersedes #1214 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on a change in pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on a change in pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches URL: https://github.com/apache/lucene-solr/pull/1214#discussion_r384913995 ## File path: lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java ## @@ -113,38 +139,10 @@ import org.junit.ClassRule; import org.junit.Rule; import org.junit.Test; +import org.junit.internal.AssumptionViolatedException; import org.junit.rules.RuleChain; import org.junit.rules.TestRule; import org.junit.runner.RunWith; -import org.junit.internal.AssumptionViolatedException; - -import com.carrotsearch.randomizedtesting.JUnit4MethodProvider; -import com.carrotsearch.randomizedtesting.LifecycleScope; -import com.carrotsearch.randomizedtesting.MixWithSuiteName; -import com.carrotsearch.randomizedtesting.RandomizedContext; -import com.carrotsearch.randomizedtesting.RandomizedRunner; -import com.carrotsearch.randomizedtesting.RandomizedTest; -import com.carrotsearch.randomizedtesting.annotations.Listeners; -import com.carrotsearch.randomizedtesting.annotations.SeedDecorators; -import com.carrotsearch.randomizedtesting.annotations.TestGroup; -import com.carrotsearch.randomizedtesting.annotations.TestMethodProviders; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakFilters; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakGroup.Group; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakGroup; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakLingering; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope.Scope; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakZombies.Consequence; -import com.carrotsearch.randomizedtesting.annotations.ThreadLeakZombies; -import com.carrotsearch.randomizedtesting.annotations.TimeoutSuite; -import com.carrotsearch.randomizedtesting.generators.RandomPicks; -import com.carrotsearch.randomizedtesting.rules.NoClassHooksShadowingRule; -import com.carrotsearch.randomizedtesting.rules.NoInstanceHooksOverridesRule; -import com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule; - -import junit.framework.AssertionFailedError; Review comment: Yeah, seems like the original strategy caused a lot of changes to files inadvertently. I have taken a clean slate approach with minimal code changes and opened a new PR which addresses your comments. Sorry about that, but refactoring the original changes to this PR seemed riskier than raising a new PR with only the necessary changes. The new PR is #1294 . Lets continue the discussion there? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on issue #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on issue #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches URL: https://github.com/apache/lucene-solr/pull/1214#issuecomment-591781782 Superseded by #1294 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris closed pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris closed pull request #1214: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches URL: https://github.com/apache/lucene-solr/pull/1214 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on a change in pull request #1294: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on a change in pull request #1294: LUCENE-9074: Slice
Allocation Control Plane For Concurrent Searches
URL: https://github.com/apache/lucene-solr/pull/1294#discussion_r384914799
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -211,6 +215,18 @@ public IndexSearcher(IndexReaderContext context, Executor
executor) {
assert context.isTopLevel: "IndexSearcher's ReaderContext must be topLevel
for reader" + context.reader();
reader = context.reader();
this.executor = executor;
+this.sliceExecutionControlPlane = executor == null ? null :
getSliceExecutionControlPlane(executor);
+this.readerContext = context;
+leafContexts = context.leaves();
+this.leafSlices = executor == null ? null : slices(leafContexts);
+ }
+
+ // Package private for testing
+ IndexSearcher(IndexReaderContext context, Executor executor,
SliceExecutionControlPlane sliceExecutionControlPlane) {
Review comment:
To quote @jpountz :
"this doesn't need an executor, does it?"
It is needed since executor is a final member of IndexSearcher hence needs
to be initialized? We could potentially always set it null since it anyways
wont be used but that seemed a bit counter intuitive. WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on a change in pull request #1294: LUCENE-9074: Slice Allocation Control Plane For Concurrent Searches
atris commented on a change in pull request #1294: LUCENE-9074: Slice
Allocation Control Plane For Concurrent Searches
URL: https://github.com/apache/lucene-solr/pull/1294#discussion_r384915037
##
File path:
lucene/core/src/java/org/apache/lucene/search/SliceExecutionControlPlane.java
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.search;
+
+import java.util.Collection;
+
+/**
+ * Execution control plane which is responsible
+ * for execution of slices based on the current status
+ * of the system and current system load
+ */
+public interface SliceExecutionControlPlane {
+ /**
+ * Invoke all slices that are allocated for the query
+ */
+ C invokeAll(Collection tasks);
Review comment:
To quote @jpountz
"the generics on this interface look over-engineered?"
I am not sure as to what the correct generics should be. Could you please
advice?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14270) export command to have an option to write to a zip file
[
https://issues.apache.org/jira/browse/SOLR-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046214#comment-17046214
]
ASF subversion and git services commented on SOLR-14270:
Commit edefcf156b03a612d66361738470470410a9767e in lucene-solr's branch
refs/heads/master from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=edefcf1 ]
SOLR-14270: fix incorrect date format
> export command to have an option to write to a zip file
> ---
>
> Key: SOLR-14270
> URL: https://issues.apache.org/jira/browse/SOLR-14270
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Labels: cli
> Fix For: 8.5
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Plain json files are too big. Export to a compressed file
> {{bin/solr export -url http://localhost:8983/solr/gettingstarted -out
> gettingstarted.json.gz}}
> This will write the data to a file called {{gettingstarted.json.gz}} in a zip
> format
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14270) export command to have an option to write to a zip file
[
https://issues.apache.org/jira/browse/SOLR-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046217#comment-17046217
]
ASF subversion and git services commented on SOLR-14270:
Commit 4e70711c95d50ee202c7b6f745af434e8f0a3dfe in lucene-solr's branch
refs/heads/branch_8x from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4e70711 ]
SOLR-14270: fix incorrect date format
> export command to have an option to write to a zip file
> ---
>
> Key: SOLR-14270
> URL: https://issues.apache.org/jira/browse/SOLR-14270
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Labels: cli
> Fix For: 8.5
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Plain json files are too big. Export to a compressed file
> {{bin/solr export -url http://localhost:8983/solr/gettingstarted -out
> gettingstarted.json.gz}}
> This will write the data to a file called {{gettingstarted.json.gz}} in a zip
> format
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
