[jira] [Created] (SOLR-14376) Optimize SolrIndexSearcher.getDocSet and getProcessedFilter for empty fq
David Smiley created SOLR-14376: --- Summary: Optimize SolrIndexSearcher.getDocSet and getProcessedFilter for empty fq Key: SOLR-14376 URL: https://issues.apache.org/jira/browse/SOLR-14376 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: search Reporter: David Smiley Assignee: David Smiley If either SolrIndexSearcher.getDocSet or getProcessedFilter are called with an null/empty query list, we should be able to short-circuit this with a getLiveDocSet response. Today getDocSet builds up a new DocSet, bit by bit :-/ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
dsmiley commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401318959
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
+} else {
+ qStr = macroExpander.expand(qStr);
+ if (qStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
+ }
+ scoreQuery = QParser.getParser(qStr, req).getQuery();
+ // note: QParser can return a null Query sometimes, such as if the
query is a stopword or just symbols
+ if (scoreQuery == null) {
+scoreQuery = new MatchNoDocsQuery(); // debatable; all or none?
+ }
+}
+
// Build the filter queries
-queryAndFilters = new ArrayList(); // If there are no fqs we
just want an empty list
-if (fqs != null) {
- for (String fq : fqs) {
-if ((fq != null) && (fq.trim().length() != 0)) {
- fq = macroExpander.expand(fq);
- if (fq == null) {
-throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
+Query filterDocSetQuery = null;
+if (fq != null) {
+ List filterQueries = new ArrayList<>(); // If there are no
fqs we just want an empty list
+ for (String fqStr : fq) {
+if (fqStr != null) {
+ fqStr = macroExpander.expand(fqStr);
+ if (fqStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
}
- final QParser fqp = QParser.getParser(fq, req);
- final Query filterQuery = fqp.getQuery();
+ final Query filterQuery = QParser.getParser(fqStr,
req).getQuery();
if (filterQuery != null) {
-queryAndFilters.add(filterQuery);
+filterQueries.add(filterQuery);
}
}
}
+
+ DocSet filtersDocSet = searcher.getDocSet(filterQueries); // execute
Review comment:
Good eye. It's possible filterQueries might be empty if say someone added a
blank fq. And I verified in a debugger get getDocSet does the right thing.
It's taking a long time to do its job however (accumulating a complete docset,
bit by bit). In another issue I'm targeting at 9.0, I optimized this case to a
live-docs DocSet, among other things relating to TwoPhaseIterator. I should
pull that out and make it its own improvement, along with a bit more
documentation to getDocSet to indicate that's what the semantics are if an
empty list is passed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail:
[GitHub] [lucene-solr] bringyou commented on issue #1389: LUCENE-9298: fix clearDeletedDocIds in BufferedUpdates
bringyou commented on issue #1389: LUCENE-9298: fix clearDeletedDocIds in BufferedUpdates URL: https://github.com/apache/lucene-solr/pull/1389#issuecomment-606987371 @s1monw sorry to bother, but can you take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
dsmiley commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401313568
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
+} else {
+ qStr = macroExpander.expand(qStr);
+ if (qStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
+ }
+ scoreQuery = QParser.getParser(qStr, req).getQuery();
+ // note: QParser can return a null Query sometimes, such as if the
query is a stopword or just symbols
+ if (scoreQuery == null) {
+scoreQuery = new MatchNoDocsQuery(); // debatable; all or none?
+ }
+}
+
// Build the filter queries
-queryAndFilters = new ArrayList(); // If there are no fqs we
just want an empty list
-if (fqs != null) {
- for (String fq : fqs) {
-if ((fq != null) && (fq.trim().length() != 0)) {
Review comment:
It seemed like a needless check since a blank fq would likely get parsed to
null and we'd drop it then any way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
dsmiley commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401310167
##
File path:
solr/core/src/java/org/apache/solr/search/grouping/CommandHandler.java
##
@@ -228,12 +226,8 @@ private void searchWithTimeLimiter(Query query,
collector = MultiCollector.wrap(collector, hitCountCollector);
}
-if (filter.filter != null) {
- query = new BooleanQuery.Builder()
- .add(query, Occur.MUST)
- .add(filter.filter, Occur.FILTER)
- .build();
-}
+query = QueryUtils.combineQueryAndFilter(query, filter.filter);
Review comment:
I wouldn't define combineQueryAndFilter by who uses it to reverse engineer
how it might work; it has its own javadocs. It behaves the same no matter who
calls it. Do you think it's defaults are counterintuitive?
BTW I think it's a mistake that Solr's 'q' effectively defaults to semantics
of MatchNoDocsQuery.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072269#comment-17072269
]
Chris M. Hostetter commented on SOLR-13132:
---
{quote}The changes presented here now consist of: ...
{quote}
Yup yup ... having the jiras split out *definitely* makes it easier to review
them and understand what chagnes were made for what purposes.
I have been reviewing the "sweep" collection changes a bit more in depth
lately, but i've been distracted with a lot of other things, sorry about the
delay.
I'm still wrapping my head around a lot of it – I've mainly been focused on the
higher level changes (to the internal APIs/abstractions), and thinking about
what the corner cases are that might not be obvious in existing testing, and
wondering wether there may be different/better ways to accomplish the same
objective with a smaller "patch footprint" that affects less internal logic.
(i'm not saying there are, just trying to do due-dillagence).
I haven't really spent any time at all yet looking into the lower level changes
in {{FacetFieldProcessorByArrayDV}} and {{UnInvertedField}}, but I do have a
few questions / concerns that i'll go ahead and share now – note that I haven't
fully examinied / thought through all of these, they are starting points for
discussions, not requests that you make any sorts of specific changes...
Questions:
* the " {{// XX temporary!}} " comment in {{SKGSlotAcc}} suggests that
{{SKGSlotAcc}} can/should be removed once {{SweepSKGSlotAcc}} is adopted ...
but {{SweepSKGSlotAcc}} doesn't appear to be a "universal" replacement for
{{SKGSlotAcc}}. {{RelatednessAgg}} uses hueristics + the
{{DISABLE_SWEEP_COLLECTION}} param to decide whne/what to use ... is this
comment old / a mistake?
* the introduction of the {{CountSlotAccFactory}} interface seems like dead
code? ... the only instance i see is {{DEFAULT_COUNT_ACC_FACTORY}} and even
that is never used directly, just as the implicit default for the 2 arg version
of {{getSweepCountAcc()}} to call the 3 arg version ... why not eliminate the 3
arg version?
* do {{ReadOnlyCountSlotAcc}} and {{ReadOnlyCountSlotAccWrapper}} really need
to exist as new classes? why can't we just introduce {{ReadOnlyCountSlotAcc}}
as an interface implemented by {{CountSlotAcc}} that only offers the (read
only) "getter" methods we want callers to have access to? (and then only expose
this interface in the "sweep data strcutures" that the Sweep Accs have access
to) Is there a specific reason for the "wrapper" pattern here that i'm not
understanding?
** the fact that the wrapper just seems to throw UnsupportedOperationException
on methods that aren't read-only suggests it's just ment to help us find any
'miss-use' type bugs during testing, why not weed those bugs out at compile
time?
* why is {{FilterCtStruct}} a distinct data structure from
{{SweepCountAccStruct}}? ... if the {{Filter}} in a {{FilterCtStruct}} _always_
comes from calling {{docSet.getTopFilter()}} then can't that just be exposed
via a new {{SweepCountAccStruct.getFilter()}} method?
Concerns:
* It feels like there is some "brittleness" (in terms of future
changes/improvements in JSON Faceting internals) related to when/how/where
Sweep can be used – in particularly because both the FacetProcessor and the
Aggregation have to _support_ sweeping in the first place, but they never
actually "communicate" that info directly. {{RelatednessAgg}} has it's own
hueristics/assumptions for deciding (based on reflection checks against the
Processor) if/when to return a SweepAcc – but if/when future Processors (or
riskier: future Processor subclasses w/new "optimizations") come around that
heuristic might break. Then the SweepAcc kind of "goes in the backdoor" of the
Processor to setup the {{CountSlotAcc}} / {{SweepCountAccStruct}} instances it
wants/needs – but there is no guarantee that the Processor will even look at
those.
** at the very least, I suppose we could make
{{FacetFieldProcessor.getSweepCountAcc()}} default to returning 'null', making
subclasses that support sweeping override it, and Aggs that want to leverage
sweeping would use 'null' to recognize that sweeping isn't supported?
** but meanwhile: we also have the (empty) {{SweepAcc}} marker interface that
Processors can use in instanceof checks to take conditional action, but this
also seems brittle in it's own way: notable the main (only?) use at the moment
is {{collectAcc instanceof SweepAcc}}, but this doesn't account for the
possibility that {{collectAcc = new MultiAcc(fcontext, accs)}} in the case of a
{{singlePassSlotAccCollection}} ... a situation where IIUC sweep collection
over _multiple_ {{relatedness()}} calls (nested under the same term facet)
would make a lot of sense? ... IIUC in the current code that posssibility might
be broken break? both {{RelatednessAggs}} would
[GitHub] [lucene-solr] MarcusSorealheis commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag
MarcusSorealheis commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag URL: https://github.com/apache/lucene-solr/pull/1391#discussion_r401246229 ## File path: solr/bin/solr ## @@ -2089,6 +2089,14 @@ else SECURITY_MANAGER_OPTS=() fi +# Enable ADMIN UI by default, and give the option for users to disable it when. Review comment: it's fixed. Psyched even myself out. I have also changed the names. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9266) ant nightly-smoke fails due to presence of build.gradle
[ https://issues.apache.org/jira/browse/LUCENE-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072197#comment-17072197 ] Mike Drob commented on LUCENE-9266: --- Follow on issues identified so far: 1) Dawid's idea to stamp the xsum and gw-version into the gradlew script during release build, and then ignore this during normal dev 2) Lucene's src-tar doesn't bundle gradlew, so it might not be able to build at all? 3) I don't think there is a gradle equivalent of nightly-smoke yet? Or any gradle commands that will build src release tarballs? > ant nightly-smoke fails due to presence of build.gradle > --- > > Key: LUCENE-9266 > URL: https://issues.apache.org/jira/browse/LUCENE-9266 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Mike Drob >Priority: Major > Time Spent: 5h 10m > Remaining Estimate: 0h > > Seen on Jenkins - > [https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1617/console] > > Reproduced locally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova commented on issue #1351: LUCENE-9280: Collectors to skip noncompetitive documents
mayya-sharipova commented on issue #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#issuecomment-606889736 @jpountz Thank you for the review. > I wonder whether we could make it easier to write implementations. I haven't spent much time thinking about it, but for instance would it be possible to wrap existing comparators to add the skipping functionality? Alternatively we could add the skipping logic to the existing comparators, but the fact that Lucene doesn't require that the same data be stored in indexes and doc values makes me a bit nervous about enabling it by default, and I'd like to avoid adding a new constructor argument. Would it make sense for each numeric FieldComparator to add an extra class that would wrap a numeric comparator and provide additional methods for skipping logic (getting an iterator and updating an iterator)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
mayya-sharipova commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r401228325 ## File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java ## @@ -93,4 +93,11 @@ */ void collect(int doc) throws IOException; + /* + * optionally returns an iterator over competitive documents Review comment: Thanks @jpountz > It's probably worth making explicit as null iterators are elsewhere interpreted as matching no documents What is the way to make this explicit? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
mayya-sharipova commented on a change in pull request #1351: LUCENE-9280:
Collectors to skip noncompetitive documents
URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r401227871
##
File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java
##
@@ -93,4 +93,11 @@
*/
void collect(int doc) throws IOException;
+ /*
+ * optionally returns an iterator over competitive documents
+ */
+ default DocIdSetIterator iterator() {
Review comment:
Would `competitiveIterator` make sense? If yes, addressed in 6c628f7
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
mayya-sharipova commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r401227632 ## File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java ## @@ -93,4 +93,11 @@ */ void collect(int doc) throws IOException; + /* Review comment: Addressed in 6c628f7 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072149#comment-17072149 ] Mike Drob commented on LUCENE-9170: --- Sure, mine is very similar to [~ichattopadhyaya]'s patch. > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: master (9.0), 8.6 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated LUCENE-9170: -- Attachment: LUCENE-9170.patch > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: master (9.0), 8.6 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated LUCENE-9170: -- Attachment: (was: LUCENE-9170.patch) > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: master (9.0), 8.6 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated LUCENE-9170: -- Attachment: LUCENE-9170.patch > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: master (9.0), 8.6 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14265) Move collections admin API to v2 completely
[ https://issues.apache.org/jira/browse/SOLR-14265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-14265: Summary: Move collections admin API to v2 completely (was: Move to admin API to v2 completely ) > Move collections admin API to v2 completely > > > Key: SOLR-14265 > URL: https://issues.apache.org/jira/browse/SOLR-14265 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Anshum Gupta >Assignee: Anshum Gupta >Priority: Major > > V2 admin API has been available in Solr for a very long time, making it > difficult for both users and developers to remember and understand which > format to use when. We should move to v2 API completely for all Solr Admin > calls for the following reasons: > # converge code - there are multiple ways of doing the same thing, there's > unwanted back-compat code, and we should get rid of that > # POJO all the way - no more NamedList. I know this would have split > opinions, but I strongly think we should move in this direction. I created > Jira about this specific task in the past and went half way but I think we > should just close this one out now. > # Automatic documentation > # Others > This is just an umbrella Jira for the task. Let's create sub-tasks and split > this up as it would require a bunch of rewriting of the code and it makes a > lot of sense to get this out with 9.0 so we don't have to support v1 forever! > There have been some conversations going on about this and it feels like most > folks are happy to go this route. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401172383
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
Review comment:
I think it should be exactly the same version as the one we "require". Same
as with JAR dependencies. If the checksum of the existing (or downloaded) file
doesn't match, we can issue a warning and overwrite it with a newly downloaded
one (after verifying its checksum). This gives us the liberty to update gradle
wrapper on different branches, for example: if you switch branches, the wrapper
would be re-downloaded and upgraded for you to match the branch's required
version.
I specifically don't *want* people to have the liberty of running arbitrary
gradle versions (or wrapper versions). This is where I got burned many times
before. If we are to drop versioned wrapper jar it has to be matching exactly
what is required by each git revision of the code (so that branch switching
works seamlessly and everyone is using the same version on the same git
revision).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401166452
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
Review comment:
If the wrapper is from a different version, then the checksum won't match.
And the jar itself isn't versioned. So there's a lot more to do here if we want
to always compute sums.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401165360
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
Review comment:
I'm afraid to ask what happened. :) I'd bump the limit to a reasonable 500K
just in case the wrapper grows and we forget about the check...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401164140
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
+
+File temp = Files.createTempFile("gradle-wrapper", null).toFile();
+try (ReadableByteChannel in =
Channels.newChannel(url.openStream());
+ FileOutputStream out = new FileOutputStream(temp)) {
+out.getChannel().transferFrom(in, 0, maxSize);
+}
+
+try (FileInputStream fis = new FileInputStream(temp)) {
+MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
+
+// Convert binary digest to hex checksum value
+// This will strip leading zeroes, deal with that when we need
to
+String sha256sum = new BigInteger(1,
sha256.digest(fis.readAllBytes())).toString(16);
+
+if (args[2].equals(sha256sum)) {
+if (!temp.renameTo(jar)) {
+System.err.println("Failed renaming wrapper jar,
leaving download at " + temp);
Review comment:
The user might be able to manually move it into place since we've already
validated the checksum. I'll update the error.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401163791
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
+
+File temp = Files.createTempFile("gradle-wrapper", null).toFile();
+try (ReadableByteChannel in =
Channels.newChannel(url.openStream());
+ FileOutputStream out = new FileOutputStream(temp)) {
+out.getChannel().transferFrom(in, 0, maxSize);
+}
+
+try (FileInputStream fis = new FileInputStream(temp)) {
+MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
+
+// Convert binary digest to hex checksum value
+// This will strip leading zeroes, deal with that when we need
to
+String sha256sum = new BigInteger(1,
sha256.digest(fis.readAllBytes())).toString(16);
+
+if (args[2].equals(sha256sum)) {
+if (!temp.renameTo(jar)) {
Review comment:
NIO's `Files.move` checks if it can rename and if not will do a copy+delete.
I'll switch to that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
madrob commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r401163433
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
Review comment:
I'd like to have the check, yes. Got burned by something like this once, and
never again :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13579) Create resource management API
[
https://issues.apache.org/jira/browse/SOLR-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072035#comment-17072035
]
Andrzej Bialecki commented on SOLR-13579:
-
As explained above, in theory CoreContainer-level components could be managed
by the framework but it would require adding quite a bit of scaffolding so the
framework knows how to manage a particular "type" of component - and all this
just to manage a single instance... so it wouldn't make much sense, it's best
to use it for situations where there are multiple components competing to use
limited resources.
OTOH, the {{ConcurrentMergeScheduler}} is a perfect candidate for using this
API, because it's instantiated at an IndexWriter level, there are multiple
instances of it across the cores, and they all compete for CPU and IO. This is
one of the use cases described in the SIP-4.
> Create resource management API
> --
>
> Key: SOLR-13579
> URL: https://issues.apache.org/jira/browse/SOLR-13579
> Project: Solr
> Issue Type: New Feature
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Attachments: SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch,
> SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch,
> SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Resource management framework API supporting the goals outlined in SOLR-13578.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9266) ant nightly-smoke fails due to presence of build.gradle
[ https://issues.apache.org/jira/browse/LUCENE-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072034#comment-17072034 ] Dawid Weiss commented on LUCENE-9266: - bq. I'm not confident that running the java download class only on src releases is simpler than running it out of our source checkout. It isn't simpler but it doesn't add additional overhead whereas a call to javac and java will be noticeable on each gradlew script invocation -- this is something I would like to avoid... bq. download the wrapper jar in exactly the version matching that of the git revision the release was created with When you're building a release package you know which exact version of gradle (and the wrapper) you need so you can precompute the checksum, download URL and put these in the build scripts (gradlew.*). I think it'd be best to explain what I have in mind by showing a patch rather than explaining... but It's a really busy time right now and I don't know when I'll have a minute to devote to it. I think what you did in the patch is great though. There are just small bits missing that would make it functional already - the download tool's hash validation so that it updates automatically on upgrades of the wrapper in the repository. > ant nightly-smoke fails due to presence of build.gradle > --- > > Key: LUCENE-9266 > URL: https://issues.apache.org/jira/browse/LUCENE-9266 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Mike Drob >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > Seen on Jenkins - > [https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1617/console] > > Reproduced locally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401059644
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
+} else {
+ qStr = macroExpander.expand(qStr);
+ if (qStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
+ }
+ scoreQuery = QParser.getParser(qStr, req).getQuery();
+ // note: QParser can return a null Query sometimes, such as if the
query is a stopword or just symbols
+ if (scoreQuery == null) {
+scoreQuery = new MatchNoDocsQuery(); // debatable; all or none?
Review comment:
Would 'all' be consistent with both prior q-is-missing code path using `*:*`
and the new `QueryUtils.combineQueryAndFilter` choosing all?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401065312
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -280,52 +243,10 @@ public float score() throws IOException {
@Override
public float getMaxScore(int upTo) throws IOException {
-return Float.POSITIVE_INFINITY;
- }
-}
-
-/**
- * An iterator that allows to iterate only on the documents for which a
feature has
- * a value.
- **/
-public class SolrFeatureScorerIterator extends DocIdSetIterator {
-
- final private DocIdSetIterator filterIterator;
- final private DocIdSetIterator scorerFilter;
-
- SolrFeatureScorerIterator(DocIdSetIterator filterIterator,
- DocIdSetIterator scorerFilter) {
-this.filterIterator = filterIterator;
-this.scorerFilter = scorerFilter;
- }
-
- @Override
- public int docID() {
-return filterIterator.docID();
- }
-
- @Override
- public int nextDoc() throws IOException {
-int docID = filterIterator.nextDoc();
-scorerFilter.advance(docID);
-return docID;
- }
-
- @Override
- public int advance(int target) throws IOException {
-// We use iterator to catch the scorer up since
-// that checks if the target id is in the query + all the filters
-int docID = filterIterator.advance(target);
-scorerFilter.advance(docID);
-return docID;
- }
-
- @Override
- public long cost() {
-return filterIterator.cost() + scorerFilter.cost();
+return solrScorer.getMaxScore(upTo);
}
+ // TODO delegate more methods?
Review comment:
Good question. I'll take a look to see if we could have some reflective unit
test coverage either way (similar to existing coverage elsewhere) to ensure
delegation for any methods that might be added in future.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401059139
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
Review comment:
observation: i like how use of `qStr` here hints towards String type which
is clearer to read than the prior `solrQuery` being String type but namewise
hinting towards Query type
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401067383
##
File path:
solr/core/src/java/org/apache/solr/search/grouping/CommandHandler.java
##
@@ -228,12 +226,8 @@ private void searchWithTimeLimiter(Query query,
collector = MultiCollector.wrap(collector, hitCountCollector);
}
-if (filter.filter != null) {
- query = new BooleanQuery.Builder()
- .add(query, Occur.MUST)
- .add(filter.filter, Occur.FILTER)
- .build();
-}
+query = QueryUtils.combineQueryAndFilter(query, filter.filter);
Review comment:
Thanks for the pointer, i see the `MatchNoDocsQuery` now in SolrFeature,
hadn't looked before.
Outside of `ltr/SolrFeature` though the `QueryUtils.combineQueryAndFilter`
is using `MatchAllDocsQuery`, right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401064123
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -209,49 +211,11 @@ private LocalSolrQueryRequest makeRequest(SolrCore core,
String solrQuery,
@Override
public FeatureScorer scorer(LeafReaderContext context) throws IOException {
- Scorer solrScorer = null;
- if (solrQueryWeight != null) {
-solrScorer = solrQueryWeight.scorer(context);
- }
-
- final DocIdSetIterator idItr = getDocIdSetIteratorFromQueries(
- queryAndFilters, context);
- if (idItr != null) {
-return solrScorer == null ? new ValueFeatureScorer(this, 1f, idItr)
-: new SolrFeatureScorer(this, solrScorer,
-new SolrFeatureScorerIterator(idItr, solrScorer.iterator()));
- } else {
Review comment:
Very nice how all this vanishes!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401061433
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
+} else {
+ qStr = macroExpander.expand(qStr);
+ if (qStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
+ }
+ scoreQuery = QParser.getParser(qStr, req).getQuery();
+ // note: QParser can return a null Query sometimes, such as if the
query is a stopword or just symbols
+ if (scoreQuery == null) {
+scoreQuery = new MatchNoDocsQuery(); // debatable; all or none?
+ }
+}
+
// Build the filter queries
-queryAndFilters = new ArrayList(); // If there are no fqs we
just want an empty list
-if (fqs != null) {
- for (String fq : fqs) {
-if ((fq != null) && (fq.trim().length() != 0)) {
Review comment:
observation: the `(fq.trim().length() != 0)` is being removed as part of the
refactor but that seems fair since blank `fq` or `fq` with trailing blanks
would seem unusual
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401063622
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
+} else {
+ qStr = macroExpander.expand(qStr);
+ if (qStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
+ }
+ scoreQuery = QParser.getParser(qStr, req).getQuery();
+ // note: QParser can return a null Query sometimes, such as if the
query is a stopword or just symbols
+ if (scoreQuery == null) {
+scoreQuery = new MatchNoDocsQuery(); // debatable; all or none?
+ }
+}
+
// Build the filter queries
-queryAndFilters = new ArrayList(); // If there are no fqs we
just want an empty list
-if (fqs != null) {
- for (String fq : fqs) {
-if ((fq != null) && (fq.trim().length() != 0)) {
- fq = macroExpander.expand(fq);
- if (fq == null) {
-throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
+Query filterDocSetQuery = null;
+if (fq != null) {
+ List filterQueries = new ArrayList<>(); // If there are no
fqs we just want an empty list
+ for (String fqStr : fq) {
+if (fqStr != null) {
+ fqStr = macroExpander.expand(fqStr);
+ if (fqStr == null) {
+throw new FeatureException(this.getClass().getSimpleName() + "
requires efi parameter that was not passed in request.");
}
- final QParser fqp = QParser.getParser(fq, req);
- final Query filterQuery = fqp.getQuery();
+ final Query filterQuery = QParser.getParser(fqStr,
req).getQuery();
if (filterQuery != null) {
-queryAndFilters.add(filterQuery);
+filterQueries.add(filterQuery);
}
}
}
+
+ DocSet filtersDocSet = searcher.getDocSet(filterQueries); // execute
Review comment:
observation: i've sort of convinced myself that at this point
`filterQueries` should be a non-empty list but if it were an empty list then i
haven't yet dived fully enough into the `getDocSet` implementation to
understand if that is doing what we want or if `filterDocSetQuery` remaining
`null` might be clearer. perhaps you've thought about this already too?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR SolrFeature fq improvements
cpoerschke commented on a change in pull request #1381: SOLR-14364: LTR
SolrFeature fq improvements
URL: https://github.com/apache/lucene-solr/pull/1381#discussion_r401058472
##
File path:
solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
##
@@ -120,67 +123,66 @@ protected void validate() throws FeatureException {
": Q or FQ must be provided");
}
}
+
/**
* Weight for a SolrFeature
**/
public class SolrFeatureWeight extends FeatureWeight {
-final private Weight solrQueryWeight;
-final private Query query;
-final private List queryAndFilters;
+private final Weight solrQueryWeight;
-public SolrFeatureWeight(IndexSearcher searcher,
-SolrQueryRequest request, Query originalQuery, Map
efi) throws IOException {
+public SolrFeatureWeight(SolrIndexSearcher searcher,
+ SolrQueryRequest request, Query originalQuery,
Map efi) throws IOException {
super(SolrFeature.this, searcher, request, originalQuery, efi);
try {
-String solrQuery = q;
-final List fqs = fq;
-
-if ((solrQuery == null) || solrQuery.isEmpty()) {
- solrQuery = "*:*";
-}
-
-solrQuery = macroExpander.expand(solrQuery);
-if (solrQuery == null) {
- throw new FeatureException(this.getClass().getSimpleName()+"
requires efi parameter that was not passed in request.");
-}
-
-final SolrQueryRequest req = makeRequest(request.getCore(), solrQuery,
-fqs, df);
+final SolrQueryRequest req = makeRequest(request.getCore(), q, fq, df);
if (req == null) {
throw new IOException("ERROR: No parameters provided");
}
+// Build the scoring query
+Query scoreQuery;
+String qStr = q;
+if (qStr == null || qStr.isEmpty()) {
+ scoreQuery = null; // ultimately behaves like MatchAllDocs
Review comment:
minor: s/MatchAllDocs/MatchAllDocsQuery
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071957#comment-17071957 ] Alan Woodward commented on LUCENE-9170: --- I'm hitting this in elasticsearch CI as well; [~mdrob] can you post a patch with your suggested fix? > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: master (9.0), 8.6 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9198) Remove news section from TLP website
[ https://issues.apache.org/jira/browse/LUCENE-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071943#comment-17071943 ] Uwe Schindler commented on LUCENE-9198: --- Thanks! > Remove news section from TLP website > > > Key: LUCENE-9198 > URL: https://issues.apache.org/jira/browse/LUCENE-9198 > Project: Lucene - Core > Issue Type: Improvement > Components: general/website >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Attachments: new-tlp-conditional-news.png, > new-tlp-frontpage-layout.png, new-tlp-frontpage-layout.png > > Time Spent: 20m > Remaining Estimate: 0h > > On the front page [https://lucene.apache.org|https://lucene.apache.org/] we > today show a list of TLP news. > For every release we author one news article for Solr, one news article for > LuceneCore, and one news article for TLP site, combining the two. > In all these years we have never published a news item to TLP that is not a > release announcement, except in 2014 when we announced that OpenRelevance sub > project closed. > I thus propose to remove this news section, and replace it with two widgets > that automatically display the last 5 news headings from LuceneCore, Solr and > PyLucene sub projects. > If we have an important TLP announcement to make at some point, that can be > done right there on the front page, not? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config
HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk
StatusHandler should know about dynamic zk config
URL: https://github.com/apache/lucene-solr/pull/1392#discussion_r401045606
##
File path: solr/solrj/src/java/org/apache/solr/common/cloud/SolrZkClient.java
##
@@ -818,6 +834,49 @@ public void downloadFromZK(String zkPath, Path dir)
throws IOException {
ZkMaintenanceUtils.downloadFromZK(this, zkPath, dir);
}
+ /**
+ * Represents one line in dynamic ZK config
+ */
+ public static class ZkConfigDyn {
+// server. = ::[:role];[:]
+public static Pattern linePattern =
Pattern.compile("server\\.(?\\d+) ?=
?(?[^:]+):(?\\d+):(?\\d+)(:(?.*?))?(;((?.*?):)?(?\\d+))?");
+public int serverId;
Review comment:
these can all likely be final correct?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config
HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk
StatusHandler should know about dynamic zk config
URL: https://github.com/apache/lucene-solr/pull/1392#discussion_r401042663
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/ZookeeperStatusHandler.java
##
@@ -108,27 +128,30 @@ public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throw
if ("true".equals(String.valueOf(stat.get("ok" {
numOk++;
}
-String state = String.valueOf(stat.get("zk_server_state"));
+String state = zk.role != null ? zk.role :
String.valueOf(stat.get("zk_server_state"));
if ("follower".equals(state)) {
followers++;
} else if ("leader".equals(state)) {
leaders++;
reportedFollowers =
Integer.parseInt(String.valueOf(stat.get("zk_followers")));
} else if ("standalone".equals(state)) {
standalone++;
+} else if ("participant".equals(state)) {
+ // NOCOMMIT: What does participant mean vs leader or follower?
Review comment:
Looking at the docs here
https://zookeeper.apache.org/doc/r3.5.7/zookeeperReconfig.html, in the
"Specifying the client port" section, it looks like the role of the zk instance
is either observer or participant (where participant is the default).
Participant isn't a new thing, just a name for a regular member of the ensemble
(not an observer).
This state check change may be making everything a "participant" as that's
the default role. Maybe keep the original state logic, but add in an additional
check in the beginning to see if the role is observer?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config
HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config URL: https://github.com/apache/lucene-solr/pull/1392#discussion_r400993317 ## File path: solr/core/src/java/org/apache/solr/handler/admin/ZookeeperStatusHandler.java ## @@ -24,15 +24,12 @@ import java.io.Writer; import java.lang.invoke.MethodHandles; import java.net.Socket; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Locale; -import java.util.Map; +import java.nio.charset.StandardCharsets; +import java.util.*; Review comment: The precommit may dislike this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config
HoustonPutman commented on a change in pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config URL: https://github.com/apache/lucene-solr/pull/1392#discussion_r401042877 ## File path: solr/core/src/test/org/apache/solr/handler/admin/ZookeeperStatusHandlerTest.java ## @@ -47,8 +48,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.*; Review comment: precommit issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9198) Remove news section from TLP website
[ https://issues.apache.org/jira/browse/LUCENE-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071905#comment-17071905 ] Alan Woodward commented on LUCENE-9198: --- I've removed the combined release messages > Remove news section from TLP website > > > Key: LUCENE-9198 > URL: https://issues.apache.org/jira/browse/LUCENE-9198 > Project: Lucene - Core > Issue Type: Improvement > Components: general/website >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Attachments: new-tlp-conditional-news.png, > new-tlp-frontpage-layout.png, new-tlp-frontpage-layout.png > > Time Spent: 20m > Remaining Estimate: 0h > > On the front page [https://lucene.apache.org|https://lucene.apache.org/] we > today show a list of TLP news. > For every release we author one news article for Solr, one news article for > LuceneCore, and one news article for TLP site, combining the two. > In all these years we have never published a news item to TLP that is not a > release announcement, except in 2014 when we announced that OpenRelevance sub > project closed. > I thus propose to remove this news section, and replace it with two widgets > that automatically display the last 5 news headings from LuceneCore, Solr and > PyLucene sub projects. > If we have an important TLP announcement to make at some point, that can be > done right there on the front page, not? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9266) ant nightly-smoke fails due to presence of build.gradle
[ https://issues.apache.org/jira/browse/LUCENE-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071889#comment-17071889 ] Mike Drob commented on LUCENE-9266: --- > but distribute source tarballs without it Where does the tarball assembly happen such that we can exclude the wrapper jar? > ant nightly-smoke fails due to presence of build.gradle > --- > > Key: LUCENE-9266 > URL: https://issues.apache.org/jira/browse/LUCENE-9266 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Mike Drob >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > Seen on Jenkins - > [https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1617/console] > > Reproduced locally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9266) ant nightly-smoke fails due to presence of build.gradle
[ https://issues.apache.org/jira/browse/LUCENE-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071887#comment-17071887 ] Mike Drob commented on LUCENE-9266: --- > Seems like we can keep it in the repository (for convenience) but distribute >source tarballs without it. I'm not confident that running the java download class only on src releases is simpler than running it out of our source checkout. At that point, the Groovy solution asking people to generate the wrapper jar locally seems minimally inconvenient, since presumably the only people we are inconveniencing is developers, not end users. > download the wrapper jar in exactly the version matching that of the git >revision the release was created with I'm having trouble parsing this. Can you give a specific example? I think my brain is getting stuck on the overly general stance here. > ant nightly-smoke fails due to presence of build.gradle > --- > > Key: LUCENE-9266 > URL: https://issues.apache.org/jira/browse/LUCENE-9266 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Mike Drob >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > Seen on Jenkins - > [https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1617/console] > > Reproduced locally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14273) Autoscaling "cores" tag should support "strict : false"
[ https://issues.apache.org/jira/browse/SOLR-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki resolved SOLR-14273. - Resolution: Later I'm resolving this as Later - perhaps one day we need an explicit option to allow for even more relaxed placement of EQUALS. > Autoscaling "cores" tag should support "strict : false" > --- > > Key: SOLR-14273 > URL: https://issues.apache.org/jira/browse/SOLR-14273 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: AutoScaling >Affects Versions: 8.4.1 >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Fix For: 7.7.3, 8.6 > > > Currently when the "cores" global tag is parsed the parser automatically > refuses to accept any other attributes - and since the "strict" attribute > defaults to true there's no way to relax this rule. > This becomes important when trying to spread the cores evenly across the > cluster - even minor variations will generate violations, while in most cases > a good enough spread is sufficient. > It appears there's no other reason for this limitation apart from the rule > parser limitation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14014) Allow Solr to start with Admin UI disabled
[
https://issues.apache.org/jira/browse/SOLR-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071852#comment-17071852
]
Jan Høydahl edited comment on SOLR-14014 at 3/31/20, 2:47 PM:
--
Remember that the Admin UI is full of inter-links to other Solr nodes, so if
you disable it on all nodes but one, you'll get 404's when you try to navigate
e.g. in the Graph screen or in the Nodes screen, and several other places.
I made a class {{AdminHandlersProxy}} to be able to pull responses from other
nodes through a single Solr node. It is currently used by Nodes screen to
request data from {{MetricsHandler}} and {{SystemInfoHandler}} for other nodes
by adding a {{=node1[,node2...]}} parameter to the request. Perhaps we
could expand on that concept and let any node be a proxy to get e.g.
/admin/metrics or /admin/info/system. Imagine a node-selector dropdown on the
"Dashboard" screen of Admin UI, where you can switch between all nodes, and the
dashboard will then show info for the selected node, and not for the node in
the URL. Then we won't need the inter-node links, we could just change local
Angular state of what node is "current". The "cores" dropdown could then also
change with to show cores from and interact with the "current" selected node.
was (Author: janhoy):
Remember that the Admin UI is full of inter-links to other Solr nodes, so if
you disable it on all nodes but one, you'll get 404's when you try to navigate
e.g. in the Graph screen or in the Nodes screen, and several other places.
I made a class {{AdminHandlersProxy}} to be able to pull responses from other
nodes through a single Solr node. Perhaps we could expand on that concept and
let any node be a proxy to get e.g. /admin/metrics or /admin/info/system.
Imagine a node-selector dropdown on the "Dashboard" screen of Admin UI, where
you can switch between all nodes, and the dashboard will then show info for the
selected node, and not for the node in the URL. Then we won't need the
inter-node links, we could just change local Angular state of what node is
"current". The "cores" dropdown could then also change with to show cores from
and interact with the "current" selected node.
> Allow Solr to start with Admin UI disabled
> --
>
> Key: SOLR-14014
> URL: https://issues.apache.org/jira/browse/SOLR-14014
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI, security
>Affects Versions: master (9.0), 8.3.1
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently Solr always runs the Admin UI. With the history of XSS issues and
> other security concerns that have been found in the Admin UI, Solr should
> offer a mode where the Admin UI is disabled. Maybe, and this is a topic
> that'll need some serious discussion, this should even be the default when
> Solr starts.
> NOTE: Disabling the Admin UI removes XSS and other attack vectors. But even
> with the Admin UI disabled, Solr will still be inherently unsafe without
> firewall protection on a public network.
> *Proposed design:*
> A java system property called *headless* will be used as an internal flag for
> starting Solr in headless mode. This property will default to true. A java
> property can be used at startup to set this flag to false.
> Here is an example:
> {code:java}
> bin/solr start -Dheadless=false {code}
> A message will be added following startup describing the mode.
> In headless mode the following message will be displayed:
> "solr is running in headless mode. The admin console is unavailable. To to
> turn off headless mode and allow the admin console use the following
> parameter startup parameter:
> -Dheadless=false
>
> In non-headless mode the following message will be displayed:
> "solr is running with headless mode turned off. The admin console is
> available in this mode. Disabling the Admin UI removes XSS and other attack
> vectors"
> If a user attempts to access the admin console while Solr is in headless mode
> it Solr will return 401 unauthorized.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14014) Allow Solr to start with Admin UI disabled
[
https://issues.apache.org/jira/browse/SOLR-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071852#comment-17071852
]
Jan Høydahl commented on SOLR-14014:
Remember that the Admin UI is full of inter-links to other Solr nodes, so if
you disable it on all nodes but one, you'll get 404's when you try to navigate
e.g. in the Graph screen or in the Nodes screen, and several other places.
I made a class {{AdminHandlersProxy}} to be able to pull responses from other
nodes through a single Solr node. Perhaps we could expand on that concept and
let any node be a proxy to get e.g. /admin/metrics or /admin/info/system.
Imagine a node-selector dropdown on the "Dashboard" screen of Admin UI, where
you can switch between all nodes, and the dashboard will then show info for the
selected node, and not for the node in the URL. Then we won't need the
inter-node links, we could just change local Angular state of what node is
"current". The "cores" dropdown could then also change with to show cores from
and interact with the "current" selected node.
> Allow Solr to start with Admin UI disabled
> --
>
> Key: SOLR-14014
> URL: https://issues.apache.org/jira/browse/SOLR-14014
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI, security
>Affects Versions: master (9.0), 8.3.1
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently Solr always runs the Admin UI. With the history of XSS issues and
> other security concerns that have been found in the Admin UI, Solr should
> offer a mode where the Admin UI is disabled. Maybe, and this is a topic
> that'll need some serious discussion, this should even be the default when
> Solr starts.
> NOTE: Disabling the Admin UI removes XSS and other attack vectors. But even
> with the Admin UI disabled, Solr will still be inherently unsafe without
> firewall protection on a public network.
> *Proposed design:*
> A java system property called *headless* will be used as an internal flag for
> starting Solr in headless mode. This property will default to true. A java
> property can be used at startup to set this flag to false.
> Here is an example:
> {code:java}
> bin/solr start -Dheadless=false {code}
> A message will be added following startup describing the mode.
> In headless mode the following message will be displayed:
> "solr is running in headless mode. The admin console is unavailable. To to
> turn off headless mode and allow the admin console use the following
> parameter startup parameter:
> -Dheadless=false
>
> In non-headless mode the following message will be displayed:
> "solr is running with headless mode turned off. The admin console is
> available in this mode. Disabling the Admin UI removes XSS and other attack
> vectors"
> If a user attempts to access the admin console while Solr is in headless mode
> it Solr will return 401 unauthorized.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11245) Cloud native Dockerfile
[ https://issues.apache.org/jira/browse/SOLR-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071846#comment-17071846 ] Jan Høydahl commented on SOLR-11245: Ok, I dumped a branch I had lying around into PR [#1393|https://github.com/apache/lucene-solr/pull/1393] > Cloud native Dockerfile > --- > > Key: SOLR-11245 > URL: https://issues.apache.org/jira/browse/SOLR-11245 > Project: Solr > Issue Type: Bug > Components: Build >Affects Versions: 6.6 >Reporter: jay vyas >Priority: Major > Labels: docker > Time Spent: 10m > Remaining Estimate: 0h > > SOLR Should have its own Dockerfile, ideally one that is cloud native (i.e. > doesn't expect anything special from the operating system in terms of user > IDs, etc), for deployment, that we can curate and submit changes to as part > of the official ASF process, rather then externally. The idea here is that > testing SOLR regression, as a microservice, is something we should be doing > as part of our continuous integration, rather then something done externally. > We have a team here that would be more then happy to do the work to port > whatever existing SOLR dockerfiles are out there into something that is ASF > maintainable, and cloud native, and easily testable, as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy opened a new pull request #1393: SOLR-11245 Cloud native Dockerfile
janhoy opened a new pull request #1393: SOLR-11245 Cloud native Dockerfile URL: https://github.com/apache/lucene-solr/pull/1393 See https://issues.apache.org/jira/browse/SOLR-11245 This is a fork of Solr's official Dockerfile, simplified for quickly building local image. Very much work in progress, it should probably be simplified even more, get rid of most of the scripts, be able to build from gradle's target folder instead of requiring a tgz etc. Just dumping a branch out there that I worked on several months ago, so it has deviated from docker-solr a bit. But it works :) @makuk66 @dsmiley This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[ https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071840#comment-17071840 ] Jan Høydahl commented on SOLR-14371: Ok, first PR draft is out https://github.com/apache/lucene-solr/pull/1392 Tries to unify the old parsing of ZK_HOST and parsing of dynamic config. Needs more testing. I'll try to test with the Solr Operator and see if we get info for all ZKs. Also needs some more work in UI for how to interpret leader vs follower vs participant vs observer as zk-node roles. Now we require exactly one leader and assume the rest are followers, but I think there are other allowed mixes too. > Zk StatusHandler should know about dynamic zk config > > > Key: SOLR-14371 > URL: https://issues.apache.org/jira/browse/SOLR-14371 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator > for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB > (Service) in front of all zookeepers, and the zkclient will then fetch list > of all zookeepers from special zknode /zookeeper/config and reconfigure > itself with connection to all zk nodes listed. So you can then scale up/down > number of zk nodes dynamically without restarting solr. > However, the Admin UI displays errors since it believes it is connected to > only one zk, which is contradictory to what zk itself reports. We need to > make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient > what current zkHost is instead of relying on Zk_HOST static setting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy opened a new pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config
janhoy opened a new pull request #1392: SOLR-14371 Zk StatusHandler should know about dynamic zk config URL: https://github.com/apache/lucene-solr/pull/1392 See https://issues.apache.org/jira/browse/SOLR-14371 Still draft This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9280) Add ability to skip non-competitive documents on field sort
[ https://issues.apache.org/jira/browse/LUCENE-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071835#comment-17071835 ] Adrien Grand commented on LUCENE-9280: -- I was just talking about it with Jim earlier today. I think this would make this change more complete indeed. > Add ability to skip non-competitive documents on field sort > > > Key: LUCENE-9280 > URL: https://issues.apache.org/jira/browse/LUCENE-9280 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mayya Sharipova >Priority: Minor > Time Spent: 7h 10m > Remaining Estimate: 0h > > Today collectors, once they collect enough docs, can instruct scorers to > update their iterators to skip non-competitive documents. This is applicable > only for a case when we need top docs by _score. > It would be nice to also have an ability to skip non-competitive docs when we > need top docs sorted by other fields different from _score. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-9575) Initialize an empty solr-home
[ https://issues.apache.org/jira/browse/SOLR-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071811#comment-17071811 ] Jan Høydahl commented on SOLR-9575: --- So, time passes. We've worked around this both in docker-solr and in the linux install script. But not in Solr itself. What are your thoughts now David? My preference is to make Solr happy even if there is no solr.xml in SOLR_HOME, let it use a built-in default. And for zoo.cfg, that is only for dev/test anyway, so if Solr is started with -DzkRun then perhaps it could put a default cfg file there or configure it in code for the embedded zk? > Initialize an empty solr-home > - > > Key: SOLR-9575 > URL: https://issues.apache.org/jira/browse/SOLR-9575 > Project: Solr > Issue Type: Improvement >Reporter: David Smiley >Priority: Major > Labels: docker > > The user may not want to use Solr's default solr-home dir location -- most > likely to use a separate disk. If you do this, there are two main problems: > * solr.xml & zoo.cfg aren't there > * configsets aren't there > Of course you could copy it manually but that's an extra step, and it's > particularly annoying to add this step to a Docker setup. Docker is all the > rage these days, and for good reason. If I mount a volume at > /opt/solr/server/solr then it basically masks this part of the built-in Solr > image (thus making configsets completely invisible) and points to some place > that will be empty. Solr obviously complains. I could set the solr-home to > some other path that I mount, but Solr would still complain about an empty > solr-home -- no solr.xml > If solr-home is empty, and if it's a dir other than the default solr-home, > then I think the solr-home should be initialized with solr.xml and zoo.cfg > copied from the default solr-home. I think configsets should be referenced > from the default solr-home if there is no configsets dir in solr-home. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9280) Add ability to skip non-competitive documents on field sort
[ https://issues.apache.org/jira/browse/LUCENE-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071799#comment-17071799 ] Mayya Sharipova commented on LUCENE-9280: - [~jpountz] Thanks Adrien for the explanation about index sorting. It makes sense I was just wondering about this sentence of yours: " it's common to sort data by descending timestamp, e.g. to live-tail events that match a filter, but you sometimes also need to sort in ascending order". I guess this PR is not going to help for this case either, as segments and docs within them are processed in order they are written (in desc order), and we will not reach bottom values until the very end. But I was wondering if as a follow-up work to address this case, we can imitate the work of [~jim.ferenczi] that during a query pre-sorts segments based on a query sort field, and process segments in that sort order. > Add ability to skip non-competitive documents on field sort > > > Key: LUCENE-9280 > URL: https://issues.apache.org/jira/browse/LUCENE-9280 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mayya Sharipova >Priority: Minor > Time Spent: 7h 10m > Remaining Estimate: 0h > > Today collectors, once they collect enough docs, can instruct scorers to > update their iterators to skip non-competitive documents. This is applicable > only for a case when we need top docs by _score. > It would be nice to also have an ability to skip non-competitive docs when we > need top docs sorted by other fields different from _score. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-7871) Platform independent config file instead of solr.in.sh and solr.in.cmd
[
https://issues.apache.org/jira/browse/SOLR-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071773#comment-17071773
]
Jan Høydahl commented on SOLR-7871:
---
Agree.
My vision is that once we get a solr.jar with a SolrBootstrap class, we can do
pretty much anything we want in pure Java wrt startup and parsing ENV vars and
JavaOpts.
What still needs to stay in bin/solr[.cmd] is parsing solr memory settings and
other things that MUST be passed to the java cmdline.
But I hope we'll be able to kill 90% of those scripts and instead create a new
bootstrap class that can parse cmdline args, SysProps and EnvVars as well as
parse solr.in.sh or its unified replacement, and make sense of it all. Perhaps
we can even introduce some conventions, that {{SOLR_FOO_BAR}} env.var gets
converted to {{-Dsolr.foo.bar}} SysProp without any boilerplate code, except
perhaps some validation layer for fail-fast?, and thus if some component
requires {{-Dsolr.my.setting=true}}, users can add {{SOLR_MY_SETTING=true}} to
the property/yaml file instead of current practice {{SOLR_OPTS=$SOLR_OPTS
-Dsolr.my.setting=true}}.
> Platform independent config file instead of solr.in.sh and solr.in.cmd
> --
>
> Key: SOLR-7871
> URL: https://issues.apache.org/jira/browse/SOLR-7871
> Project: Solr
> Issue Type: Improvement
> Components: scripts and tools
>Affects Versions: 5.2.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Labels: bin/solr
> Attachments: SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch
>
>
> Spinoff from SOLR-7043
> The config files {{solr.in.sh}} and {{solr.in.cmd}} are currently executable
> batch files, but all they do is to set environment variables for the start
> scripts on the format {{key=value}}
> Suggest to instead have one central platform independent config file e.g.
> {{bin/solr.yml}} or {{bin/solrstart.properties}} which is parsed by
> {{SolrCLI.java}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14014) Allow Solr to start with Admin UI disabled
[
https://issues.apache.org/jira/browse/SOLR-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071747#comment-17071747
]
Uwe Schindler commented on SOLR-14014:
--
IMHO, optionally disabling the admin UI is a good thing. Of course for
newcomers, it should be shown by default. The admin UI is also useful to check
status of your Solr installation. But for this, it's enough to have it enabled
on just one node!
> Allow Solr to start with Admin UI disabled
> --
>
> Key: SOLR-14014
> URL: https://issues.apache.org/jira/browse/SOLR-14014
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI, security
>Affects Versions: master (9.0), 8.3.1
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently Solr always runs the Admin UI. With the history of XSS issues and
> other security concerns that have been found in the Admin UI, Solr should
> offer a mode where the Admin UI is disabled. Maybe, and this is a topic
> that'll need some serious discussion, this should even be the default when
> Solr starts.
> NOTE: Disabling the Admin UI removes XSS and other attack vectors. But even
> with the Admin UI disabled, Solr will still be inherently unsafe without
> firewall protection on a public network.
> *Proposed design:*
> A java system property called *headless* will be used as an internal flag for
> starting Solr in headless mode. This property will default to true. A java
> property can be used at startup to set this flag to false.
> Here is an example:
> {code:java}
> bin/solr start -Dheadless=false {code}
> A message will be added following startup describing the mode.
> In headless mode the following message will be displayed:
> "solr is running in headless mode. The admin console is unavailable. To to
> turn off headless mode and allow the admin console use the following
> parameter startup parameter:
> -Dheadless=false
>
> In non-headless mode the following message will be displayed:
> "solr is running with headless mode turned off. The admin console is
> available in this mode. Disabling the Admin UI removes XSS and other attack
> vectors"
> If a user attempts to access the admin console while Solr is in headless mode
> it Solr will return 401 unauthorized.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071718#comment-17071718
]
Jeremy Novey edited comment on LUCENE-9221 at 3/31/20, 12:10 PM:
-
[~DHAVER0313]- any chance you could change the bottom half of the 'wing' into a
triangle? That would make the negative space an inverted A for Apache.
was (Author: jeremynovey):
[~DHAVER0313]- any chance you could change the bottom half of the 'wing' into a
triangle? That would make it an inverted A for Apache.
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071718#comment-17071718
]
Jeremy Novey commented on LUCENE-9221:
--
[~DHAVER0313]- any chance you could change the bottom half of the 'wing' into a
triangle? That would make it an inverted A for Apache.
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9256) Raise Lucene Webpage's HSTS max-age to 1 year
[ https://issues.apache.org/jira/browse/LUCENE-9256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-9256. --- Resolution: Fixed I committed it, but used wrong issue number (LUCENE-9227): https://github.com/apache/lucene-site/commit/9c798963249573058bfd76199c2936418095 > Raise Lucene Webpage's HSTS max-age to 1 year > - > > Key: LUCENE-9256 > URL: https://issues.apache.org/jira/browse/LUCENE-9256 > Project: Lucene - Core > Issue Type: Bug > Components: general/website >Reporter: Uwe Schindler >Assignee: Uwe Schindler >Priority: Trivial > > As noted in LUCENE-9227, the max-age of the HSTS header in htaccess.template > should be raised to the recommendation of 1 year (e.g., > https://hstspreload.org/). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14014) Allow Solr to start with Admin UI disabled
[
https://issues.apache.org/jira/browse/SOLR-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071695#comment-17071695
]
Jason Gerlowski commented on SOLR-14014:
Thanks for putting a patch together for this Marcus. (To save others time, the
patch uses {{-DenableAdminUI=true|false}} and defaults to "true".)
My concern at this point though is that there was some disagreement about
adding this improvement at all. I think Christine, Alexandre, Jan, Ishan, and
myself were (y) for having some switch to enable/disable the Admin UI (with
differences in what the default should be, ideal property name, etc.). But
David seemed pretty against. He didn't leave an explicit veto, so we're not
blocked if we want to move forward, strictly speaking. But for the sake of
courtesy I'd like to check first whether he finds any of the discussion above
persuasive, or feels strongly enough to veto. I tried to respond to some of
his points above but wasn't sure what he thought. Tagging him now: [~dsmiley]
> Allow Solr to start with Admin UI disabled
> --
>
> Key: SOLR-14014
> URL: https://issues.apache.org/jira/browse/SOLR-14014
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI, security
>Affects Versions: master (9.0), 8.3.1
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently Solr always runs the Admin UI. With the history of XSS issues and
> other security concerns that have been found in the Admin UI, Solr should
> offer a mode where the Admin UI is disabled. Maybe, and this is a topic
> that'll need some serious discussion, this should even be the default when
> Solr starts.
> NOTE: Disabling the Admin UI removes XSS and other attack vectors. But even
> with the Admin UI disabled, Solr will still be inherently unsafe without
> firewall protection on a public network.
> *Proposed design:*
> A java system property called *headless* will be used as an internal flag for
> starting Solr in headless mode. This property will default to true. A java
> property can be used at startup to set this flag to false.
> Here is an example:
> {code:java}
> bin/solr start -Dheadless=false {code}
> A message will be added following startup describing the mode.
> In headless mode the following message will be displayed:
> "solr is running in headless mode. The admin console is unavailable. To to
> turn off headless mode and allow the admin console use the following
> parameter startup parameter:
> -Dheadless=false
>
> In non-headless mode the following message will be displayed:
> "solr is running with headless mode turned off. The admin console is
> available in this mode. Disabling the Admin UI removes XSS and other attack
> vectors"
> If a user attempts to access the admin console while Solr is in headless mode
> it Solr will return 401 unauthorized.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag
gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a
disable Admin UI Flag
URL: https://github.com/apache/lucene-solr/pull/1391#discussion_r400839031
##
File path: solr/core/src/java/org/apache/solr/servlet/LoadAdminUiServlet.java
##
@@ -33,20 +34,34 @@
import java.io.Writer;
import java.nio.charset.StandardCharsets;
+
+
/**
* A simple servlet to load the Solr Admin UI
- *
+ *
* @since solr 4.0
*/
public final class LoadAdminUiServlet extends BaseSolrServlet {
+ // check system properties for whether or not admin UI is enabled, default
is true
+ private static final boolean enabled =
Boolean.parseBoolean(System.getProperty("enableAdminUI", "true"));
+
+ public void init() throws ServletException {
+super.init();
+ }
+
@Override
public void doGet(HttpServletRequest _request,
HttpServletResponse _response)
throws IOException {
HttpServletRequest request = SolrDispatchFilter.closeShield(_request,
false);
HttpServletResponse response = SolrDispatchFilter.closeShield(_response,
false);
-
+ if(!enabled){
+ response.sendError(404, "Solr Admin UI is disabled. To enable it, change
the default value of SOLR_ADMIN_UI_" +
Review comment:
[0] Interesting, I didn't know this existed. I would've made the Java-side
change somewhere else, like HttpSolrCall. But I like this better at first
glance, good digging.
I'll review this file in more detail and then maybe follow up here if
anything looks off.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag
gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag URL: https://github.com/apache/lucene-solr/pull/1391#discussion_r400837339 ## File path: solr/bin/solr ## @@ -2089,6 +2089,14 @@ else SECURITY_MANAGER_OPTS=() fi +# Enable ADMIN UI by default, and give the option for users to disable it when. Review comment: [-1] The way I read this, the Admin UI will be disabled whenever SOLR_ADMIN_UI_ENABLED is empty, unset, etc. And I don't see that value set anywhere, except in commented-out lines in solr.in.sh/solr.in.cmd. So by default the Admin UI is disabled, which doesn't seem like your intention? Or am I missing something? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag
gerlowskija commented on a change in pull request #1391: SOLR-14014 Add a disable Admin UI Flag URL: https://github.com/apache/lucene-solr/pull/1391#discussion_r400834539 ## File path: solr/CHANGES.txt ## @@ -8,6 +8,13 @@ https://github.com/apache/lucene-solr/blob/master/solr/solr-ref-guide/src/solr-u Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. +Improvements +- + +* SOLR-14014: Introducing a system property that allows users to disabled the Admin UI, which is enabled by default. Review comment: [+1] Thanks for including CHANGES text! David made a point recently about ensuring this gets included in PRs, but I still personally always forget. [-0] Typo here: "disabled" -> "disable" [0] This blurb should probably mention the name of the system property, so users can add it easily if they want to. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on issue #1361: LUCENE-8118: Throw exception if DWPT grows beyond it's maximum ram limit
s1monw commented on issue #1361: LUCENE-8118: Throw exception if DWPT grows beyond it's maximum ram limit URL: https://github.com/apache/lucene-solr/pull/1361#issuecomment-606555768 > Another thing we could consider is changing DWPT's postings addressing from int to long so we don't need retry logic. I will look into this. My biggest fear is that we miss places where we exceed limits and testing will be difficult. Yet, there is no rush with this and I have time to explore fixing IntBlockPool addressing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-7871) Platform independent config file instead of solr.in.sh and solr.in.cmd
[
https://issues.apache.org/jira/browse/SOLR-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071676#comment-17071676
]
Jason Gerlowski commented on SOLR-7871:
---
The scripts/include files are a huge testing gap, source of duplication, and
blocker for improvements. Many are reluctant to do the windows-testing
required to make script changes, so they either skip the testing or only update
the nix script - both of which have bitten us recently. So I think this is
still a good idea.
As to why it stalled out, there's probably a few reasons. I didn't have the
karma to commit it myself back in '17, and though Jan was giving me reviews I
think maybe it was a pretty large-impact change to merge from a someone that
wasn't a committer yet, or that hadn't yet proven he would stick around.
Additionally it's a big change to an area where there aren't really any tests,
which also made it difficult to pursue.
In any case, I agree with Jan - this is going to be impacted by the SIP-6
effort if someone takes that on. If SIP-6 work starts getting momentum in the
short term, maybe this should wait until after (or be rolled into that?). If
SIP-6 doesn't get momentum or stalls out for whatever reason, I'd be all for
this ticket resumed (and would be happy to review with a mind toward
committing).
> Platform independent config file instead of solr.in.sh and solr.in.cmd
> --
>
> Key: SOLR-7871
> URL: https://issues.apache.org/jira/browse/SOLR-7871
> Project: Solr
> Issue Type: Improvement
> Components: scripts and tools
>Affects Versions: 5.2.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Labels: bin/solr
> Attachments: SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch,
> SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch, SOLR-7871.patch
>
>
> Spinoff from SOLR-7043
> The config files {{solr.in.sh}} and {{solr.in.cmd}} are currently executable
> batch files, but all they do is to set environment variables for the start
> scripts on the format {{key=value}}
> Suggest to instead have one central platform independent config file e.g.
> {{bin/solr.yml}} or {{bin/solrstart.properties}} which is parsed by
> {{SolrCLI.java}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400816894
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400814597
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400803187 ## File path: .github/workflows/gradle-precommit.yml ## @@ -19,5 +19,8 @@ jobs: java-version: 11 - name: Grant execute permission for gradlew run: chmod +x gradlew +- name: Run something to download the gradle wrapper jar Review comment: Oh.. now I see which validator you had in mind. Sorry for being dim. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400800461
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400788511
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400780360
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400780360
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071619#comment-17071619
]
Cao Manh Dat commented on SOLR-14365:
-
[~jbernste] [~shalin] please take a look at the patch.
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r400739959 ## File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java ## @@ -93,4 +93,11 @@ */ void collect(int doc) throws IOException; + /* Review comment: Let's make it proper javadocs, not comments? ```suggestion /** ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to
skip noncompetitive documents
URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r400736631
##
File path:
lucene/core/src/java/org/apache/lucene/search/ConstantScoreQuery.java
##
@@ -115,7 +115,7 @@ public Weight createWeight(IndexSearcher searcher,
ScoreMode scoreMode, float bo
return new ConstantScoreWeight(this, boost) {
@Override
public BulkScorer bulkScorer(LeafReaderContext context) throws
IOException {
- if (scoreMode == ScoreMode.TOP_SCORES) {
+ if (scoreMode == ScoreMode.TOP_SCORES || scoreMode ==
ScoreMode.TOP_DOCS || scoreMode == ScoreMode.TOP_DOCS_WITH_SCORES) {
Review comment:
maybe add a `isExhaustive()` method on the enum to avoid these large
conditions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r400741063 ## File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java ## @@ -93,4 +93,11 @@ */ void collect(int doc) throws IOException; + /* + * optionally returns an iterator over competitive documents Review comment: Can you document that the default is to return `null` which Lucene interprets as the collector doesn't filter any documents. It's probably worth making explicit as null iterators are elsewhere interpreted as matching no documents. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to skip noncompetitive documents
jpountz commented on a change in pull request #1351: LUCENE-9280: Collectors to
skip noncompetitive documents
URL: https://github.com/apache/lucene-solr/pull/1351#discussion_r400739533
##
File path: lucene/core/src/java/org/apache/lucene/search/LeafCollector.java
##
@@ -93,4 +93,11 @@
*/
void collect(int doc) throws IOException;
+ /*
+ * optionally returns an iterator over competitive documents
+ */
+ default DocIdSetIterator iterator() {
Review comment:
maybe give it a more descriptive name, e.g. `competitiveFilter`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400756154
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071610#comment-17071610
]
Cao Manh Dat commented on SOLR-14365:
-
Attached a WIP patch that includes
* seperate out the logic of creating {{map}}s (backed by {{array}} or
{{hashMap}}) and how collapse are using it. So we just need to create
{{mapFactory}} correspond to a method.
* WIP benchmark test, to ensure that we did achieve something in case of
collapsing on partial of the result.
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400751491
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[jira] [Updated] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat updated SOLR-14365:
Attachment: SOLR-14365.patch
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on issue #1390: LUCENE-9266 remove gradle wrapper jar from source URL: https://github.com/apache/lucene-solr/pull/1390#issuecomment-606473807 I like the direction, Mike. If we can keep gradle wrapper jar in the repository (see that comment on legal jira) then this patch could be easily extended to make the whole download check conditional and only applicable if you're running a stand-alone source release bundle. This is definitely a follow-up work. I can try to help out with Windows testing and maybe cosmetics; let me know if you want me to file a pr against your repo or attach a patch when you're ready. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400724627 ## File path: gradlew.bat ## @@ -81,6 +81,12 @@ set CMD_LINE_ARGS=%* set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar +@rem LUCENE-9266 Download the gradle wrapper jar if we don't have one +set XSUM=28b330c20a9a73881dfe9702df78d4d78bf72368e8906c70080ab6932462fe9e +"%JAVA_EXE%" buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java 6.0.1 "%CLASSPATH%" %XSUM% + +@rem TODO check output of previous command and maybe fail Review comment: https://ss64.com/nt/errorlevel.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400723323
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
Review comment:
I think we should check the checksum even if the file exists. If the
checksum doesn't match then we can try to download again. This sort of solves
the wrapper-upgrade issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400721876
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
+
+File temp = Files.createTempFile("gradle-wrapper", null).toFile();
+try (ReadableByteChannel in =
Channels.newChannel(url.openStream());
+ FileOutputStream out = new FileOutputStream(temp)) {
+out.getChannel().transferFrom(in, 0, maxSize);
+}
+
+try (FileInputStream fis = new FileInputStream(temp)) {
+MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
+
+// Convert binary digest to hex checksum value
+// This will strip leading zeroes, deal with that when we need
to
+String sha256sum = new BigInteger(1,
sha256.digest(fis.readAllBytes())).toString(16);
+
+if (args[2].equals(sha256sum)) {
+if (!temp.renameTo(jar)) {
+System.err.println("Failed renaming wrapper jar,
leaving download at " + temp);
Review comment:
Always cleanup the temp file. What's the use of leaving it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400719401
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
Review comment:
Do you think we really need this check? I'd say simplify and just
url.openStream().transferTo(outputStream).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400721515
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
+if (!jar.exists()) {
+final URL url = new URL("https://github.com/gradle/gradle/raw/v; +
args[0] + "/gradle/wrapper/gradle-wrapper.jar");
+
+// As of v6.0.1 the wrapper is approximately 60K
+// Can increase this is gradle wrapper ever goes beyond 100K, but
keep a safety check
+final int maxSize = 100 * 1024;
+
+File temp = Files.createTempFile("gradle-wrapper", null).toFile();
+try (ReadableByteChannel in =
Channels.newChannel(url.openStream());
+ FileOutputStream out = new FileOutputStream(temp)) {
+out.getChannel().transferFrom(in, 0, maxSize);
+}
+
+try (FileInputStream fis = new FileInputStream(temp)) {
+MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
+
+// Convert binary digest to hex checksum value
+// This will strip leading zeroes, deal with that when we need
to
+String sha256sum = new BigInteger(1,
sha256.digest(fis.readAllBytes())).toString(16);
+
+if (args[2].equals(sha256sum)) {
+if (!temp.renameTo(jar)) {
Review comment:
Rename won't work on Windows between drives. I'd do a copy and delete temp.
If you want to do a rename then you'd have to create that temp file in the same
directory as gradle wrapper is expected to finally be located.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle wrapper jar from source
dweiss commented on a change in pull request #1390: LUCENE-9266 remove gradle
wrapper jar from source
URL: https://github.com/apache/lucene-solr/pull/1390#discussion_r400718042
##
File path:
buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.gradle;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.math.BigInteger;
+import java.net.URL;
+import java.nio.channels.Channels;
+import java.nio.channels.ReadableByteChannel;
+import java.nio.file.Files;
+import java.security.MessageDigest;
+
+public class WrapperDownloader {
+public static void main(String[] args) throws Exception {
+if (args.length != 3) {
+System.err.println("Usage: java WrapperDownloader.java
");
+System.exit(1);
+}
+
+File jar = new File(args[1]);
Review comment:
NIO maybe? Paths.get(.). Files.exists()... these give a lot more reasonable
exception messages.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400716712
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
dweiss commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400715656
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
Review comment:
I'm not sure, to be honest. In that setting, which JVM is used for running
tests? My assumption was that it'd be the first one one from path (or
JAVA_HOME) but maybe I'm wrong.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
noblepaul commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r400712704
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
Review comment:
must be fixed now
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
noblepaul commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r400712562
##
File path:
solr/core/src/test/org/apache/solr/handler/admin/ZookeeperReadTest.java
##
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.lang.invoke.MethodHandles;
+import java.net.URL;
+import java.util.Map;
+
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.util.Utils;
+import org.apache.zookeeper.CreateMode;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.apache.solr.common.util.StrUtils.split;
+import static org.apache.solr.common.util.Utils.getObjectByPath;
+
+public class ZookeeperReadTest extends SolrCloudTestCase {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ @BeforeClass
+ public static void setupCluster() throws Exception {
+configureCluster(1)
+.addConfig("conf", configset("cloud-minimal"))
+.configure();
+ }
+
+ @Before
+ @Override
+ public void setUp() throws Exception {
+super.setUp();
+ }
+
+ @After
+ @Override
+ public void tearDown() throws Exception {
+super.tearDown();
+ }
+
+ @Test
+ public void testZkread() throws Exception {
+URL baseUrl = cluster.getJettySolrRunner(0).getBaseUrl();
+String basezk = baseUrl.toString().replace("/solr", "/api") +
"/cluster/zk";
+
+try (HttpSolrClient client = new
HttpSolrClient.Builder(baseUrl.toString()).build()) {
+ Object o = Utils.executeGET(client.getHttpClient(),
+ basezk + "/security.json",
+ Utils.JSONCONSUMER);
+ assertNotNull(o);
+ o = Utils.executeGET(client.getHttpClient(),
+ basezk + "/configs",
+ Utils.JSONCONSUMER);
+ assertEquals("0", String.valueOf(getObjectByPath(o, true,
split(":/configs:_default:dataLength", ':';
+ assertEquals("0", String.valueOf(getObjectByPath(o, true,
split(":/configs:conf:dataLength", ':';
+
+ o = Utils.executeGET(client.getHttpClient(),
+ basezk + "/configs?leaf=true",
+ Utils.JSONCONSUMER);
+ assertTrue(((Map)o).containsKey("/configs"));
Review comment:
NO. I couldn't see it. Maybe you can just commit that directly
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9280) Add ability to skip non-competitive documents on field sort
[ https://issues.apache.org/jira/browse/LUCENE-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071550#comment-17071550 ] Adrien Grand commented on LUCENE-9280: -- Woohoo. I can try to give a little more background about this feature. The question that usually comes first is the benefit of this approach compared to index sorting. Index sorting provides greater speedups than this change, but is limited to one field and one sort order. While it's possible to create multiple indexes that all have a different sort order, it's quite expensive in terms of resources, and creates new problems e.g. if you need all your indices to have a consistent view of the documents that exist in the index. Even if you only sort by a single field, this feature might be relevant. For instance with time series, it's common to sort data by descending timestamp, e.g. to live-tail events that match a filter, but you sometimes also need to sort in ascending order to find the first occurrence of a problem, which is often a useful piece of information for root cause analysis. With index sorting, sorting in reverse order of the index sort is a worst-case scenario. The approach to store min/max values per block in doc values would work too. Something I like with this approach is that it doesn't require adding new APIs to doc values, and that it doesn't only work with numbers. For instance, this approach could also be used to speed up sorting by geo-distance too (using the same principle as LatLonPointDistanceFeatureQuery) without requiring the introduction of a new type of doc values that supports multiple dimensions. > Add ability to skip non-competitive documents on field sort > > > Key: LUCENE-9280 > URL: https://issues.apache.org/jira/browse/LUCENE-9280 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mayya Sharipova >Priority: Minor > Time Spent: 6h 40m > Remaining Estimate: 0h > > Today collectors, once they collect enough docs, can instruct scorers to > update their iterators to skip non-competitive documents. This is applicable > only for a case when we need top docs by _score. > It would be nice to also have an ability to skip non-competitive docs when we > need top docs sorted by other fields different from _score. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
noblepaul commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r400704740
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
Review comment:
I fail to see the problem. Could you please explain
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
noblepaul commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r400704943
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZookeeperRead.java
##
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZookeeperRead {
Review comment:
APIs are named as *API
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
noblepaul commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r400704740
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
Review comment:
I fail to see the problem. Could you please explain
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400699833
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
+
doFirst {
def srcDirs = sourceSets.main.java.srcDirs.findAll { dir ->
dir.exists() }
-ant.javadoc(
-overview: file("src/java/overview.html"),
-packagenames: "org.apache.lucene.*,org.apache.solr.*",
-destDir: javadocDestDir,
-access: "protected",
-encoding: "UTF-8",
-charset: "UTF-8",
-docencoding: "UTF-8",
-noindex: "true",
-includenosourcepackages: "true",
-author: "true",
-version: "true",
-linksource: linksource,
-use: "true",
-failonerror: "true",
-locale: "en_US",
-windowtitle: title,
-doctitle: title,
-maxmemory: "512m",
-classpath: sourceSets.main.compileClasspath.asPath,
-bottom: "Copyright 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
-) {
- srcDirs.collect { srcDir ->
-packageset(dir: srcDir)
+project.exec {
+ executable javadocCmd
+
+ args += [ "-overview", file("src/java/overview.html").toString() ]
+ args += [ "-sourcepath", srcDirs.join(" ") ]
+ args += [ "-subpackages", project.path.startsWith(":lucene") ?
"org.apache.lucene" : "org.apache.solr"]
+ args += [ "-d", project.javadoc.destinationDir.toString() ]
+ args += [ "-protected" ]
+ args += [ "-encoding", "UTF-8" ]
+ args += [ "-charset", "UTF-8" ]
+ args += [ "-docencoding", "UTF-8" ]
+ args += [ "-noindex" ]
+ args += [ "-author" ]
+ args += [ "-version" ]
+ if (linksource) {
+args += [ "-linksource" ]
}
+ args += [ "-use" ]
+ args += [ "-locale", "en_US" ]
+ args += [ "-windowtitle", title ]
+ args += [ "-doctitle", title ]
+ args += [ "-classpath", sourceSets.main.compileClasspath.asPath ]
+ args += [ "-bottom", "Copyright 2000-${buildYear} Apache
Software Foundation. All Rights Reserved." ]
- tag(name: "lucene.experimental", description: "WARNING: This API is
experimental and might change in incompatible ways in the next release.")
- tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
- tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+ args += [ "-tag",
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1388: LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects
mocobeta commented on a change in pull request #1388: LUCENE-9278: Use
-linkoffline instead of relative paths to make links to other projects
URL: https://github.com/apache/lucene-solr/pull/1388#discussion_r400682318
##
File path: gradle/render-javadoc.gradle
##
@@ -15,93 +15,105 @@
* limitations under the License.
*/
-// generate javadocs by using Ant javadoc task
+// generate javadocs by calling javadoc tool
+// see https://docs.oracle.com/en/java/javase/11/tools/javadoc.html
+
+// utility function to convert project path to document output dir
+// e.g.: ':lucene:analysis:common' => 'analysis/common'
+def pathToDocdir = { path -> path.split(':').drop(2).join('/') }
allprojects {
plugins.withType(JavaPlugin) {
-ext {
- javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
- javadocDestDir = "${javadocRoot}/${project.name}"
-}
-
task renderJavadoc {
- description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ description "Generates Javadoc API documentation for the main source
code. This directly invokes javadoc tool."
group "documentation"
ext {
-linksource = "no"
+linksource = false
linkJUnit = false
-linkHref = []
+linkLuceneProjects = []
+linkSorlProjects = []
}
dependsOn sourceSets.main.compileClasspath
inputs.files { sourceSets.main.java.asFileTree }
- outputs.dir project.javadocRoot
+ outputs.dir project.javadoc.destinationDir
def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
def title = "${libName} ${project.version} ${project.name}
API".toString()
+ // absolute urls for "-linkoffline" option
+ def javaSEDocUrl = "https://docs.oracle.com/en/java/javase/11/docs/api/;
+ def junitDocUrl = "https://junit.org/junit4/javadoc/4.12/;
+ def luceneDocUrl =
"https://lucene.apache.org/core/${project.version.replace(".", "_")}".toString()
+ def solrDocUrl =
"https://lucene.apache.org/solr/${project.version.replace(".", "_")}".toString()
+
+ def javadocCmd =
org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()
Review comment:
I set the default java home to JDK14 (that is used to launch gradle) and
"org.gradle.java.home" to JDK11.
```
$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/adoptopenjdk-14.jdk/Contents/Home
$ java -version
openjdk version "14" 2020-03-17
$ cat gradle.properties
org.gradle.java.home=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
```
I did just print debug for
`org.gradle.internal.jvm.Jvm.current().getJavadocExecutable()`, that said the
`JVM.current()` points the same jvm to org.gradle.java.home property.
```
/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home/bin/javadoc
```
Is this expected behaviour (or is there another configuration point)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
