nick-boss-tech opened a new pull request, #5169: URL: https://github.com/apache/solr/pull/5169
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-12505 ## What happens today **fetch() returns the tuples without the fetched fields when the target handler's default query parser is not lucene.** fetch() sends its batch as `{! df=<join field> q.op=OR cache=false }` followed by the escaped key values. It sends no `defType`, so the handler's default parser reads the batch. Solr reads the `{!` prefix only when that default is lucene or func ([QParser.java lines 387 to 396, merge-base](https://github.com/nick-boss-tech/solr/blob/14c7aac0d151402b00259e2fb9bf5eed7049ec5d/solr/core/src/java/org/apache/solr/search/QParser.java#L387-L396)). With another default, the batch is read as plain query text and the fetched fields are missing. The ticket's assignee, Eric Pugh, found that removing `defType=edismax` from the `/select` handler fixed it. The batch is built at [FetchStream.java lines 239 to 246](https://github.com/apache/solr/blob/14c7aac0d151402b00259e2fb9bf5eed7049ec5d/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/FetchStream.java#L239-L246) (merge-base; current main has the same lines). ## What this change does **fetch() sends `defType=lucene` with every batch, so the prefix is read whatever the handler's default is.** [FetchStream.java lines 273 to 276](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/FetchStream.java#L273-L276) add the parameter. Behavior change: a `defType` default on the handler no longer applies to fetch batches. A handler with the lucene default, which is Solr's default and the test setup, sends the same batch as before. **fetch() also rejects a join field name that would break its batch query.** A join field name containing whitespace, braces, `!`, quotes or a backslash now fails when the expression is built, and the error names the field ([FetchStream.java lines 153 to 190](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/FetchStream.java#L153-L190)). The whitespace in that check is `Character.isWhitespace`, the same definition the local-params reader splits on, so Unicode spaces are rejected as well as ASCII ones. Before, such a name ended the `df=` value early in the batch query and corrupted the fetched-field list. The name is rejected rather than escaped: escaping would let the batch accept a name that is not in the schema, while the check fails with a message that says what is wrong. ## Proof **Each new test fails without its fix and passes with it.** [`testFetchStreamWithNonLuceneDefaultDefType`](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamDecoratorTest.java#L951-L1006) sets the test collection's `/select` default to edismax. On the merge-base FetchStream, with the test as it stands at 5f20e1171bc, it fails with `ComparisonFailure: expected:<alpha> but was:<null>`. With this change, StreamDecoratorTest runs 58 tests, 5 skipped, 0 failures at head `248a9715b1c3b00793fc4efca58738894c9acd65`, verified 2026-10-14. That includes the two existing fetch tests, which use the default parser and do not change. [`testFetchStreamRejectsJoinFieldNameThatBreaksLocalParams`](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamDecoratorTest.java#L1009-L1036) sends the join field name `a i`. On the parent commit 5f20e1171bc, which has no join check, the drain fails with the server's parser error, `invalid query parser 'i'`, which does not name the join field. With this change it fails at construction with the message on the cause chain (see Limits). [`testFetchStreamRejectsJoinFieldNameWithUnicodeSpace`](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamDecoratorTest.java#L1039-L1069) sends the join field name `a\u2003i`, with an em space (U+2003) instead of an ASCII space; the local-params reader splits on that character too. On the merge-base code, which has no join check, nothing rejects the name and no error is thrown. The ASCII-only pattern the check used before did not catch it either. With this change it fails at construction with the same message. ## A choice to check **The batch sets `defType=lucene`. The ticket thread also named the terms query parser.** Joel Bernstein proposed setting the parameter, and David Smiley agreed ([his comment](https://issues.apache.org/jira/browse/SOLR-12505?focusedCommentId=16519637)). Smiley also suggested the terms parser ([plugin name, merge-base](https://github.com/nick-boss-tech/solr/blob/14c7aac0d151402b00259e2fb9bf5eed7049ec5d/solr/core/src/java/org/apache/solr/search/TermsQParserPlugin.java#L74), [comma separator, merge-base](https://github.com/nick-boss-tech/solr/blob/14c7aac0d151402b00259e2fb9bf5eed7049ec5d/solr/core/src/java/org/apache/solr/search/TermsQParserPlugin.java#L150)). That alternative sets `defType=terms` with the keys as the query. It changes the query text of every batch, and the keys would need the terms separator instead of the escaping the batch uses now. Was setting `defType` on the batch the right call, or should the batch use the terms parser? ## Limits **The batch no longer depends on a handler's default defType. The test uses edismax only. Not covered: the realtime get endpoint.** - Only the batch request and the join field check change. The search side of the expression is untouched. - StreamFactory reports only a generic message for a constructor failure, so the check's message sits on the cause chain. Fixing that is outside this change; a follow-up ticket and PR can be opened on request. - The tests use two bad names, an ASCII space and an em space. Brace, bang, quote and backslash names are not tested. - The realtime get endpoint, which the ticket raised, works only when the join field is the uniqueKey. It is not used here. - Base: this change is written against merge-base 14c7aac0d151402b00259e2fb9bf5eed7049ec5d, which is 77 commits behind upstream main at d38e0f16237, as counted on 2026-10-13. It has not been rebased. Changelog: [changelog/unreleased/SOLR-12505-fetch-stream-deftype.yml](https://github.com/nick-boss-tech/solr/blob/248a9715b1c3b00793fc4efca58738894c9acd65/changelog/unreleased/SOLR-12505-fetch-stream-deftype.yml) ### AI assistance AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
