nick-boss-tech opened a new pull request, #5199: URL: https://github.com/apache/solr/pull/5199
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-14200 ## What happens today **When a collection is missing from the manual shards map, streaming expressions fail with a NullPointerException or return the wrong result, each stream in its own way.** TupleStream.getShards returns the collection's entry from the manual shards map as is, so a missing collection gives callers null ([TupleStream.java at the base](https://github.com/nick-boss-tech/solr/blob/b73f03976fa/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/TupleStream.java#L221)). Most callers then throw NullPointerException. SignificantTermsStream returns an empty result with no error. DeepRandomStream divides by the shard count and throws ArithmeticException. The local core filter also removes entries from that same list, so the shard list stored in the stream context changes ([same file at the base](https://github.com/nick-boss-tech/solr/blob/b73f03976fa/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/TupleStream.java#L224)). The crash in the ticket, a search over a shard whose only replica was deleted, no longer reproduces on current main. The manual shards map path above is what is still broken. ## What this change does **getShards returns an empty list instead of null for a missing collection, the streams that need shards fail with a clear message, and the core filter works on a copy.** [TupleStream.getShards](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/TupleStream.java#L221-L226) returns an empty list when the map has no entry for the collection, and copies the configured list before the core filter removes entries, so the list in the stream context is unchanged. [SqlStream](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/SqlStream.java#L195-L197) and [ParallelStream](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/ParallelStream.java#L293-L295) throw IOException "No shards available for collection X" on an empty list. [SignificantTermsStream](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/SignificantTermsStream.java#L284-L290) and [DeepRandomStream](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/DeepRandomStream.java#L281-L286) throw IllegalArgumentException "No shards for collection X. Check the manual shards entry in t he stream context or the replica state." ## Proof **Five new StreamingTest tests fail on current main and pass with this change; the full class passes at this head, verified 2026-10-22.** - StreamingTest at this head: 39 run, 2 skipped, 0 failures. The solrj-streaming module checks also pass. - On main without this change, the same class fails 5 tests: four NullPointerExceptions from the missing-mapping tests ([the direct getShards test](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamingTest.java#L3083), [CloudSolrStream](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamingTest.java#L3272), SignificantTermsStream and DeepRandomStream) and the [caller-list mutation assertion](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/solr/solrj-streaming/src/test/org/apache/solr/client/solrj/io/stream/StreamingTest.java#L3093). ## A choice to check **Recommendation: keep the empty-list return, so no caller of getShards can get null.** getShards is a public static method. Today it returns null in one case and a list in the rest, and each caller copes on its own; four of them cope badly. This change makes it always return a list, and the streams that cannot run without shards say so. Code outside Solr that calls getShards and checks for null would see an empty list instead. Two questions decide it: 1. Is the contract change acceptable? The alternative is to keep null and add a null check to each caller, which leaves the next new stream class free to repeat the bug. 2. Should getShards itself throw for a missing collection instead? That would also fail streams that can run with an empty list, so the checks sit in the streams that need shards. Keep the empty-list return, or keep null and check at each caller? ## Limits **Only the manual shards map path changes; shard lookup from cluster state is untouched, and the change is all in solrj-streaming.** - Streams that call getShards with a manual shards map and can run with an empty list keep their behavior, minus the null. - The four streams named above are the ones that cannot proceed without shards; no other stream class gains a new error. Changelog: [changelog/unreleased/SOLR-14200-streaming-manual-shards.yml](https://github.com/nick-boss-tech/solr/blob/35b3d94d566d730798bc71fc8c8b08f67933ef17/changelog/unreleased/SOLR-14200-streaming-manual-shards.yml) *AI-generated description. Nick Shanin directed the work and takes responsibility for this contribution.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
