nick-boss-tech opened a new pull request, #5200: URL: https://github.com/apache/solr/pull/5200
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-16155 ## What happens today **Failed document adds put the rejected field values, and sometimes the whole document, into error messages that are logged and returned in the update response.** Field creation errors name the value: "Error while creating field 'x' from value '...'" ([FieldType.java at the base](https://github.com/nick-boss-tech/solr/blob/b73f03976fa/solr/core/src/java/org/apache/solr/schema/FieldType.java#L311)). A DocumentBuilder failure message carries the underlying exception text, which contains the value ([DocumentBuilder.java at the base](https://github.com/nick-boss-tech/solr/blob/b73f03976fa/solr/core/src/java/org/apache/solr/update/DocumentBuilder.java#L279-L291)). The javabin loader's "ERROR adding document" appends the whole document ([JavabinLoader.java at the base](https://github.com/nick-boss-tech/solr/blob/b73f03976fa/solr/core/src/java/org/apache/solr/handler/loader/JavabinLoader.java#L126)). Field values can be sensitive. ## What this change does **Error messages keep the document id and the field name. The value is logged at TRACE instead.** - Field creation: [FieldType.createField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/schema/FieldType.java#L309-L313), [DenseVectorField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java#L371-L375), [StrFloatLateInteractionVectorField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/schema/StrFloatLateInteractionVectorField.java#L212-L216) and [RankField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/schema/RankField.java#L117-L122) throw "Error while creating field 'x'" with no value in the message, and log the value at TRACE. - DocumentBuilder builds one message, "ERROR: [doc=id] Error adding field 'x'", through [a single helper](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/update/DocumentBuilder.java#L106-L110) at both throw sites, and logs the value at TRACE. - JavabinLoader names the document by its unique key only, through [describeDocument](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/java/org/apache/solr/handler/loader/JavabinLoader.java#L148-L156). With no unique key it adds nothing. - The update error response no longer carries the exception text, only the message above. This part goes beyond the throw sites. ## Proof **The new tests fail on current main and pass here: 95 focused tests, 0 failures, verified 2026-10-22 at this head.** DenseVectorFieldTest 50 run (2 skipped), RankFieldTest 14 (1 skipped), DocumentBuilderTest 17, JavabinLoaderTest 3, TolerantUpdateProcessorTest 11. The new tests assert the rejected value is absent from the thrown message, for [FieldType.createField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java#L765), for [the vector path through an update](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java#L795), and for [RankField](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/solr/core/src/test/org/apache/solr/schema/RankFieldTest.java#L63). The solr core module checks also pass. ## A choice to check **Recommendation: the update response carries the same scrubbed message as the log, with no exception text.** Take a document with id 42 whose price field holds the text abc. Today the error in the update response reads: `ERROR: [doc=42] Error adding field 'price'='abc' msg=Error while creating field 'price' from value 'abc'` With this change, the same failure reads: `ERROR: [doc=42] Error adding field 'price'` The value abc no longer reaches the client at all. The reason is gone too: the old text said why the field failed, inside the nested message, and the new text does not. And the guarantee no longer depends on that nested message: it comes from whatever code failed during the add, so a value inside any other error, from a path nobody scrubbed, would be forwarded to the client in the same way. Dropping the nested text closes that route, not just this instance. The debugging detail is not lost to the operator: the server log keeps the value and the full exception at TRACE. The alternative is to drop only the `='abc'` part and keep the nested message, so clients keep the reason. That leaves the leak open through any unscrubbed error text, in this code and in future code. Keep the response scrub, or revert just that piece? It stands on its own; the message changes do not depend on it. ## A note on #1151 **The field-creation part of this change follows Jan Høydahl's [PR #1151](https://github.com/apache/solr/pull/1151), which has carried this fix since 2022. Thank you to Jan for the work.** This ticket has waited a long time and the leak is worth closing, so this PR opens the fix fresh on current main, with tests that check the messages themselves. ## Limits **The values still reach the log at TRACE, and error paths outside these two areas were not audited.** TRACE keeps the value available for debugging, as the ticket discussion asked; it is the only place the values remain. RankField gains a logger, used only for that TRACE line. A wider audit of error messages can be a follow-up on request. Changelog: [changelog/unreleased/SOLR-16155-no-field-values-in-update-errors.yml](https://github.com/nick-boss-tech/solr/blob/9d1d0e5df9d91d5b18bc62c277c6baff5faee69c/changelog/unreleased/SOLR-16155-no-field-values-in-update-errors.yml) *AI-generated description. Nick Shanin directed the work and takes responsibility for this contribution.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
