janhoy opened a new pull request, #4910: URL: https://github.com/apache/solr/pull/4910
Adds [Newline Delimited JSON](https://ndjson.org/) (ND-JSON / JSON Lines) as an update format. ### What it does - `/update` selects the format from `Content-Type: application/x-ndjson` on both v1 and v2. - Adds an `/update/ndjson` path on v1 and v2, which selects the format regardless of content type. - Indexes one line at a time, so peak memory depends on the longest line rather than on the request size. - `bin/solr post` now sends `.jsonl` and `.ndjson` files to `/update` (they previously went to `/update/extract` and were indexed as a single Tika blob). - Bounds a single line, defaulting to ~25% of the heap while parsing and overridable with `solr.ndjson.maxLineLength` or a `maxLineLength` handler argument, failing with a 400 instead of taking the node down with an `OutOfMemoryError`. ### What it does not do - No update commands. `add`, `delete`, `commit` etc - No atomic/partial updates. A nested JSON object is always a child document, exactly as on `/update/json/docs`. - No change to any existing format. `/update` with `application/json`, XML, CSV, javabin and CBOR behave as before. ### Notes for reviewers - Content-type matching is now case-insensitive and tolerates whitespace before `;`, per RFC 9110. This fixes pre-existing behavior. - v2 `/update` only reroutes when *every* content stream is NDJSON, since the rewritten path applies to the whole request. - One known limitation: on v2 `/update`, `update.contentType` is only honored as a query parameter, not as a handler default/invariant, because the rewrite happens before `RequestHandlerBase` merges defaults. https://issues.apache.org/jira/browse/SOLR-18445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
