oscerd opened a new pull request, #29063: URL: https://github.com/apache/camel/pull/29063
Fixes [CAMEL-26216](https://issues.apache.org/jira/browse/CAMEL-26216), found while adding the catalog body types (CAMEL-24961). ## 1. QUERY and QUERY_BY_ID without CamelPineconeQueryTopK fail with NullPointerException **Was:** Both actions unboxed the `CamelPineconeQueryTopK` header into an `int`. Without the header, they failed with a `NullPointerException`. **Now:** Without the header, both actions return 10 matches, the same default as camel-weaviate. The header metadata now has the default value 10 and a description of what the header does. ## 2. The pinecone:rag transformer never finds the query results **Was:** The transformer read the body as a List. QUERY and QUERY_BY_ID leave a `QueryResponseWithUnsignedIndices` as the body, so the transformer always gave an empty List. **Now:** For a `QueryResponseWithUnsignedIndices` body, the transformer reads the matches of the response. A List of `ScoredVectorWithUnsignedIndices` works as before. The transformerIn body types add the pojo carrier. Their description says that the text comes from the `text_segment` metadata, and that a query returns this metadata when the `CamelPineconeQueryIncludeMetadata` header is true. ## 3. A vector as a List of Double fails with ClassCastException **Was:** UPSERT, UPDATE and QUERY passed the body List as is to the Pinecone SDK, which takes a List of Float. A List of Double (what a JSON array unmarshals to) failed with a raw `ClassCastException`. So did a List of another number type, a `double[]` and a `Double[]`. **Now:** Each element is converted with `Number.floatValue()` into a List of Float. A `float[]` works as before. A List with an element that is null or not a number fails with an `IllegalArgumentException` that gives the index and the type of the element, but not its value. Any other body fails with an `InvalidPayloadException` whose message names the expected types. ## 4. UPSERT without CamelPineconeIndexId fails **Was:** UPSERT passed a null id to the SDK, which failed with a `PineconeValidationException`. **Now:** UPSERT stores the vector under a random UUID, as the `pinecone:embeddings` transformer does. It also sets the `CamelPineconeIndexId` header to that id, so the route can read it; the body is the `UpsertResponse`, which carries no ids. ## 5. camel-test-infra pinecone maps only port 5080 **Was:** The pinecone-local container exposed only port 5080, the control plane. The control plane gives each index it creates as `localhost:5081` and up, and the client connects to that address for upsert, query and fetch. A test could not reach these data-plane addresses. **Now:** - The container binds the index ports 5081 to 5085 to the same ports on the host. - It sets `PINECONE_HOST` to `localhost`, as Pinecone's documented `docker run` command for pinecone-local does. - The control plane keeps its current port setup: a random host port in tests, a fixed one under `camel infra run`. - The container waits only for the control-plane port. Without this, the default wait strategy would also wait for the bound index ports. ## Tests `PineconeVectorDbProducerTest` is new. It runs against a fake Pinecone: the control plane (describe index) on a JDK `HttpServer`, and the data plane of the index on a gRPC server. The fake records the requests the client sends, so the test needs no Pinecone. - UPSERT, UPDATE and QUERY each take a List of Float, a List of Double, a List of BigDecimal, a `float[]`, a `double[]` and a `Double[]` (18 cases). - QUERY and QUERY_BY_ID without the TopK header ask for 10 matches. - UPSERT without the id header stores the vector under a generated UUID and sets the header to it. - A route that runs QUERY and then `transformDataType` to `pinecone:rag` gives the text of the match. - A String body, a List of String and a List with a null element fail with the expected exception and message, and nothing is sent. `PineconeReverseEmbeddingsDataTypeTransformerTest` used to assert that a `QueryResponseWithUnsignedIndices` gives an empty List. It now asserts the texts of the matches, the pojo carrier, and an empty List for a response without matches. Without the fix, 18 of the 24 new and changed tests fail: - the List of Double, List of BigDecimal, `double[]` and `Double[]` cases of the three actions fail with the old `ClassCastException`; - the two TopK tests fail with the old `NullPointerException`; - the UPSERT id test fails with the old `PineconeValidationException`; - the rag route test fails because the old transformer gave an empty List; - the transformer test fails on the old body types; - the failure test fails because the old message does not name the expected types. The 6 that pass are the List of Float and `float[]` cases of the three actions, which already worked. Module: 27 tests, 0 failures. The test infra module was built, and its container configuration was checked without starting it: the port bindings 5081 to 5085, `PINECONE_HOST`, and the wait on port 5080 with the 3-minute timeout. The container ITs were not run because they need Docker. ## Docs and generated files - `CamelPineconeQueryTopK` has the default value 10 and a description. The `CamelPineconeIndexId` description says that UPSERT generates the id. These header changes regenerate `pinecone.json`, its catalog copy, and the endpoint DSL. - The producerIn body types description of the endpoint now says the vector is a List of numbers or a float or double array, and that UPSERT generates the id. It no longer says a List of Double fails. - The `pinecone:rag` transformerIn carriers and description changed. This regenerates `pinecone-rag.json` and its catalog copy. - `pinecone-component.adoc` and its catalog copy give the default of 10 matches. A new paragraph covers the vector types, the generated UPSERT id and the `pinecone:rag` transformer. - The upgrade guide has an entry for the default topK, the generated id and the new `pinecone:rag` output. ## Not changed - Item 1: QUERY and QUERY_BY_ID default to 10 matches, as camel-weaviate does, rather than failing with a message that names the header. camel-pgvector defaults to 3. No endpoint option was added. - Item 4: UPSERT generates an id, as the `pinecone:embeddings` transformer and camel-pgvector UPSERT do, rather than failing with a message that names the header. UPDATE and QUERY_BY_ID still need the header, because they work on an existing vector. - Item 5, the index container: `PineconeIndexContainer` is not changed. It runs the single-index pinecone-index image, which serves its index on the one port it maps, and no test uses it. - Item 5, the port range: only five index ports (5081 to 5085) are bound, as the issue proposes. Pinecone's own examples map a wider range; its Pinecone Local blog post maps 5081 to 6000. The bound ports must be free on the host. - Item 5, end of life: Pinecone's documentation now says Pinecone Local reached end of life on 7 October 2026. The test infra still uses pinecone-local v1.0.0.rc0. - UPSERT still sends no metadata, so the `text_segment` metadata that the `pinecone:rag` transformer reads must be written by another client. - An empty List is passed to Pinecone as is. - The javaType of two headers does not match what the producer reads: `CamelPineconeQueryFilter` is declared as String but read as a Struct, and `CamelPineconeQueryIncludeMetadata` is declared as Struct but read as a Boolean. This change leaves them as they are. Main only, as the other fixes of the CAMEL-24961 follow-ups. _Claude Code on behalf of Andrea Cosentino (oscerd)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
