oscerd commented on PR #3026: URL: https://github.com/apache/camel-kamelets/pull/3026#issuecomment-5664263210
Thanks @yuvalif — that is exactly the detail I needed. Pushed, following your calls one for one. ## Added as headers | field | header | why, per your comment | |---|---|---| | `s3.object.metadata` | `ceph-object-metadata` | user attributes; usable for notification filtering, and useful feeding a catalog or database | | `s3.object.tags` | `ceph-object-tags` | same but more structured | | `eventId` | `ceph-event-id` | spotting duplicates after a Kafka retransmit | | `opaqueData` | `ceph-opaque-data` | user information on the topic | `s3.bucket.id` left out, as you suggested — internal id, debugging only. Each also gets a `ce-` CloudEvents counterpart (`ce-cepheventid` and so on), which is how this catalog names source output headers — `infinispan-source` and `ftp-source` both do the plain-plus-`ce-` pair. That keeps the JSON paths out of the consumer's way. Verified with a populated payload: ``` id=[evt-12345] opaque=[[email protected]] meta=[[{key=x-amz-meta-owner, val=alice}]] tags=[[{key=env, val=prod}]] ``` `metadata` and `tags` arrive as structured lists, not flattened strings, so your "more structured" point survives into the consumer. ## One thing I would like your read on You named `eventId` for duplicate detection, which is the use I had hoped for too. But the event structure in the Ceph docs carries it **empty**: ```json "eventId":"", ``` and with that exact payload the header is set but blank: ``` EDGE id=[] opaque=[] meta=[[]] tags=[[]] ``` So anyone deduplicating on `ceph-event-id` would get a header that is always equal and silently dedupe nothing — the worst kind of failure, because it looks like it is working. Is `eventId` populated in practice, and does it depend on something — persistent topics, a particular RGW version? If it is only populated in some configurations, I would rather say so in the Kamelet description than let people build idempotency on it and find out later. ## One behaviour change worth flagging Reading the `Records` array means a message that is **not** a Ceph notification now fails the exchange: ``` PathNotFoundException: Missing property in path $['Records'] ``` Previously, with `getObject` off, the Kamelet passed anything through untouched. I kept the strict behaviour deliberately — silently emitting blank Ceph headers for foreign messages would be worse than failing — and the description now says the topic is expected to carry only Ceph notifications. Happy to relax it if you think a shared topic is a realistic deployment. --- _Claude Code on behalf of Andrea Cosentino_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
