Hi Enrique, in general knative eventing deliver the events to a target service in a format like this:
https://github.com/cloudevents/spec/blob/main/cloudevents/bindings/http-protocol-binding.md#31-binary-content-mode See the example below, and what we have is that all extensions that you have in the cloud event are send as http-headers, like ce-myExtension1 ce-myExtension2 etc. This is why if we do objectMapper.readValue(message.getPayload().toString(), MyClass.class) It might happen that some values are not cached up, because all the cloud event extensions goes as headers and not as part of the message.getPayload(). HTTP/1.1 200 OK ce-specversion: 1.0 ce-type: com.example.someevent ce-time: 2018-04-05T03:56:24Z ce-id: 1234-1234-1234 ce-myExtension1: my value1 ce-myExtension2: my value2 ce-source: /mycontext/subcontext .... further attributes ... Content-Type: application/json; charset=utf-8 Content-Length: nnnn { ... application data ... } Cheers, Walter. El mar, 24 oct 2023 a las 13:38, Enrique Gonzalez Martinez (< [email protected]>) escribió: > I think is fair point. From my perspective there is no problem. In the end > we just need to duplicate fields in what u call binary format and unwrap > those changes in the data index to make it work. > > Unfortunately the events requires need the context of the event as it is > big event was replace by raw diff events. > > My guess is that knative unwrap the cloud event and send that wrapped data > (otherwise) there is no eventpublisher which sends no cloud event afaik.. > is that correct ? > > El mar, 24 oct 2023, 13:25, Walter Medvedeo <[email protected]> > escribió: > > > Hello Guys, > > > > I just wanted to raise up a tricky part of the cloud events that I think > > it's important to take into account when we introduce modifications on > the > > events produced / received by our infrastructure in general. > > > > When we send cloud events over the wire, there are basically two modes > that > > can be used to send them, "structured" and "binary", all good. In > general, > > every time our code send those events, it does by using the "structured" > > mode. And, in general, our services like the data-index, and the > > jobs-service will receive them in "structured" mode too. > > > > For instance, if we use kafka, and the process produces an event, that > > event will be delivered in the "structured" mode to the data-index. > > > > BUT, there are some scenarios, where even when we produce these events > in " > > structured" mode, we don't have 100% control, on which mode they will be > > delivered. That is the case of the knative eventing infrastructure. When > we > > have a installation that relies on knative eventing, very simplified, we > > have something like this: > > > > Process --> (produce event in "structured" mode) --> knative broker > > (receives the event) --> knative trigger (delivers the event in "binary" > > mode) --> Data Index (receives the event in binary mode) > > > > To deliver the originally event created in structured mode, in binary > mode > > instead, is something internal to the knative eventing system, and they > > have good reasons for that. On reason, is for example, that by managing > the > > events in the binary mode internally, it's easier to do filtering/routing > > operations, based on the ce-xxx headers that are used by the binary mode. > > > > Why I am commenting this? > > > > As a side effect of the "binary" mode delivery, we must warrantee that > our > > services, like Data Index, and Jobs Service a prepared to understand > those > > events in both formats. > > > > Why? well, because if our architecture is deployed by using the knative > > eventing infrastructure (instead of kafka), those events will arrive in > > binary mode. > > > > Not sure if I could explain the "problem" clearly, but I'm happy to do a > > demo, or whatever of a full setup in the knative system, and show the > > involved components to ensure the events delivery, etc, np. > > > > Now, going the concrete situation, I think this last change in the events > > reception at the data-index: > > https://github.com/apache/incubator-kie-kogito-apps/pull/1881, is > > introducing an issue when we are in knative eventing system. (and we > > receive the events in the binary format) > > > > Before, far from perfect there was kind of "tricky" processing that tried > > to couple with both formats, and more or less worked. Now after the > change, > > I can see some errors when we are in knative. > > > > And in parallel, we had an open PR > > https://github.com/apache/incubator-kie-kogito-apps/pull/1889, with more > > improvements in particular on the treatment on this events to improve > this > > "tricky" code I was mentioning. > > > > So we had a kind of too many changes at the same piece of the pie, > > basically. I think > > https://github.com/apache/incubator-kie-kogito-apps/pull/1889 should > > probably have been merged first since it provided an improvement on the > > binary/structured, and thus, cloud be used for the processing of the new > > events defined in the other PR. > > > > To be clear, it's no a complain, but I wanted to introduce the topic > > because I think it's important to keep in mind that both "binary" and > > "structured" mode for future changes. > > > > I'll now try to resolve all the merge conflicts on the PR and see how to > be > > sure both modes continue working. > > >
