yep, the good part here is that considering that this converter in invoked only when the quarkus-http conector is present, we can use Vertx classes that already deals with the ce-myExtension1, ce-myExtension2, that was a kind of change that was about to be applied at the time this PR new code came in.
https://github.com/apache/incubator-kie-kogito-apps/pull/1889/files#diff-0e2dab7d52fd876281f39dcfdbffcdd0d02d32eaef774fdf300b5505dfef7bd7 El mar, 24 oct 2023 a las 16:16, Enrique Gonzalez Martinez (< [email protected]>) escribió: > Ok, so basically what we want is different kind of processing when http is > used as this feature unwraps the data within cloud event and send the cloud > attribute as headers. > > El mar, 24 oct 2023, 16:02, Walter Medvedeo <[email protected]> > escribió: > > > Hi Enrique, > > in general knative eventing deliver the events to a target service in a > > format like this: > > > > > > > https://github.com/cloudevents/spec/blob/main/cloudevents/bindings/http-protocol-binding.md#31-binary-content-mode > > > > See the example below, and what we have is that all extensions that you > > have in the cloud event are send as http-headers, like > > ce-myExtension1 > > ce-myExtension2 > > > > etc. > > > > > > This is why if we do > > > > objectMapper.readValue(message.getPayload().toString(), MyClass.class) > > > > It might happen that some values are not cached up, because all the cloud > > event extensions goes as headers and not as part of the > > message.getPayload(). > > > > > > HTTP/1.1 200 OK > > ce-specversion: 1.0 > > ce-type: com.example.someevent > > ce-time: 2018-04-05T03:56:24Z > > ce-id: 1234-1234-1234 > > > > ce-myExtension1: my value1 > > > > ce-myExtension2: my value2 > > > > ce-source: /mycontext/subcontext > > .... further attributes ... > > Content-Type: application/json; charset=utf-8 > > Content-Length: nnnn > > > > { > > ... application data ... > > } > > > > > > > > > > Cheers, > > Walter. > > > > El mar, 24 oct 2023 a las 13:38, Enrique Gonzalez Martinez (< > > [email protected]>) escribió: > > > > > I think is fair point. From my perspective there is no problem. In the > > end > > > we just need to duplicate fields in what u call binary format and > unwrap > > > those changes in the data index to make it work. > > > > > > Unfortunately the events requires need the context of the event as it > is > > > big event was replace by raw diff events. > > > > > > My guess is that knative unwrap the cloud event and send that wrapped > > data > > > (otherwise) there is no eventpublisher which sends no cloud event > afaik.. > > > is that correct ? > > > > > > El mar, 24 oct 2023, 13:25, Walter Medvedeo <[email protected]> > > > escribió: > > > > > > > Hello Guys, > > > > > > > > I just wanted to raise up a tricky part of the cloud events that I > > think > > > > it's important to take into account when we introduce modifications > on > > > the > > > > events produced / received by our infrastructure in general. > > > > > > > > When we send cloud events over the wire, there are basically two > modes > > > that > > > > can be used to send them, "structured" and "binary", all good. In > > > general, > > > > every time our code send those events, it does by using the > > "structured" > > > > mode. And, in general, our services like the data-index, and the > > > > jobs-service will receive them in "structured" mode too. > > > > > > > > For instance, if we use kafka, and the process produces an event, > that > > > > event will be delivered in the "structured" mode to the data-index. > > > > > > > > BUT, there are some scenarios, where even when we produce these > events > > > in " > > > > structured" mode, we don't have 100% control, on which mode they will > > be > > > > delivered. That is the case of the knative eventing infrastructure. > > When > > > we > > > > have a installation that relies on knative eventing, very simplified, > > we > > > > have something like this: > > > > > > > > Process --> (produce event in "structured" mode) --> knative broker > > > > (receives the event) --> knative trigger (delivers the event in > > "binary" > > > > mode) --> Data Index (receives the event in binary mode) > > > > > > > > To deliver the originally event created in structured mode, in binary > > > mode > > > > instead, is something internal to the knative eventing system, and > they > > > > have good reasons for that. On reason, is for example, that by > managing > > > the > > > > events in the binary mode internally, it's easier to do > > filtering/routing > > > > operations, based on the ce-xxx headers that are used by the binary > > mode. > > > > > > > > Why I am commenting this? > > > > > > > > As a side effect of the "binary" mode delivery, we must warrantee > that > > > our > > > > services, like Data Index, and Jobs Service a prepared to understand > > > those > > > > events in both formats. > > > > > > > > Why? well, because if our architecture is deployed by using the > knative > > > > eventing infrastructure (instead of kafka), those events will arrive > in > > > > binary mode. > > > > > > > > Not sure if I could explain the "problem" clearly, but I'm happy to > do > > a > > > > demo, or whatever of a full setup in the knative system, and show the > > > > involved components to ensure the events delivery, etc, np. > > > > > > > > Now, going the concrete situation, I think this last change in the > > events > > > > reception at the data-index: > > > > https://github.com/apache/incubator-kie-kogito-apps/pull/1881, is > > > > introducing an issue when we are in knative eventing system. (and we > > > > receive the events in the binary format) > > > > > > > > Before, far from perfect there was kind of "tricky" processing that > > tried > > > > to couple with both formats, and more or less worked. Now after the > > > change, > > > > I can see some errors when we are in knative. > > > > > > > > And in parallel, we had an open PR > > > > https://github.com/apache/incubator-kie-kogito-apps/pull/1889, with > > more > > > > improvements in particular on the treatment on this events to improve > > > this > > > > "tricky" code I was mentioning. > > > > > > > > So we had a kind of too many changes at the same piece of the pie, > > > > basically. I think > > > > https://github.com/apache/incubator-kie-kogito-apps/pull/1889 should > > > > probably have been merged first since it provided an improvement on > the > > > > binary/structured, and thus, cloud be used for the processing of the > > new > > > > events defined in the other PR. > > > > > > > > To be clear, it's no a complain, but I wanted to introduce the topic > > > > because I think it's important to keep in mind that both "binary" and > > > > "structured" mode for future changes. > > > > > > > > I'll now try to resolve all the merge conflicts on the PR and see how > > to > > > be > > > > sure both modes continue working. > > > > > > > > > >
