dpol1 commented on issue #9016: URL: https://github.com/apache/storm/issues/9016#issuecomment-5677246381
I looked into this a while ago for StormCrawler, following one URL through fetch, parse, index and status with OpenTelemetry. I never proposed it, but some of it applies here. Carrying `traceparent` in the tuple payload only works as long as every bolt forwards it. A bolt that holds a tuple and emits later, or works on another thread, breaks the chain and nothing tells you. So I think this belongs in Storm, not in each topology. `ITaskHook` is not enough: `boltExecute` fires after `execute()` returns, too late to make the context current. On master I would start from `BoltExecutor.tupleActionFn` (span around `execute()`), `BoltOutputCollectorImpl.boltEmit` (inject into the emitted tuple), `SpoutOutputCollectorImpl` (root span) and `KryoTupleSerializer` (the context has to travel with the tuple). Open questions before any code: is an emitted tuple's parent the span of the `execute()` that emits it, with its anchors as span links? That should keep bolts that emit later or join inputs correct whatever the anchor order. Which sampling do you want, since a decision taken at the spout cannot keep every failed tuple? And does the tuple format change stay compatible when a cluster runs more than one Storm version (`supervisor.worker.version.classpath.map`)? I can write this up properly here and validate it on a real StormCrawler topology if native support is where you want to go. @AdarshDubey4, @rzo1, @reiabreu WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
