Thanks for the context. I'll be posting later today so I'd appreciate any feedback you have there as well.
I agree with the caveat about bridging C++<->Python. I don't think this is accounted for upstream, IIRC the best you can do is ship a span context across the FFI boundary, and otherwise just use each language's respective libraries. But if pyarrow handled that for you it shouldn't be too different than having it all in Python in the first place. Best, David On 2021/04/29 20:05:38, Eric Henry <[email protected]> wrote: > Thanks for the response David. That proposal sounds exactly what I'm > looking for in terms of middleware. > > My motivation for tracing comes from previously working within a complex > real time orchestration system with 20+ microservices involved in creating > various features per request in the system. Tracing is a must or you end up > not being able to determine which microservice is the culprit for high > latency requests - and even more granularly - which part of the offending > microservice is slow (e.g. feature preprocessing, exception handling, etc.) > > OpenTelemetry implemented at the c++ level sounds awesome with the caveat > that it would be great to have some way to create additional spans and > inject headers into external calls from python space (e.g. i might want a > span for my feature preprocessing, a span for getting data from another > microservice, etc.). > > Thanks, > Eric > > On Thu, Apr 29, 2021 at 3:16 PM David Li <[email protected]> wrote: > > > Hey Eric, > > > > It so happens that I am literally in the middle of drafting a proposal > > to the mailing list to integrate with OpenTelemetry on the C++ > > level. I would fully support having these capabilities for Flight, and > > I think in the past we've discussed having a contrib directory for > > such modules, if we don't directly just integrate it into Arrow > > itself. While gRPC in theory has some of this built in, either way > > we'd need to expose it (and gRPC uses OpenCensus which is being phased > > out in favor of OpenTelemetry anyways). > > > > The usecase here is fairly obvious, I think - but it'd be great if we > > could hear more about how OpenTracing is useful for you/your > > org. Personally, at my past job, it was a while before people really > > grasped the value of distributed tracing & structured logging. > > > > Also, side note - I've used structlog in Python for JSON logging in > > the past and it was fantastic, I would definitely recommend that over > > just dumping JSON into a logger yourself. > > > > Best, > > David > > > > > > On 2021/04/29 18:59:30, Eric Henry <[email protected]> wrote: > > > Is there currently any interest or effort underway in providing reusable > > > Flight middleware for observability and production usage (e.g. logging, > > > tracing, metrics)? > > > > > > For my own usage and experimentation I created a very basic json logging > > > and OpenTracing implementation, but I was wondering if there was anything > > > else out there (or in the works) that's production grade. > > > > > > Here's my code for reference: > > > https://github.com/ehenry2/xgbatch/blob/main/xgbatch/middleware.py > > > > > > Thanks, > > > Eric > > > > > >
