I wonder if it would be better to just rip out tracing in 2.0. I know it's a big leap... but I really don't know anybody using it regularly, and given the potential compatibility problems, potential impact of migrating, and general lack of care from the HTrace project for API compatibility, as well as a myriad of various tracing-related bugs that are hard to fix (ACCUMULO-4415 off the top of my head, but I think there's also generally a problem of unrooted traces due to trace-unaware executors and similar problems).
Ripping it out of 2.0 could give us some breathing room to do some internal modularization/refactoring/cleaning up code debt, before potentially reintroducing it more carefully in the future. I'm not saying I like this option... but it might be worth considering. On Tue, Feb 27, 2018 at 1:09 PM Ed Coleman <d...@etcoleman.com> wrote: > For general discussion - Facebook recently (Oct 28, 2017) published a > paper on tracing: Canopy: An End-to-End Performance Tracing and Analysis > System ( > https://research.fb.com/publications/canopy-end-to-end-performance-tracing-at-scale/ > ) > > As a bonus, they referenced Accumulo and HTrace in section 2.2 > > "Mismatched models affected compatibility between mixed system versions; > e.g. Accumulo and Hadoop were impacted by the “continued lack of concern in > the HTrace project around tracing during upgrades” > > > -----Original Message----- > From: Tony Kurc [mailto:tk...@apache.org] > Sent: Tuesday, February 27, 2018 12:57 PM > To: dev@accumulo.apache.org > Subject: Re: [DISCUSS] tracing framework updates > > I have some experience with opentracing, and it definitely seems > promising, however, potentially promising in the same way htrace was... > That being said, I did a cursory thought exercise of what it would take to > do a swap of the current tracing in accumulo to opentracing, and I didn't > come across any hard problems, meaning it could be a fairly straightforward > refactor. I was hoping to explore the community a bit more at some upcoming > conferences > > On Feb 27, 2018 11:59 AM, "Sean Busbey" <bus...@apache.org> wrote: > > > > > > > On 2018/02/27 16:39:02, Christopher <ctubb...@apache.org> wrote: > > > I didn't realize HTrace was struggling in incubation. Maybe some of > > > us > > can > > > start participating? The project did start within Accumulo, after all. > > What > > > does it need? I also wouldn't want to go back to maintaining > cloudtrace. > > > > > > > I suspect it's too late for HTrace. The last commit to the main > > development branch was May 2017. They had a decent run of activity in > > 2015 and an almost-resurgence in 2016, but they never really got > > enough community traction to survive the normal ebb and flow of > > contributor involvement. > > > > They need the things any project needs to be sustainable: regular > > release cadences, a responsive contribution process, and folks to do > > the long slog of building interest via e.g. production adoption. > > > > > I'm unfamiliar with OpenTracing, but it was my understanding that > > > Zipkin was more of a tracing sink, than an instrumentation API. > > > HTrace is > > actually > > > listed as an instrumentation library for Zipkin (among others). > > > > > > > I think the key is that for a instrumentation library to get adoption > > it needs a good sink that provides utility to operators looking to > > diagnose problems. It took too long for HTrace to provide any tooling > > that could help with even simple performance profiling. Maybe hooking > > it into Zipkin would get around that. Personally, I never managed to > > get the two to actually work together. > > > > My listing Zipkin as an option merely reflects my prioritization of > > practical impact of whatever we go to. I don't want to adopt some > > blue-sky effort. FWIW, OpenTracing docs at least claim to also provide > > a zipkin-sink compatible runtime. > > > > There's a whole community that just does distributed monitoring, maybe > > someone has time to survey some spaces and see if OpenTracing has any > legs. > > > >
