On Wed, Sep 13, 2017, at 09:01, Mike Drob wrote: > Hi folks, > > Want to pull this conversation back into people's minds to summarize and > reposit some questions. > > It looks like there is consensus that 1) HTrace is generally useful but > 2) > needs to refocus on more specific use cases rather than trying to be a > general purpose tool and 3) suffers from being niche on niche. > > There have been lots of folks volunteering to contribute, but I don't see > any resulting JIRA or mailing list activity. We all can have the best of > intentions while still not having the necessary time available to follow > through, myself included. > > Projects can still be successful outside of the ASF; retiring as a > podling > does not preclude others from picking up the works. As one example, I > have > seen many vibrant communities run as github projects with a core set of > dedicated contributors. I don't believe that living in the ASF makes it > any > easier or harder for other Apache projects such as Hadoop to integrate > with > us, so long as the licensing remains compatible. In the future, if a > diverse community springs up again and finds the desire to come back, I > don't think that will be a problem to re-incubate at that point either.
Hi Mike, I'm on vacation this week, so I apologize for the brevity (and delayed responses). I think it would make more sense to fold HTrace into Hadoop if we wanted to disband the podling. At a minimum, there is value in allowing Hadoop to use the trace systems that are out there (Zipkin, OpenTracing, etc.) in a software-configurable way. But I still hope we can find people interested in moving tracing forward in hadoop. Let's circle back on this in a little bit... Colin > > Thoughts? > Mike > > > On Thu, Aug 31, 2017 at 3:32 AM, Raam Rosh-Hai <[email protected]> > wrote: > > > Thank you Colin for your response, > > I was actually referring to Adrian's comment on making Htrace and Zipkin > > play more nicely together, I have been using Htrace with zipkin for a while > > now and I am doing pretty well but I was wondering if there are some > > glaring pain points that should be attended. > > > > On 30 August 2017 at 23:41, Colin McCabe <[email protected]> wrote: > > > > > On Mon, Aug 21, 2017, at 02:09, Raam Rosh-Hai wrote: > > > > I tend to agree that zipkin should be used as a frontend and I am > > pretty > > > > sure I will have some time to advance that in the coming weeks if one > > of > > > > the experts could create a few tasks. > > > > > > Hi Raam, > > > > > > If you're interested in this, check out the Zipkin trace sink. That > > > allows you to send HTrace spans to Zipkin. > > > > > > > > > > > I think no one touched the obvious thing that tracing lacks any kind of > > > > hype, it's just not glamours and in order to want to use such a > > framework > > > > you need to be versed in this world and to actually suffer from > > > > distributed > > > > debugging. > > > > > > Yeah, I think this is an area where some publicity and outreach would be > > > a good thing :) > > > > > > It would help a lot if we could identify some core use-cases, like Stack > > > said, and make sure everything works great for those. > > > > > > Colin > > > > > > > > > > > > > > so yes, I think this project deserves one last push before putting it > > in > > > > the attic and I would be happy to complete a few issues in order to > > make > > > > this happen. > > > > > > > > > > > > > > > > On 19 August 2017 at 03:55, Adrian Cole <[email protected]> > > wrote: > > > > > > > > > > Thanks Adrian for the editorial on the landscape. Helps, especially > > > > > coming > > > > > > from yourself. > > > > > we aim to please > > > > > > > > > > > Given current state of the project, a retrofit to come up on OT is > > > not > > > > > the > > > > > > solution to the topic-at-hand (and besides I have a colored opinion > > > on > > > > > > taking on the API of another after spending a bunch of time > > recently > > > > > > undoing our mistake letting third-party Interfaces and Classes show > > > > > through > > > > > > in hbase). > > > > > sensible for any api highly disconnected from the ecosystem, > > > > > especially without practice yet. > > > > > > > > > > > I appreciate the higher-level point made by Andrew, that it is hard > > > to > > > > > > thread a cross-cutting library across the Hadoop landscape whether > > > > > because > > > > > > releases happen on the geologic time scale or that there is little > > > by way > > > > > > of coordination. > > > > > I think this is indeed leading a path towards focus, eg the H in > > > Htrace :) > > > > > > > > > > > Can we do a focused 'win' like Colin suggests? E.g. hook up hbase > > and > > > > > hdfs > > > > > > end-to-end with connection to a viewer (zipkin? Or text dumps in a > > > > > > webpage?). A while back I had a go at the hbase side but it was > > > burning > > > > > up > > > > > > the hours just getting it hooked up w/ tests to scream if any spans > > > were > > > > > > broken in a refactor. I had to put it aside. > > > > > Incidentally, I wouldn't necessarily say Zipkin is ready out of box > > > > > because htrace UI and query is more advanced (in some ways due to > > some > > > > > data storage options we have available). So, something like this > > could > > > > > be a move of focus which would require investment on the other side > > to > > > > > avail features needed, or discuss how to upgrade into them (ex if > > > > > using hbase storage, certain queries would work). It is fair to say > > > > > zipkin has a great devops pipeline, we are good at fixing things. At > > > > > the same time, we are imperfect in impl and inexperienced in hadoop > > > > > ecosystem. Having some way to join together could be really > > > > > beneficial, at the cost of up-front effort (due to model, UI and > > > > > storage differences). I would be happy to direct time, though would > > > > > need some help because of my irrelevance in the data services space > > > > > (something this might correct!) > > > > > > > > > > > Like the rest of you, my time is a little occupied elsewhere these > > > times > > > > > so > > > > > > I can't revive the project, not at the moment at least. > > > > > ack > > > > > > > > > >
