Thanks, that's very useful. On Wed, Dec 23, 2020, 12:46 PM Jonathan Eagles <[email protected]> wrote:
> This feature was in progress for some time but progress was halted. > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/TEZ-3820 > > Many of the stakeholders can be found on that ticket > > The issue regarding scale depends on usage. I run atsv1.5 and write > 300,000 history files a day. But they are written directly to hdfs and not > the timeline server REST api. The rest app cannot handle the scale or > bandwidth after around 8000 jobs per day IIRC. Reading from the timeline > server has similar issues. We serve the tez UI, but bulk history files are > parsed into a hive dB for serious queries. > > On Tue, Dec 22, 2020, 11:33 PM Prabhu Joseph <[email protected]> > wrote: > >> Hi All, >> >> I am trying to integrate TEZ with ATSv2 as we are hitting scalability >> issues with ATS1.5. Looks we need a HistoryLoggingService implementation >> to >> emit Tez HistoryEvents from Tez jobs to ATSv2 and change in Tez UI code to >> use ATS2 REST API to fetch timeline events. >> >> Want to check if someone has already done this, if so can you share the >> same. >> >> Thanks, >> Prabhu Joseph >> >
