Re: Mandarin Hadoop online sync this week
This week's summary: 8/26 Mandarin online sync Weichiu, Xiaoiao, Baoloongmao, Hui, wuweiwei,, Leon Gao, Lisheng Sun, Jinglun, zhoubin86 Leon shared a DataNode improvement proposal at Uber. Different storage density. Balance disk IO among different disk size. Problem: archive disk’s IO utilization is very low. Want to use it more. The proposed change will be based on the HSM, with quite minimal change. Cold data is in GCS. A simple scheme to copy cold data to GCS. The data in GCS is not intended to be accessible readily, so don’t worry about the scheme change. Jinglun shared the solutions to an operational problem: NameNode QPS dropped, waiting time more than 1 second, processing time more than 400ms. Solution: (1) migrate a directory to a new namespace. (2) RBF can hash out a directory to multiple namespaces, reducing the pressure of a particular NN. Baoloongmao suggested we can port Ozone features into Hadoop Common. For example, Java-based configuration is a power feature which can benefit Hadoop as well. On Tue, Aug 25, 2020 at 9:47 AM Wei-Chiu Chuang wrote: > Hello, > > There hasn't been a Mandarin online sync for quite some time. I'd like to > call for one this week: > > Date/time: > > 8/27 Thursday Beijing Time 1PM > 8/26 Wednesday US Pacific Time 10PM > > Link: > https://cloudera.zoom.us/j/880548968 > > Past sync summary: > > https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit > > >
Re: Mandarin Hadoop online sync this week
Brahma, Thanks for bring this up. I don't have a specific topic in mind. For me, I'd like to use this as an opportunity to get a feeling of what features/problems the community feel interested in. And in general, what are the Hadoop versions the community are using? Can we stop 2.9.x development and concentrate on 3.2/3.3 and trunk? I am interested in arranging a meetup for a broader audience in English for topics like this soon too. Other things I'd like to explore (if there's time), is how do we involve the Chinese community better. How can we grow more Chinese/Asian committers? On Tue, Aug 25, 2020 at 9:58 AM Brahma Reddy Battula wrote: > HI, > > what you are planning for this week? > > On Tue, Aug 25, 2020 at 10:18 PM Wei-Chiu Chuang > wrote: > >> Hello, >> >> There hasn't been a Mandarin online sync for quite some time. I'd like to >> call for one this week: >> >> Date/time: >> >> 8/27 Thursday Beijing Time 1PM >> 8/26 Wednesday US Pacific Time 10PM >> >> Link: >> https://cloudera.zoom.us/j/880548968 >> >> Past sync summary: >> >> https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit >> > > > -- > > > > --Brahma Reddy Battula >
Re: Mandarin Hadoop online sync this week
HI, what you are planning for this week? On Tue, Aug 25, 2020 at 10:18 PM Wei-Chiu Chuang wrote: > Hello, > > There hasn't been a Mandarin online sync for quite some time. I'd like to > call for one this week: > > Date/time: > > 8/27 Thursday Beijing Time 1PM > 8/26 Wednesday US Pacific Time 10PM > > Link: > https://cloudera.zoom.us/j/880548968 > > Past sync summary: > > https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit > -- --Brahma Reddy Battula
Mandarin Hadoop online sync this week
Hello, There hasn't been a Mandarin online sync for quite some time. I'd like to call for one this week: Date/time: 8/27 Thursday Beijing Time 1PM 8/26 Wednesday US Pacific Time 10PM Link: https://cloudera.zoom.us/j/880548968 Past sync summary: https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit