Is shortcircuit-read (SCR) really fast?

2019-08-29 Thread Daegyu Han
Hi all, Is ShortCircuit read faster than legacy read which goes through data nodes? I have evaluated SCR and legacy local read on both HDD and NVMe SSD. However, I have not seen any results that SCR is faster than legacy. Rather, SCR was slower than legacy when using NVMe SSDs because of the op

Re: Is shortcircuit-read (SCR) really fast?

2019-08-30 Thread Wei-Chiu Chuang
Interesting benchmark. Thank you, Daegyu. Can you try a larger file too? Like 128mb or 1gb? HDFS is not optimized for smaller files. What did you use for benchmark? Daegyu Han 於 2019年8月29日 週四,下午11:40寫道: > Hi all, > > Is ShortCircuit read faster than legacy read which goes through data nodes? > >

Re: Is shortcircuit-read (SCR) really fast?

2019-09-04 Thread Wei-Chiu Chuang
Hi Daegyu, let's move this discussion to the user group, so that any one else can comment on this. I obviously don't have the best answers to the questions. But these are great questions. Re: benchmarks for SCR: I believe yes. In fact, I found a benchmark running Accumulo and HBase on HDFS http://