Re: Disk hot swap for data node while hbase use short-circuit

2019-06-01 Thread Josh Elser
Reminds me of https://issues.apache.org/jira/browse/HBASE-21915 too. Agree with Wei-Chiu that I'd start by ruling out HDFS issues first, and then start worrying about HBase issues :) On 6/1/19 8:05 PM, Wei-Chiu Chuang wrote: I think i found a similar bug report that matches your symptom: HDFS-

Re: Scan vs TableInputFormat to process data

2019-06-01 Thread Josh Elser
Hi Guillermo, Yes, you are missing something. TableInputFormat uses the Scan API just like Spark would. Bypassing the RegionServer and reading from HFiles directly is accomplished by using the TableSnapshotInputFormat. You can only read from HFiles directly when you are using a Snapshot, as t

Re: Disk hot swap for data node while hbase use short-circuit

2019-06-01 Thread Wei-Chiu Chuang
I think i found a similar bug report that matches your symptom: HDFS-12204 (Dfsclient Do not close file descriptor when using shortcircuit) On Wed, May 29, 2019 at 11:37 PM Kang Minwoo wrote: > I think these file opened for reads. because that b