Re: Number of data files and opened file descriptors are not decreasing after DROP TABLE.

2017-04-27 Thread Todd Lipcon
On Mon, Apr 24, 2017 at 8:06 PM, Jason Heo wrote: > Thanks David > > Hi Mike. I'm using Kudu 1.3.0 bundled in "Cloudera Express 5.10.0 (#85 > built by jenkins on 20170120-1037 git: aa0b5cd5eceaefe2f971c13ab65702 > 0d96bb842a)" > > My concern is that something does not free up cleanly and somethin

Re: Number of data files and opened file descriptors are not decreasing after DROP TABLE.

2017-04-24 Thread Jason Heo
Thanks David Hi Mike. I'm using Kudu 1.3.0 bundled in "Cloudera Express 5.10.0 (#85 built by jenkins on 20170120-1037 git: aa0b5cd5eceaefe2f971c13ab657020d96bb842a)" My concern is that something does not free up cleanly and something wastes of my resources. eg) I dropped a 30TB table, but in tabl

Re: Number of data files and opened file descriptors are not decreasing after DROP TABLE.

2017-04-24 Thread Mike Percy
HI Jason, I would strongly recommend upgrading to Kudu 1.3.1 as 1.3.0 has a serious data-loss bug related to re-replication. Please see https://kudu.apache.org/ releases/1.3.1/docs/release_notes.html (if you are using the Cloudera version of 1.3.0, no need to worry because it includes the fix for t

Number of data files and opened file descriptors are not decreasing after DROP TABLE.

2017-04-24 Thread davidralves
Hi Jason Adar or Mike Percy know a lot about the fs might be able to help. Adar, Mike, can one of you please weigh in? Best David

Number of data files and opened file descriptors are not decreasing after DROP TABLE.

2017-04-23 Thread Jason Heo
Hi. Before dropping, there were about 30 tables, 27,000 files in tablet_data directory. I dropped most tables and there is ONLY one table which has 400 tablets in my test Kudu cluster. After dropping, there are still 27,000 files in tablet_data directory, and output of /sbin/lsof is the same befo