msck repair table internally

Hi all, May be it is a common knowledge. Can anyone tell whether "msck repair table" also spawns multiple mappers like entity (or multiple processes) to perform. We have a large external table to migrate from one hive to another, hence we need to create it in the new hive and t

Re: adding parquet partitions to external table (msck repair table not working)

On Sun, Dec 30, 2018 at 11:08 PM Colin Williams > wrote: >> >> Hello, >> >> I have a parquet datasource that I'm writing via spark. For some >> reason this particular source will not pick up added partitions with >> msck repair table. However if I alter table tablen

Re: adding parquet partitions to external table (msck repair table not working)

riting via spark. For some > reason this particular source will not pick up added partitions with > msck repair table. However if I alter table tablename / add partition > (key=value) then it works. However this is more cumbersome than msck > repair table. > > Is there an alternative

adding parquet partitions to external table (msck repair table not working)

Hello, I have a parquet datasource that I'm writing via spark. For some reason this particular source will not pick up added partitions with msck repair table. However if I alter table tablename / add partition (key=value) then it works. However this is more cumbersome than msck repair table

Re: msck repair table and hive v2.1.0

Hi Rajesh, sure. i'll give that setting a try. thanks. re: s3 vs. hdfs. indeed. I figured i'd eliminate the s3 angle when posting here given the msck repair table failed in both cases. but yeah my real use case is using s3. ok. just tried that setting and got a slightly different stack trace

Re: msck repair table and hive v2.1.0

> > > so essentially it does what msck repair tables does but in a non-portable > way. oh well. gotta do what ya gotta do. > > On Wed, Jul 13, 2016 at 9:29 PM, Stephen Sprague <sprag...@gmail.com> > wrote: > >> hey guys, >> i'm using hive version 2.1.0 and i

Re: msck repair table and hive v2.1.0

but in a non-portable way. oh well. gotta do what ya gotta do. On Wed, Jul 13, 2016 at 9:29 PM, Stephen Sprague <sprag...@gmail.com> wrote: > hey guys, > i'm using hive version 2.1.0 and i can't seem to get msck repair table to > work. no matter what i try i get the 'ol NPE. I

msck repair table and hive v2.1.0

hey guys, i'm using hive version 2.1.0 and i can't seem to get msck repair table to work. no matter what i try i get the 'ol NPE. I've set the log level to 'DEBUG' but yet i still am not seeing any smoking gun. would anyone here have any pointers or suggestions to figure out what's going wrong

hive 0.13 msck repair table only lists partitions not in metastore

Hi, I'm trying to use Hive(0.13) msck repair table command to recover partitions and it only lists the partitions not added to metastore instead of adding them to metastore as well. here's the ouput of the command partitions not in metastore externalexample:CreatedAt=26 04%3A50%3A56 UTC 2014

MSCK REPAIR TABLE

Hey everyone, I have a table with currently 5541 partitions. Daily there are 14 partitions added. I will switch the update for the metastore from msck repair table to alter table add partition, since its performing better, but sometimes this might fail, and i need the msck repair table command

Re: MSCK REPAIR TABLE

can you grab more logs from hiveserver2 log file? On Thu, Mar 27, 2014 at 2:31 PM, fab wol darkwoll...@gmail.com wrote: Hey everyone, I have a table with currently 5541 partitions. Daily there are 14 partitions added. I will switch the update for the metastore from msck repair table

Re: MSCK REPAIR TABLE

everyone, I have a table with currently 5541 partitions. Daily there are 14 partitions added. I will switch the update for the metastore from msck repair table to alter table add partition, since its performing better, but sometimes this might fail, and i need the msck repair table command

Re: MSCK REPAIR TABLE

there are 14 partitions added. I will switch the update for the metastore from msck repair table to alter table add partition, since its performing better, but sometimes this might fail, and i need the msck repair table command. But unfortunately its not working anymore with this table size it seems: 0

Re: MSCK REPAIR TABLE

this all tasktrackers are deleted and newly setup (HDFS and storage is not touched at all, neither are configs). After that the msck repair table stmt is going well and its actually not so slow at all, as i thought it might be (ca. 110 secs per table). I guess there might have been some

msck repair table not adding partitions which contains data.

in /test /test/part=1 /test/part=1/1.txt /test/part=2 /test/part=2/1.txt /test/part=3 /test/part=4 Now I try to automatically add the partitions into the table using the 'msck repair' command hive msck repair table test_part; OK Partitions not in metastore:test_part:part=3

Re: msck repair table not adding partitions which contains data.

.txt /test/part=3 /test/part=4 ** ** Now I try to automatically add the partitions into the table using the 'msck repair' command hive msck repair table test_part; *OK* *Partitions not in metastore:test_part:part=3test_part:part=4* *Repair: Added

RE: msck repair table not adding partitions which contains data.

Thanks Mark From: Mark Grover [mailto:grover.markgro...@gmail.com] Sent: Thursday, February 07, 2013 2:54 PM To: user@hive.apache.org Subject: Re: msck repair table not adding partitions which contains data. Suresh, Take a look at this: https://issues.apache.org/jira/browse/HIVE-3231 On Thu, Feb