HI Stack, Thanks for the script. The cluster is in use and I am not comfortable modifying that big table right now. (only .001% data will be missing, which is ok for now) . I will definitely try the script some time this week and will post you. Regarding the table, I have 8 region servers (each hosting 5.5k). Each using 2.3G heap. I have jetty in front of this to serve data, yet to benchmark it for performance. Will definitely post you the metrics soon. Thanks for the upload tool, couldn't think of loading such a table earlier , I indeed tried and gave up tableoutputfornmat idea some time back :)
Thanks, Murali Krishna ________________________________ From: stack <[email protected]> To: [email protected] Sent: Mon, 9 November, 2009 9:12:55 PM Subject: Re: Issue with bulk loader tool On Mon, Nov 9, 2009 at 2:26 AM, Murali Krishna. P <[email protected]>wrote: > hi Stack, > I attached the changes to that patch to the jira. Thanks for fixing my hackup. > Is it possible correct an existing table created via old loadtable? I > have a table with few billon records and 40k regions, took almost 2 days for > the MR to create this. Don't want to do it again :) > > 40k regions is good going. How many per regionserver? See the script bin/add_table.rb. See how the first thing it does is delete old table. I'd suggest you comment out the moving aside of a table directory at line #72 and then all from line #103 on where it adds into .META. the new table. I uploaded to the issue 1962 a version with commented out sections. Review before running. St.Ack
