Hoping to merge HDFS-1073 branch soon

2011-06-30 Thread Todd Lipcon
Hey all, Work on the HDFS-1073 branch has been progressing steadily, and I believe we're coming close to the point where it can be merged. To briefly summarize the status: - NameNode and SecondaryNameNode are both fully working and have undergone some stress/fault testing in addition to a over

Re: Hadoop´s Internationalization

2011-06-30 Thread Eric Baldeschwieler
Do other apache projects have a good localization framework for error messages? I'd think there would be interest in starting this, although since we'd need to add the framework, this would be an investment. On Jun 25, 2011, at 1:38 AM, Owen O'Malley wrote: On Fri, Jun 24, 2011 at 7:55 PM,

Re: Hadoop´s Internationalization

2011-06-30 Thread Jakob Homan
I'd be hesitant about this given our experience with the Chinese version of the Hadoop documents. Picking the Chinese version of hdfs_quota_admin_guide.xml, as an example, shows that it has not actually been updated (beyond svn moves and copyright years) since its original commit back on 12/10/08

RE: Hadoop Java Versions

2011-06-30 Thread Evert Lammerts
You can get 12-24 TB in a server today, which means the loss of a server generates a lot of traffic -which argues for 10 Gbe. But -big increase in switch cost, especially if you (CoI warning) go with Cisco -there have been problems with things like BIOS PXE and lights out management

RE: Hadoop Java Versions

2011-06-30 Thread Evert Lammerts
That's not a question I'm qualified to answer. I do know we're now buying an Arista for a different cluster, but there's probably loads others out there. *forwarded to general@...* From: Abhishek Mehta [abhis...@tresata.com] Sent: Thursday, June 30, 2011

Re: Hadoop Java Versions

2011-06-30 Thread Aaron Eng
Keeping the amount of disks per node low and the amount of nodes high should keep the impact of dead nodes in control. It keeps the impact of dead nodes in control but I don't think thats long-term cost efficient. As prices of 10GbE go down, the keep the node small arguement seems less fitting.

Re: Hadoop Java Versions

2011-06-30 Thread Ted Dunning
You have to consider the long-term reliability as well. Losing an entire set of 10 or 12 disks at once makes the overall reliability of a large cluster very suspect. This is because it becomes entirely too likely that two additional drives will fail before the data on the off-line node can be

Re: Hadoop Java Versions

2011-06-30 Thread Todd Lipcon
On Thu, Jun 30, 2011 at 5:16 PM, Ted Dunning tdunn...@maprtech.com wrote: You have to consider the long-term reliability as well. Losing an entire set of 10 or 12 disks at once makes the overall reliability of a large cluster very suspect. This is because it becomes entirely too likely

Re: Hadoop Java Versions

2011-06-30 Thread Ted Dunning
Good point Todd. I was speaking from the experience of people I know who are using 0.20.x On Thu, Jun 30, 2011 at 5:24 PM, Todd Lipcon t...@cloudera.com wrote: On Thu, Jun 30, 2011 at 5:16 PM, Ted Dunning tdunn...@maprtech.com wrote: You have to consider the long-term reliability as well.

Re: Hadoop Java Versions

2011-06-30 Thread M. C. Srivas
On Thu, Jun 30, 2011 at 5:24 PM, Todd Lipcon t...@cloudera.com wrote: I'd advise you to look at stock hadoop again. This used to be true, but was fixed a long while back by HDFS-457 and several followup JIRAs. If MapR does something fancier, I'm sure we'd be interested to hear about it so

Re: Hadoop Java Versions

2011-06-30 Thread Ian Holsman
On Jul 1, 2011, at 2:08 PM, M. C. Srivas wrote: On Thu, Jun 30, 2011 at 5:24 PM, Todd Lipcon t...@cloudera.com wrote: I'd advise you to look at stock hadoop again. This used to be true, but was fixed a long while back by HDFS-457 and several followup JIRAs. If MapR does something

Re: Hadoop Java Versions

2011-06-30 Thread M. C. Srivas
No worries. I read Todd's post as asking for elaboration ... sometimes knowing what another similar system does helps in improving your own. On Thu, Jun 30, 2011 at 9:47 PM, Ian Holsman had...@holsman.net wrote: On Jul 1, 2011, at 2:08 PM, M. C. Srivas wrote: On Thu, Jun 30, 2011 at 5:24

Re: Hadoop Java Versions

2011-06-30 Thread Ted Dunning
Ian, Good point. Srivas was responding to Todd's question, but there might be better fora as you suggest. We have a good one for specific questions about MapR at http://answers.mapr.com That doesn't, however, really provide a useful forum for questions like Todd's which really spans both

Re: Hadoop´s Internationalization

2011-06-30 Thread Harsh J
Having spent some time in KDE before, proper intl. could only be achieved if string freeze dates are set and notifications are pushed out to translator teams for a translating period of newly exported strings (it could be an ongoing process, but before a release may be cut there ought to be a

Re: Hadoop´s Internationalization

2011-06-30 Thread Owen O'Malley
On Thu, Jun 30, 2011 at 9:06 AM, Eric Baldeschwieler eri...@yahoo-inc.comwrote: Do other apache projects have a good localization framework for error messages? Java has very good localization capabilities. However, it is a huge pervasive change if we want to get each and every user-facing