On Wed, Feb 22, 2012 at 7:51 PM, Vinod Kumar Vavilapalli <vino...@hortonworks.com> wrote: > Todd, > > From your analysis at HDFS-2991, looks like this was there in 0.23 > too. Also, seems this happens only at scale, and only (paraphrasing > you) "when the file is reopened for append on an exact block > boundary".
Let me clarify: HDFS-2991 basically has two halves: First half (been present "forever"): when we append() on a block boundary, we don't log an OP_ADD Second half (new due to HDFS-2718): if we get an OP_CLOSE for a file we haven't OP_ADDed, we'll get a ClassCastException on startup. So even though the first half isn't a regression, the regression in the second half means that this longstanding bug will now actually prevent startup. Also, there's nothing related to scale here. I happened to run into it doing scale tests, but it turned out to not be relevant. You'll see it if you run TestDFSIO with standard parameters on trunk or 23.1 (that's how I discovered it). > > Agree it is a critical fix, but given above, can we proceed along with > 0.23.1? Anyways, 0.23.1 is still an alpha (albeit of next level), so > I'd think we can get that in for 0.23.2. Alright, consider me -0, though it's pretty nasty once you run into it. The only way I could start my NN again without losing data was to recompile with the fix in place. -Todd > > On Wed, Feb 22, 2012 at 6:43 PM, Todd Lipcon <t...@cloudera.com> wrote: >> -1, unfortunately. HDFS-2991 is a blocker regression introduced in >> 0.23.1. See the JIRA for instructions on how to reproduce on the rc2 >> build. >> >> -Todd >> >> On Fri, Feb 17, 2012 at 11:23 PM, Arun C Murthy <a...@hortonworks.com> wrote: >>> I've created another release candidate for hadoop-0.23.1 that I would like >>> to release. >>> >>> It is available at: http://people.apache.org/~acmurthy/hadoop-0.23.1-rc2/ >>> The hadoop-0.23.1-rc2 svn tag: >>> https://svn.apache.org/repos/asf/hadoop/common/tags/release-0.23.1-rc2 >>> The maven artifacts for hadoop-0.23.1-rc2 are also available at >>> repository.apache.org. >>> >>> Please try the release and vote; the vote will run for the usual 7 days. >>> >>> thanks, >>> Arun >>> >>> -- >>> Arun C. Murthy >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera -- Todd Lipcon Software Engineer, Cloudera
