Since it is per a file, you'd need to check at file create too.
-- Owen
On Mar 16, 2009, at 4:29, Steve Loughran <[email protected]> wrote:
Steve Loughran wrote:
Owen O'Malley wrote:
I seem to remember someone saying that blocks over 2^31 don't
work. I don't know if there is a jira already.
Looking at the stack trace, int is being used everywhere, which
implies an upper limit of (2^31)-1, for blocks. Easy to fix, though
it may change APIs, and then there is the testing.
thinking about this a bit more, a quick early patch would be to
print a warning whenever you try to bring up a namenode with a block
size >= 2GB ; have the system continue so that people can test and
fix the code, but at least it stops end users being surprised.
I spoke with someone from the local university on their High Energy
Physics problems last week -their single event files are about 2GB,
so that's the only sensible block size to use when scheduling work.
He'll be at ApacheCon next week, to make his use cases known.
-steve
