Raghu Angadi wrote:
Is it there also that we would find what is involved making append
work in 0.19.1?
If one knew what is enough to fix properly, it would be easy. But over
last couple of months, there have been many fixes (some of these jiras
are listed in one Konstantins HADOOP-4663).
Konstantin's comment I referred to (it was also linked from HADOOP-4663,
but harder to find).
https://issues.apache.org/jira/browse/HADOOP-5027#action_12668136
Raghu.
The discussions are still
bringing up more cases where the implementation or algorithm should
change. But these are improvements for sure. But doubt if I would be
ready to call it is 'completely fixed'. It needs time and a lot of
testing in large clusters.
Personally I am +1 for getting these into 0.19 branch. Most importantly
even clusters and application not using append or sync were also
affected, thats why extra caution.
my 2 cents. hope this does not digress too much from the main topic.
Raghu.
Thanks,
St.Ack
On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[email protected]>
wrote:
Nigel Daley wrote:
Folks,
Some Hadoop deployments have upgraded to 0.19.0. Clearly, the 0.19
branch
has issues and a 0.19.1 release is needed.
Quality issues in the changes made for the file append feature have
prevented some from deploying Hadoop 0.19. One of these changes
(sync) has
now been "fixed" by reducing its semantics in Hadoop 0.18.3
(HADOOP-4997).
This was necessary to stabilize the 0.18 branch.
I would like to propose that we apply this same "fix" to sync in 0.19.1
and 0.20.0. Since append requires the full semantics of sync, I
propose we
also disable append (perhaps throw UnsupportedOperationException
from API?).
Yes, this would unfortunately be an incompatible change between
0.19.0 and
0.19.1. We can then take the time needed to fix append properly in
0.21.0.
I can see some people being unhappy about this, but giving them a choice
between having the filesystem work or not, hopefully they will see the
merits of the change. And I am +1 to taking time to fix things; fast
fixes
often create new problems
