On 22 April 2013 14:00, Karthik Kambatla <ka...@cloudera.com> wrote:
> Hadoop devs,
>
>
> This doc does not intend to propose new policies. The idea is to have one
> document that outlines the various compatibility concerns (lots of areas
> beyond API compatibility), captures the respective policies that exist, and
> if we want to define policies for the items where it’s not clear we have
> something to iterate on.
>
> The first draft just lists the types of compatibility. In the next step, we
> can add existing policies and subsequently work towards policies for
> others.
>
>
I don't see -yet- a definition of compatible at the API signature level vs
semantics level.
The @ interface attributes say "these methods are
internal/external/stable/unstable (there's also @VisibleForTesting,that
comes out of guava (yes?).
There's a separate issue that says "we make some guarantee that the
behaviour of a interface remains consistent over versions", which is hard
to do without some rigorous definition of what the expected behaviour of an
implementation should be. Even then, there are epiphenomena whose behaviour
isn't a direct part of the specification but which people ended up relying
on. As an example, look how many junit tests have broken in Java7 after
they changed the order in which methods were enumerated. The new ordering
fulfils the java lan specification "no order is guaranteed", but the
pre-java6 implementation offered "same order as you wrote them in the
file". In Hadoop, there's a lot of code that assumes that close() of an
output stream happens within a short period of time, and if you change that
some things break, but nowhere does {{OutputStream.close()}} ever say
"completes within 30s".
Interface semantics are far more important than simple signatures, and they
are the hardest to guarantee -especially when we don't have rigorous enough
definitions of what implementations should really do.
As an example what does the method Seekable.seek() do?
It's header says
/**
* Seek to the given offset from the start of the file.
* The next read() will be from that location. Can't
* seek past the end of the file.
*/
void seek(long pos) throws IOException;
Now, without looking at the source, what do you think the reference
implementation - BufferedFSInputStream - does on any of the sequences:
seek(-1); read()
seek(0); seek(0)
close(); seek(0)
The javadocs don't specify what happens, the implementations don't behave
in ways you'd expect, and that leaves a problem: do you now define the
semantics of seek more rigorously -and fix our implementations -or do you
conclude that some people may expect the current behaviour on negative
seeks (they are ignored, for the curious), and so we can't actually change
them.
