[
https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659336#comment-13659336
]
Sanjay Radia commented on HADOOP-9517:
--------------------------------------
During the discussions on compatibility I had proposed up the following for
data compatibility (I found these in my old notes and should be in some email
thread).
* Data Compatibility
** HDFS metadata and data can change across minor or major releases , but such
changes are transparent to user application. A release upgrade must
automatically convert the metadata and data as needed. Further, a release
upgrade must allow a cluster to roll back to the older version and its older
disk format. (rollback needs to restore the original data but not any updated
data).
Motivation: Users expect File systems preserve data transparently across
releases.
** Stronger version of the above
HDFS metadata and data can change across minor or major releases, but such
changes are transparent to user application. A release upgrade must
automatically convert the metadata and data as needed. During *minor* releases,
disk format changes have to backward and forward compatible; i.e. an older
version of Hadoop can be started on a newer version of the disk format. Hence
a version roll back is simple, just restart the older version of Hadoop.
Major releases allow more significant changes to the disk format and have be
only backward compatible; however major release upgrade must allow a cluster to
roll back to the older version and its older disk format.
With this minor release are very easy to roll back for an admin.
Note this will restrict the kinds of changes that be made in minor releases.
** Weaker: Limited Automatic Conversion:
HDFS metadata and data can change across minor or major releases , but such
changes are transparent to user application. A release upgrade must
automatically convert the metadata and data as needed, but automatic conversion
is supported across a small number of releases. If a user
wants to jump across multiple releases he may be forced to go through a few
intermediate release to get to the final desired release. Further, a release
upgrade must allow a cluster to roll back to the older version and its older
disk format. (rollback needs to restore the original data but not any updated
data).
Automatic conversion is support across a small number of releases. If a user
wants to jump across multiple releases he may be forced to go through a few
intermediate release to get to the final desired release.
We currently support the weaker automatic conversion in HDFS.
> Define Hadoop Compatibility
> ---------------------------
>
> Key: HADOOP-9517
> URL: https://issues.apache.org/jira/browse/HADOOP-9517
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
> Reporter: Arun C Murthy
> Assignee: Karthik Kambatla
> Attachments: hadoop-9517.patch, hadoop-9517.patch, hadoop-9517.patch,
> hadoop-9517.patch
>
>
> As we get ready to call hadoop-2 stable we need to better define 'Hadoop
> Compatibility'.
> http://wiki.apache.org/hadoop/Compatibility is a start, let's document
> requirements clearly and completely.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
