[ https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659336#comment-13659336 ]
Sanjay Radia commented on HADOOP-9517: -------------------------------------- During the discussions on compatibility I had proposed up the following for data compatibility (I found these in my old notes and should be in some email thread). * Data Compatibility ** HDFS metadata and data can change across minor or major releases , but such changes are transparent to user application. A release upgrade must automatically convert the metadata and data as needed. Further, a release upgrade must allow a cluster to roll back to the older version and its older disk format. (rollback needs to restore the original data but not any updated data). Motivation: Users expect File systems preserve data transparently across releases. ** Stronger version of the above HDFS metadata and data can change across minor or major releases, but such changes are transparent to user application. A release upgrade must automatically convert the metadata and data as needed. During *minor* releases, disk format changes have to backward and forward compatible; i.e. an older version of Hadoop can be started on a newer version of the disk format. Hence a version roll back is simple, just restart the older version of Hadoop. Major releases allow more significant changes to the disk format and have be only backward compatible; however major release upgrade must allow a cluster to roll back to the older version and its older disk format. With this minor release are very easy to roll back for an admin. Note this will restrict the kinds of changes that be made in minor releases. ** Weaker: Limited Automatic Conversion: HDFS metadata and data can change across minor or major releases , but such changes are transparent to user application. A release upgrade must automatically convert the metadata and data as needed, but automatic conversion is supported across a small number of releases. If a user wants to jump across multiple releases he may be forced to go through a few intermediate release to get to the final desired release. Further, a release upgrade must allow a cluster to roll back to the older version and its older disk format. (rollback needs to restore the original data but not any updated data). Automatic conversion is support across a small number of releases. If a user wants to jump across multiple releases he may be forced to go through a few intermediate release to get to the final desired release. We currently support the weaker automatic conversion in HDFS. > Define Hadoop Compatibility > --------------------------- > > Key: HADOOP-9517 > URL: https://issues.apache.org/jira/browse/HADOOP-9517 > Project: Hadoop Common > Issue Type: Bug > Components: documentation > Reporter: Arun C Murthy > Assignee: Karthik Kambatla > Attachments: hadoop-9517.patch, hadoop-9517.patch, hadoop-9517.patch, > hadoop-9517.patch > > > As we get ready to call hadoop-2 stable we need to better define 'Hadoop > Compatibility'. > http://wiki.apache.org/hadoop/Compatibility is a start, let's document > requirements clearly and completely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira