emkornfield commented on code in PR #598:
URL: https://github.com/apache/parquet-format/pull/598#discussion_r3568500976
##########
CONTRIBUTING.md:
##########
@@ -138,65 +147,42 @@ For the purposes of this discussion we classify features
into the following buck
New features are intended to be widely beneficial to users of Parquet, and
therefore it is hoped third-party implementations will adopt them quickly after
-they are introduced. It is assumed that writing new parts of the format, and
-especially forward incompatible features, will be configured with a feature
flag
-defaulted to "off", and at some future point the feature is turned on by
default
-(reading of the new feature will typically be enabled without configuration or
-defaulted to on). Some amount of lead time is desirable to ensure a critical
-mass of Parquet implementations support a feature to avoid compatibility issues
+they are introduced. It is expected that implementations will provide a
configuration
+mechanism for users to enable features when writing parquet file. It is
recommended that implementations provide
+at least a way to enable all relevant features given a specification version
+(e.g. major and minor version). In addition, implementations might choose to
+enable features at a finer-grained level with feature flags.
+
+Some amount of lead time is desirable to ensure a critical
+mass of Parquet implementations support a given specification version
across the ecosystem. Therefore, the Parquet PMC gives the following
recommendations for managing features:
1. Backward compatibility is the concern of implementations but given the
ubiquity of Parquet and the length of time it has been used, libraries
should
support reading older versions of the format to the greatest extent
possible.
-2. Forward compatible features/changes may be enabled and used by default in
- implementations once the parquet-format containing those changes has been
- formally released. For features that may pose a significant performance
- regression to older format readers, libraries should consider delaying
default
- enablement until 1 year after the release of the parquet-java implementation
- that contains the feature implementation.
-
-3. Forward incompatible features/changes should not be turned on by default
- until 2 years after the parquet-java implementation containing the feature
is
- released. It is recommended that changing the default value for a forward
- incompatible feature flag should be clearly advertised to consumers (e.g.
via
- a major version release if using Semantic Versioning, or highlighted in
- release notes).
-
-For forward compatible changes which have a high chance of performance
-regression for older readers and forward incompatible changes, implementations
-should clearly document the compatibility issues. Additionally, while it is up
-to maintainers of individual open-source implementations to make the best
decision to serve
-their ecosystem, they are encouraged to start enabling features by default
along
-the same timelines as `parquet-java`. Parquet-java will wait to enable features
-by default until the most conservative timelines outlined above have been
-exceeded. This timeline is an attempt to balance ensuring
-new features make their way into the ecosystem and avoiding
-breaking compatibility for readers that are slower to adopt new standards. We
-encourage earlier adoption of new features when an organization using Parquet
+2. New release versions should NOT be enabled for writers by default (i.e.
end-users must opt-in to new versions) before 6 months has
Review Comment:
IMO 6 months is the minimal reasonable amount of time before a feature can
be enabled by default. It guarantees at least 1 quarter has passed and
therefore at least one release of readers for an implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
