alamb commented on code in PR #189:
URL: https://github.com/apache/parquet-site/pull/189#discussion_r3483105530
##########
data/implementations/features/encodings.yaml:
##########
@@ -2,37 +2,67 @@ category_id: encodings
features:
- id: encoding-plain
display_name: PLAIN
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-plain-dictionary
display_name: PLAIN_DICTIONARY
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-rle-dictionary
display_name: RLE_DICTIONARY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-rle
display_name: RLE
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-bit-packed
display_name: BIT_PACKED (deprecated)
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#bit-packed-deprecated-bit_packed--4
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-delta-binary-packed
display_name: DELTA_BINARY_PACKED
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-delta-length-byte-array
display_name: DELTA_LENGTH_BYTE_ARRAY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-length-byte-array-delta_length_byte_array--6
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-delta-byte-array
display_name: DELTA_BYTE_ARRAY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-strings-delta_byte_array--7
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-byte-stream-split
display_name: BYTE_STREAM_SPLIT
- format_version:
- version: "2.8.0"
- date: "2020-01-13"
-
+ compatibility: forward_incompatible
+ released_in: 2.8.0
+ release_date: '2020-01-13'
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#byte-stream-split-byte_stream_split--9
+ source_url:
https://github.com/apache/parquet-format/compare/apache-parquet-format-2.7.0...apache-parquet-format-2.8.0
+ version_note: "[Approved
2019-12-03](https://lists.apache.org/thread/xs5qt2odm299pxgqb22mty2csc1so5yr)"
- id: encoding-byte-stream-split-extended
display_name: BYTE_STREAM_SPLIT (Additional Types)
spec_url:
https://github.com/apache/parquet-format/commit/e517ac4dbe08d518eb5c2e58576d4c711973db94
- format_version:
Review Comment:
Good idea -- I quoted them all in commit 9f8d58a2
##########
layouts/shortcodes/format-versions.html:
##########
@@ -0,0 +1,103 @@
+{{- /*
+ Render Parquet format version feature tables from
data/implementations/features/*.yaml.
+ See layouts/shortcodes/README.md for details.
+ Usage: {{< format-versions table="forward_incompatible" >}}
+*/ -}}
+{{- $tableName := .Get "table" -}}
+{{- $validTables := slice "forward_incompatible" "forward_compatible" -}}
+{{- if not (in $validTables $tableName) -}}
+ {{- errorf "unknown format versions table %q" $tableName -}}
+{{- end -}}
+
+{{- $features := slice -}}
+{{- range site.Data.implementations.categories -}}
+ {{- $featuresData := index site.Data.implementations.features .id -}}
+ {{- if $featuresData -}}
+ {{- range $featuresData.features -}}
+ {{- if eq .compatibility $tableName -}}
+ {{- $releasedIn := printf "%v" .released_in -}}
+ {{- $versionParts := split $releasedIn "." -}}
+ {{- $major := int (index $versionParts 0) -}}
+ {{- $minor := 0 -}}
+ {{- $patch := 0 -}}
+ {{- if ge (len $versionParts) 2 -}}
+ {{- $minor = int (index $versionParts 1) -}}
+ {{- end -}}
+ {{- if ge (len $versionParts) 3 -}}
+ {{- $patch = int (index $versionParts 2) -}}
+ {{- end -}}
+ {{- $sortKey := printf "%04d.%04d.%04d.%s" $major $minor $patch .id -}}
+ {{- $features = $features | append (dict "sort_key" $sortKey "feature"
.) -}}
+ {{- end -}}
+ {{- end -}}
+ {{- end -}}
+{{- end -}}
+{{- $features = sort $features "sort_key" "desc" -}}
+
+<table class="table table-striped">
+ <thead>
+ <tr>
+ <th>Feature</th>
+ <th>Released in</th>
+ <th>Source</th>
+ <th>Approved</th>
+ </tr>
+ </thead>
+ <tbody>
+ {{- range $features }}
+ {{- $feature := .feature -}}
+ {{- $displayName := $feature.display_name -}}
+ {{- if $feature.version_display_name -}}
+ {{- $displayName = $feature.version_display_name -}}
+ {{- end -}}
+ {{- $featureUrl := $feature.feature_url -}}
+ {{- if and (not $featureUrl) $feature.spec_url -}}
+ {{- $featureUrl = $feature.spec_url -}}
+ {{- end -}}
+ {{- $releasedIn := printf "%v" $feature.released_in -}}
+ {{- $legacyReleaseTags := slice "1.0.0" "2.0.0" "2.1.0" -}}
Review Comment:
Apparently in hugo makes an array -- this is part of some overly complicated
logic to translate release names like `1.0.0` into tag names (whose format
changed over time)
I simplified the whole thing by changing this to a lookup table -- which
simplifies this logic and I think makes it clearer what is going on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
