alamb commented on code in PR #189:
URL: https://github.com/apache/parquet-site/pull/189#discussion_r3468606241
##########
content/en/docs/File Format/versions.md:
##########
@@ -72,190 +72,18 @@ the next parquet-format release.
Forward incompatible features and the format version each became available in:
-| Feature | Released in | Source | Notes |
Review Comment:
The point of the PR is to automatically derive this table from the .yml files
##########
data/implementations/features/compressions.yaml:
##########
@@ -2,24 +2,49 @@ category_id: compressions
features:
- id: compression-uncompressed
display_name: UNCOMPRESSED
-
+ compatibility: forward_incompatible
Review Comment:
I added the various fields needed for the version page to the existing data
files. I could probably make this less verbose by automatically generating the
urls, but I think this is clearer and easier to understand
##########
content/en/docs/File Format/implementationstatus.md:
##########
@@ -28,5 +28,5 @@ The value in each box means:
* [duckdb](https://github.com/duckdb/duckdb) (C++)
* [polars](https://github.com/pola-rs/polars) (Rust)
-<!-- Status source in data/implementations -->
+<!-- Data driven table, see /layouts/shortcodes -->
Review Comment:
I added a readme and new link in layouts/shortcodes
##########
layouts/shortcodes/README.md:
##########
@@ -0,0 +1,15 @@
+# Hugo shortcodes
Review Comment:
I found the whole hugo rendering thing new, so I also left some small
documentation
##########
data/implementations/features/encodings.yaml:
##########
@@ -2,37 +2,67 @@ category_id: encodings
features:
- id: encoding-plain
display_name: PLAIN
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-plain-dictionary
display_name: PLAIN_DICTIONARY
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-rle-dictionary
display_name: RLE_DICTIONARY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-rle
display_name: RLE
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-bit-packed
display_name: BIT_PACKED (deprecated)
-
+ compatibility: forward_incompatible
+ released_in: 1.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#bit-packed-deprecated-bit_packed--4
+ source_url:
https://github.com/apache/parquet-format/tree/parquet-format-1.0.0
- id: encoding-delta-binary-packed
display_name: DELTA_BINARY_PACKED
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-delta-length-byte-array
display_name: DELTA_LENGTH_BYTE_ARRAY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-length-byte-array-delta_length_byte_array--6
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-delta-byte-array
display_name: DELTA_BYTE_ARRAY
-
+ compatibility: forward_incompatible
+ released_in: 2.0.0
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-strings-delta_byte_array--7
+ source_url:
https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
- id: encoding-byte-stream-split
display_name: BYTE_STREAM_SPLIT
- format_version:
- version: "2.8.0"
- date: "2020-01-13"
-
+ compatibility: forward_incompatible
+ released_in: 2.8.0
+ release_date: '2020-01-13'
+ feature_url:
https://github.com/apache/parquet-format/blob/master/Encodings.md#byte-stream-split-byte_stream_split--9
+ source_url:
https://github.com/apache/parquet-format/compare/apache-parquet-format-2.7.0...apache-parquet-format-2.8.0
+ version_note: "[Approved
2019-12-03](https://lists.apache.org/thread/xs5qt2odm299pxgqb22mty2csc1so5yr)"
- id: encoding-byte-stream-split-extended
display_name: BYTE_STREAM_SPLIT (Additional Types)
spec_url:
https://github.com/apache/parquet-format/commit/e517ac4dbe08d518eb5c2e58576d4c711973db94
- format_version:
Review Comment:
some features already had a "format_version" which was redundant with the
information from the versions page, so I consolidated them into the same fields
From
```yaml
format_version:
version: "2.11.0"
date: "2024-03-21"
```
To
```yaml
released_in: 2.11.0
release_date: '2024-03-21'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
