[GitHub] [iceberg] nastra commented on a diff in pull request #4392: Nessie: Add Catalog tests for NessieCatalog

2022-04-03 Thread GitBox
nastra commented on code in PR #4392: URL: https://github.com/apache/iceberg/pull/4392#discussion_r841406516 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieCatalog.java: ## @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841362218 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1140,6 +1197,10 @@ public TableMetadata build() { ); } +public SnapshotRef

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841387160 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) {

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841387160 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) {

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841386980 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) {

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841382268 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) {

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841382268 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) {

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841344412 ## core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java: ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841362218 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1140,6 +1197,10 @@ public TableMetadata build() { ); } +public SnapshotRef

[GitHub] [iceberg] samredai commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
samredai commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841351631 ## docs/format/view-spec.md: ## @@ -0,0 +1,20 @@ +--- +title: "View Spec" Review Comment: Yes this is required. It adds the menu item to the left-nav on the docs si

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841350456 ## api/src/main/java/org/apache/iceberg/ManageSnapshots.java: ## @@ -80,4 +80,98 @@ * wapId */ ManageSnapshots cherrypick(long snapshotId); + + /**

[GitHub] [iceberg] samredai commented on a diff in pull request #4480: Python: Refactor to use common decimal and datetime util

2022-04-03 Thread GitBox
samredai commented on code in PR #4480: URL: https://github.com/apache/iceberg/pull/4480#discussion_r841348108 ## python/src/iceberg/utils/decimal.py: ## @@ -0,0 +1,70 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841344412 ## core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java: ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841344412 ## core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java: ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841344002 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1022,6 +1065,15 @@ public Builder removeBranch(String branch) { return this; }

[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #4464: Core: Fix Conflicting transactions are ignored during commit

2022-04-03 Thread GitBox
ajantha-bhat commented on code in PR #4464: URL: https://github.com/apache/iceberg/pull/4464#discussion_r841337405 ## core/src/main/java/org/apache/iceberg/BaseTransaction.java: ## @@ -437,6 +430,25 @@ private void commitSimpleTransaction() { } } + private void applyU

[GitHub] [iceberg] jun-he commented on a diff in pull request #3450: [Python] support iceberg transforms in python

2022-04-03 Thread GitBox
jun-he commented on code in PR #3450: URL: https://github.com/apache/iceberg/pull/3450#discussion_r841305933 ## python/src/iceberg/utils/transform_util.py: ## @@ -0,0 +1,110 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreement

[GitHub] [iceberg] felixYyu commented on a diff in pull request #4444: Static Analysis: Use Instant instead of Date for string conversions

2022-04-03 Thread GitBox
felixYyu commented on code in PR #: URL: https://github.com/apache/iceberg/pull/#discussion_r841303758 ## core/src/main/java/org/apache/iceberg/RemoveSnapshots.java: ## @@ -118,7 +118,7 @@ public ExpireSnapshots expireSnapshotId(long expireSnapshotId) { @Override

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841296832 ## docs/format/view-spec.md: ## @@ -0,0 +1,20 @@ +--- +title: "View Spec" Review Comment: @samredai, is this extra file needed? -- This is an automated message fr

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841296798 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841296537 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] felixYyu commented on a diff in pull request #4437: Static Analysis: Supressions we know are false positive

2022-04-03 Thread GitBox
felixYyu commented on code in PR #4437: URL: https://github.com/apache/iceberg/pull/4437#discussion_r841296393 ## core/src/main/java/org/apache/iceberg/util/Tasks.java: ## @@ -300,49 +301,46 @@ private void tryRunOnFailure(I item, Exception failure) { for (final I item

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841296224 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841296057 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841295759 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841295696 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841295582 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841295454 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #3188: Add common view format spec

2022-04-03 Thread GitBox
rdblue commented on code in PR #3188: URL: https://github.com/apache/iceberg/pull/3188#discussion_r841295122 ## format/view-spec.md: ## @@ -0,0 +1,264 @@ + + +# Iceberg View Spec + +## Background and Motivation + +Most compute engines (e.g. Trino and Apache Spark) support views.

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294693 ## core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java: ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294426 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1343,7 +1405,7 @@ private void setBranchSnapshotInternal(Snapshot snapshot, String branch) {

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294374 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1022,6 +1065,15 @@ public Builder removeBranch(String branch) { return this; } +publ

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294251 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1048,9 +1100,14 @@ public Builder removeSnapshots(Collection idsToRemove) { danglingRefs.a

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294224 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1140,6 +1197,10 @@ public TableMetadata build() { ); } +public SnapshotRef ref(Strin

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841294191 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841293698 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) { return t

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841293360 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) { return t

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841293240 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841293065 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1007,6 +1007,49 @@ public Builder setBranchSnapshot(long snapshotId, String branch) { return t

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292971 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292752 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292656 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292656 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292535 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292440 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841292053 ## core/src/main/java/org/apache/iceberg/MetadataUpdate.java: ## @@ -223,31 +225,106 @@ public String name() { @Override public void applyTo(TableMetadata.Bui

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841291985 ## api/src/main/java/org/apache/iceberg/ManageSnapshots.java: ## @@ -80,4 +80,98 @@ * wapId */ ManageSnapshots cherrypick(long snapshotId); + + /** + * Crea

[GitHub] [iceberg] rdblue commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
rdblue commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841291825 ## core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java: ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841290804 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841290714 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841290605 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841290553 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841290322 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841289876 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841289846 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841289524 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841289137 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841288364 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841288257 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841288148 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841287786 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics +

[GitHub] [iceberg-docs] rdblue commented on a diff in pull request #69: Add stats format specification

2022-04-03 Thread GitBox
rdblue commented on code in PR #69: URL: https://github.com/apache/iceberg-docs/pull/69#discussion_r841287675 ## landing-page/content/common/statistics-format-spec.md: ## @@ -0,0 +1,111 @@ +--- +url: statistics-format +toc: false +--- + + +# Plain Format for Iceberg Statistics

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841287307 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841287174 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841286708 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841286552 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841286428 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285831 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285497 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. Review

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285545 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285218 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285199 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes Review Comment: Most pages start with `##` and break up further sections with `###` o

[GitHub] [iceberg] rdblue commented on a diff in pull request #3432: Doc: add a page to explain row-level deletes

2022-04-03 Thread GitBox
rdblue commented on code in PR #3432: URL: https://github.com/apache/iceberg/pull/3432#discussion_r841285134 ## site/docs/row-level-deletes.md: ## @@ -0,0 +1,190 @@ + + +# Row-Level Deletes + +Iceberg supports metadata-based deletion through the `DeleteFiles` interface. +It allo

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #4071: API: Add snapshot reference create, replace, rename, and rename operations to ManageSnapshots API

2022-04-03 Thread GitBox
amogh-jahagirdar commented on code in PR #4071: URL: https://github.com/apache/iceberg/pull/4071#discussion_r841284999 ## api/src/main/java/org/apache/iceberg/ManageSnapshots.java: ## @@ -80,4 +80,98 @@ * wapId */ ManageSnapshots cherrypick(long snapshotId); + + /**

[GitHub] [iceberg] rdblue commented on a diff in pull request #3450: [Python] support iceberg transforms in python

2022-04-03 Thread GitBox
rdblue commented on code in PR #3450: URL: https://github.com/apache/iceberg/pull/3450#discussion_r841281993 ## python/src/iceberg/utils/transform_util.py: ## @@ -0,0 +1,110 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreement

[GitHub] [iceberg] rdblue commented on a diff in pull request #3450: [Python] support iceberg transforms in python

2022-04-03 Thread GitBox
rdblue commented on code in PR #3450: URL: https://github.com/apache/iceberg/pull/3450#discussion_r841281813 ## python/src/iceberg/transforms.py: ## @@ -0,0 +1,567 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See th

[GitHub] [iceberg] rdblue commented on a diff in pull request #3603: Core: Update ResolvingFileIO

2022-04-03 Thread GitBox
rdblue commented on code in PR #3603: URL: https://github.com/apache/iceberg/pull/3603#discussion_r841281467 ## core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java: ## @@ -120,15 +118,20 @@ private FileIO io(String location) { try { io = CatalogUtil.loa

[GitHub] [iceberg] rdblue commented on a diff in pull request #4427: [Spark][Core]: Release memory when rewriting RewriteFileGroup finished

2022-04-03 Thread GitBox
rdblue commented on code in PR #4427: URL: https://github.com/apache/iceberg/pull/4427#discussion_r841281326 ## core/src/main/java/org/apache/iceberg/actions/RewriteDataFilesCommitManager.java: ## @@ -198,9 +199,9 @@ public void offer(RewriteFileGroup group) { } /**

[GitHub] [iceberg] rdblue commented on a diff in pull request #4427: [Spark][Core]: Release memory when rewriting RewriteFileGroup finished

2022-04-03 Thread GitBox
rdblue commented on code in PR #4427: URL: https://github.com/apache/iceberg/pull/4427#discussion_r841281326 ## core/src/main/java/org/apache/iceberg/actions/RewriteDataFilesCommitManager.java: ## @@ -198,9 +199,9 @@ public void offer(RewriteFileGroup group) { } /**

[GitHub] [iceberg] rdblue commented on a diff in pull request #4427: [Spark][Core]: Release memory when rewriting RewriteFileGroup finished

2022-04-03 Thread GitBox
rdblue commented on code in PR #4427: URL: https://github.com/apache/iceberg/pull/4427#discussion_r841281166 ## core/src/main/java/org/apache/iceberg/util/Tasks.java: ## @@ -212,7 +212,9 @@ public boolean run(Task task) { I item = iterator.next(); try {

[GitHub] [iceberg] rdblue commented on a diff in pull request #4449: Fix assert message error

2022-04-03 Thread GitBox
rdblue commented on code in PR #4449: URL: https://github.com/apache/iceberg/pull/4449#discussion_r841280952 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -1084,14 +1084,14 @@ protected void shouldHaveACleanCache(Table

[GitHub] [iceberg] rdblue commented on a diff in pull request #4437: Static Analysis: Supressions we know are false positive

2022-04-03 Thread GitBox
rdblue commented on code in PR #4437: URL: https://github.com/apache/iceberg/pull/4437#discussion_r841280773 ## core/src/main/java/org/apache/iceberg/util/Tasks.java: ## @@ -300,49 +301,46 @@ private void tryRunOnFailure(I item, Exception failure) { for (final I item :

[GitHub] [iceberg] rdblue commented on a diff in pull request #4444: Static Analysis: Use Instant instead of Date for string conversions

2022-04-03 Thread GitBox
rdblue commented on code in PR #: URL: https://github.com/apache/iceberg/pull/#discussion_r841280570 ## core/src/main/java/org/apache/iceberg/RemoveSnapshots.java: ## @@ -118,7 +118,7 @@ public ExpireSnapshots expireSnapshotId(long expireSnapshotId) { @Override

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841279679 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841279622 ## azure/src/main/java/org/apache/iceberg/azure/AuthType.java: ## @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841279466 ## build.gradle: ## @@ -606,6 +613,101 @@ project(':iceberg-dell') { } } +project(':iceberg-azure') { + apply plugin: "com.bmuschko.docker-remote-api" + apply plu

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841279109 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobInputStream.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841279047 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobInputStream.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841278869 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobInputFile.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841278496 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobClientFactory.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841278404 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobClientFactory.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841278204 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureBlobClientFactory.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841278063 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureURI.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841277762 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureURI.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841277486 ## azure/src/main/java/org/apache/iceberg/azure/blob/AzureURI.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841277284 ## build.gradle: ## @@ -38,6 +42,9 @@ buildscript { plugins { id 'nebula.dependency-recommender' version '9.0.2' + // Since 7.x gradle-docker-plugin is compiled us

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841277091 ## build.gradle: ## @@ -606,6 +613,101 @@ project(':iceberg-dell') { } } +project(':iceberg-azure') { + apply plugin: "com.bmuschko.docker-remote-api" + apply plu

[GitHub] [iceberg] rdblue commented on a diff in pull request #4465: Add FileIO implementation for Azure Blob Storage

2022-04-03 Thread GitBox
rdblue commented on code in PR #4465: URL: https://github.com/apache/iceberg/pull/4465#discussion_r841276603 ## .baseline/checkstyle/checkstyle.xml: ## @@ -122,7 +122,8 @@ org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName.*, org.apache.s

[GitHub] [iceberg] rdblue commented on a diff in pull request #4464: Core: Fix Conflicting transactions are ignored during commit

2022-04-03 Thread GitBox
rdblue commented on code in PR #4464: URL: https://github.com/apache/iceberg/pull/4464#discussion_r841276545 ## core/src/main/java/org/apache/iceberg/BaseTransaction.java: ## @@ -437,6 +430,25 @@ private void commitSimpleTransaction() { } } + private void applyUpdates

[GitHub] [iceberg] rdblue commented on a diff in pull request #4464: Core: Fix Conflicting transactions are ignored during commit

2022-04-03 Thread GitBox
rdblue commented on code in PR #4464: URL: https://github.com/apache/iceberg/pull/4464#discussion_r841275479 ## core/src/test/java/org/apache/iceberg/TestTransaction.java: ## @@ -420,6 +421,35 @@ public void testMultipleUpdateTransactionRetryMergeCleanup() { Assert.assertF

  1   2   >