Re: [PR] Build: Bump mkdocs-material from 9.5.2 to 9.5.3 [iceberg-python]

2023-12-25 Thread via GitHub
Fokko merged PR #239: URL: https://github.com/apache/iceberg-python/pull/239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat: Add website layout [iceberg-rust]

2023-12-25 Thread via GitHub
Fokko merged PR #130: URL: https://github.com/apache/iceberg-rust/pull/130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Documentation [iceberg-rust]

2023-12-25 Thread via GitHub
Fokko closed issue #114: Documentation URL: https://github.com/apache/iceberg-rust/issues/114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-uns

Re: [PR] init writer framework [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on PR #135: URL: https://github.com/apache/iceberg-rust/pull/135#issuecomment-1869321881 I would suggest to have a class hierarchy to demonstrate the whole picture of this design. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Spark 3.5: Migrate remaining tests to JUnit5 in source directory [iceberg]

2023-12-25 Thread via GitHub
chinmay-bhat commented on PR #9380: URL: https://github.com/apache/iceberg/pull/9380#issuecomment-1869285088 @nastra please review this PR too :) Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Spark 3.5: Migrate remaining tests to JUnit5 in source directory [iceberg]

2023-12-25 Thread via GitHub
chinmay-bhat commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1436255413 ## data/src/test/java/org/apache/iceberg/data/ParameterizedDeleteReadTest.java: ## @@ -0,0 +1,591 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[PR] init writer framework [iceberg-rust]

2023-12-25 Thread via GitHub
ZENOTME opened a new pull request, #135: URL: https://github.com/apache/iceberg-rust/pull/135 related issue: #34 I drafted a writer framework which has been implemented in icelake https://github.com/icelake-io/icelake/issues/243 and proved that it's extensible and flexible. The followin

Re: [I] How to improve write speed for data in the same partition? [iceberg]

2023-12-25 Thread via GitHub
xuchang-66 commented on issue #9330: URL: https://github.com/apache/iceberg/issues/9330#issuecomment-1869222882 Thanks all, I will have a try. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Build: Bump mkdocs-material from 9.5.2 to 9.5.3 [iceberg-python]

2023-12-25 Thread via GitHub
dependabot[bot] opened a new pull request, #239: URL: https://github.com/apache/iceberg-python/pull/239 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.2 to 9.5.3. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdo

Re: [PR] Core: Remove unused sourceTransform parameter in helper method [iceberg]

2023-12-25 Thread via GitHub
amogh-jahagirdar merged PR #9379: URL: https://github.com/apache/iceberg/pull/9379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Remove unused sourceTransform parameter in helper method [iceberg]

2023-12-25 Thread via GitHub
amogh-jahagirdar commented on PR #9379: URL: https://github.com/apache/iceberg/pull/9379#issuecomment-1869045635 Thanks for review @Fokko ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] feat: Expression system. [iceberg-rust]

2023-12-25 Thread via GitHub
amogh-jahagirdar commented on code in PR #132: URL: https://github.com/apache/iceberg-rust/pull/132#discussion_r1436109732 ## crates/iceberg/src/expr/mod.rs: ## @@ -0,0 +1,49 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreem

Re: [PR] feat: Expression system. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on PR #132: URL: https://github.com/apache/iceberg-rust/pull/132#issuecomment-1868978725 I did a refactoring to make it more similar models used in java/python. cc @ZENOTME @Fokko @Xuanwo @amogh-jahagirdar PTAL -- This is an automated message from the Apache Git S

Re: [I] How to improve write speed for data in the same partition? [iceberg]

2023-12-25 Thread via GitHub
TechTinkerer42 commented on issue #9330: URL: https://github.com/apache/iceberg/issues/9330#issuecomment-1868957150 Each partition should be written by only one task to prevent multiple tasks from writing to the same partition, which can lead to the creation of small files. Here are

Re: [PR] Core: Optimize manifest evaluation for super wide tables [iceberg]

2023-12-25 Thread via GitHub
Fokko merged PR #9147: URL: https://github.com/apache/iceberg/pull/9147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [I] Core : Optimise manifest evaluation for tables with large number of columns (30x faster) [iceberg]

2023-12-25 Thread via GitHub
irshadcc closed issue #9118: Core : Optimise manifest evaluation for tables with large number of columns (30x faster) URL: https://github.com/apache/iceberg/issues/9118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] refactor: Remove `datatype()` method in `Literal`. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on issue #95: URL: https://github.com/apache/iceberg-rust/issues/95#issuecomment-1868881141 > Actually I think this is already done Oh, sorry, my bad. Let's close this now. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] refactor: Remove `datatype()` method in `Literal`. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 closed issue #95: refactor: Remove `datatype()` method in `Literal`. URL: https://github.com/apache/iceberg-rust/issues/95 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436030088 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -124,6 +150,69 @@ impl Snapshot { Utc.timestamp_millis_opt(self.timestamp_ms).unwrap() } +///

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436029994 ## crates/iceberg/src/table.rs: ## @@ -42,8 +50,624 @@ impl Table { &self.metadata } +/// Returns current metadata ref. +pub fn metadat

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436029667 ## crates/iceberg/src/table.rs: ## @@ -42,8 +50,624 @@ impl Table { &self.metadata } +/// Returns current metadata ref. +pub fn metadat

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436019447 ## crates/iceberg/src/spec/manifest.rs: ## @@ -781,10 +805,13 @@ impl ManifestMetadata { } } +/// Reference to [`ManifestEntry`]. +pub type ManifestEnt

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436018874 ## crates/iceberg/Cargo.toml: ## @@ -62,4 +62,5 @@ uuid = { workspace = true } [dev-dependencies] pretty_assertions = { workspace = true } tempfile = { work

Re: [PR] Spark 3.3: Expose action classes in SparkActions [iceberg]

2023-12-25 Thread via GitHub
bknbkn commented on code in PR #5257: URL: https://github.com/apache/iceberg/pull/5257#discussion_r1436018686 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/ExpireSnapshotsProcedure.java: ## Review Comment: It seem should use ExpireSnapshotsSparkActio

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2023-12-25 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1436018238 ## crates/iceberg/src/table.rs: ## @@ -42,8 +50,624 @@ impl Table { &self.metadata } +/// Returns current metadata ref. +pub fn metadat