HonahX commented on PR #218:
URL: https://github.com/apache/iceberg-python/pull/218#issuecomment-1858723785
Thanks @jqin61. This looks great! Shall we update the doc as well?
https://github.com/apache/iceberg-python/blob/main/mkdocs/docs/configuration.md?plain=1#L67-L76
so that people
ajantha-bhat commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428680842
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifier to
ajantha-bhat commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428680776
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifier to
HonahX commented on PR #140:
URL: https://github.com/apache/iceberg-python/pull/140#issuecomment-1858679351
Thanks @nicor88 and @jackye1995 for the initial review. I think this PR is
ready to be reviewed.
cc: @Fokko @pdames @svdimchenko
--
This is an automated message from the Apac
flyrain commented on PR #9141:
URL: https://github.com/apache/iceberg/pull/9141#issuecomment-1858649569
Sorry for the delay. Here is what happened.
1. s1 adds data file 1 in partition 1
2. s2 adds data file 2 in partition 2
3. s3 insert overwrite partition 2, which deletes data file
github-actions[bot] commented on issue #7287:
URL: https://github.com/apache/iceberg/issues/7287#issuecomment-1858628410
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] closed issue #7287: Broken Unit tests on Windows OS
URL: https://github.com/apache/iceberg/issues/7287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To u
github-actions[bot] closed issue #7440: Simplify JDK build versions using
--release flag
URL: https://github.com/apache/iceberg/issues/7440
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
github-actions[bot] commented on issue #7332:
URL: https://github.com/apache/iceberg/issues/7332#issuecomment-1858628375
This issue has been automatically marked as stale because it has been open
for 180 days with no activity. It will be closed in next 14 days if no further
activity occurs.
github-actions[bot] closed issue #7406: Question about iceberg partition table
URL: https://github.com/apache/iceberg/issues/7406
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
github-actions[bot] commented on issue #7406:
URL: https://github.com/apache/iceberg/issues/7406#issuecomment-1858628353
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] commented on issue #7440:
URL: https://github.com/apache/iceberg/issues/7440#issuecomment-1858628332
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
stevenzwu commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428500261
##
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java:
##
@@ -80,6 +87,18 @@ public RecordsWithSplitIds> fetch()
th
amogh-jahagirdar merged PR #9247:
URL: https://github.com/apache/iceberg/pull/9247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ic
amogh-jahagirdar commented on PR #9247:
URL: https://github.com/apache/iceberg/pull/9247#issuecomment-1858477802
Thanks for the review @rdblue , merging!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
amogh-jahagirdar commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428471085
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifie
pvary commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428447497
##
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java:
##
@@ -80,6 +87,18 @@ public RecordsWithSplitIds> fetch()
throws
pvary commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428446262
##
flink/v1.18/build.gradle:
##
@@ -63,6 +63,7 @@ project(":iceberg-flink:iceberg-flink-${flinkMajorVersion}") {
// These artifacts are shaded and included in the
pvary commented on PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#issuecomment-1858414638
> @pvary is the whole PR addressing the flaky test? if not, please separate
out the flaky test part (hopefully small) into a separate PR. I would like to
merge the flaky test fix ASAP. And
danielcweeks commented on code in PR #7844:
URL: https://github.com/apache/iceberg/pull/7844#discussion_r1428383394
##
core/src/main/java/org/apache/iceberg/util/ParallelIterable.java:
##
@@ -67,10 +70,17 @@ private ParallelIterator(
try (Closeable i
danielcweeks commented on PR #7844:
URL: https://github.com/apache/iceberg/pull/7844#issuecomment-1858369750
One minor comment but otherwise looks good to me. @nastra thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
stevenzwu commented on PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#issuecomment-1858352676
@pvary is the whole PR addressing the flaky test? if not, please separate
out the flaky test part (hopefully very small) into a separate PR. I would like
to merge the flaky test fix ASA
jqin61 opened a new pull request, #218:
URL: https://github.com/apache/iceberg-python/pull/218
Hi! We noticed we are running into exceptions thrown due to aws timeout when
we are using pyiceberg.io.pyarrow.PyArrowFileIO:
```
OSError: When reading information for key
'table/metadata/b7
rodmeneses commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428246189
##
flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceSplitReader.java:
##
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Soft
rodmeneses commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428244482
##
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java:
##
@@ -80,6 +87,18 @@ public RecordsWithSplitIds> fetch()
t
rodmeneses commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428240323
##
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java:
##
@@ -61,6 +67,7 @@ class IcebergSourceSplitReader implemen
rodmeneses commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1428236699
##
flink/v1.18/build.gradle:
##
@@ -63,6 +63,7 @@ project(":iceberg-flink:iceberg-flink-${flinkMajorVersion}") {
// These artifacts are shaded and included in
kuldeepsinghchauhan commented on issue #3619:
URL: https://github.com/apache/iceberg/issues/3619#issuecomment-1858175706
> As I see it, this integration would have to be done in one of two ways:
>
> 1. Integration into the Runtime components, which would take time as it
would require
kuldeepsinghchauhan commented on issue #3619:
URL: https://github.com/apache/iceberg/issues/3619#issuecomment-1858167251
We are using EMR 614 with Apache Iceberg 1.3.1 and ranger plugin for spark
and emrfs/s3 to govern access. We were hoping that existing ranger plugins for
spark and trino
lkokhreidze commented on issue #9306:
URL: https://github.com/apache/iceberg/issues/9306#issuecomment-1858166115
Thanks @pvary that looks really awesome and helpful.
I'll try to look at zorder implementation and see what we could do to
contribute.
--
This is an automated message from t
pvary commented on PR #8195:
URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1858158123
> However, there's no primary key in Spark or Trino.
I think it should be possible to provide the identity fields there too.
The java API provides the possibility through the `S
ajantha-bhat commented on PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1858140592
Re-triggering build due to flaky test in Flink
https://github.com/apache/iceberg/actions/runs/7222400549/job/19679307743?pr=8502
--
This is an automated message from the Apa
ajantha-bhat closed pull request #8502: Api: Track partition statistics via
TableMetadata
URL: https://github.com/apache/iceberg/pull/8502
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
nastra commented on PR #9302:
URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1858116202
@pvary thanks for the input, I think that's quite helpful. I'll take a
closer look and will provide an update once I know more.
--
This is an automated message from the Apache Git Servic
pvary commented on PR #9302:
URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1858113490
Maybe it was a gradle issue which is fixed by a clean build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
pvary commented on PR #9302:
URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1858112362
I am using java 8
Previuosly, I have run this:
```
$ ./gradlew :iceberg-flink:iceberg-flink-runtime-1.18:build
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use
manuzhang commented on PR #8195:
URL: https://github.com/apache/iceberg/pull/8195#issuecomment-1858100586
However, there's no primary key in Spark or Trino.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
pvary commented on PR #9301:
URL: https://github.com/apache/iceberg/pull/9301#issuecomment-1858092550
Cool! Thanks @deniskuzZ for coming back with the results!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
pvary commented on issue #9306:
URL: https://github.com/apache/iceberg/issues/9306#issuecomment-1858086083
Currently the only supported action is
https://github.com/apache/iceberg/blob/main/flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java
We
pvary commented on code in PR #9161:
URL: https://github.com/apache/iceberg/pull/9161#discussion_r1428116580
##
parquet/src/test/java/org/apache/iceberg/parquet/TestDictionaryRowGroupFilter.java:
##
@@ -223,52 +219,33 @@ public void createInputFile() throws IOException {
di
nastra commented on PR #9302:
URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1858058869
@pvary did you see this error on CI or just when running `./gradlew
checkClassUniqueness --write-locks`? I tried to repro this but I'm not able to
see that error (I tried JDK8 and JDK18).
pvary commented on PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#issuecomment-1858055733
CC: @ajantha-bhat - further fixes on the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
pvary opened a new pull request, #9308:
URL: https://github.com/apache/iceberg/pull/9308
Fix unit test failures found here: #9216
Before the changes it was not possible to run the tests multiple times using
IntelliJ.
Removed the `static` from `InMemoryReporter` so the tests could b
jbonofre opened a new pull request, #298:
URL: https://github.com/apache/iceberg-docs/pull/298
The purpose is to give visibility to the community about our release
schedule.
This is a first proposal, happy to discuss and refine ;)
@rdblue @danielcweeks @bitsondatadev wdyt ?
ajantha-bhat commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428076109
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifier to
deniskuzZ closed pull request #9301: BugFix: ORC reader is not closed when
SortedMerge iterator is used for positional deletes
URL: https://github.com/apache/iceberg/pull/9301
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and u
ajantha-bhat commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428074232
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifier to
deniskuzZ commented on code in PR #9301:
URL: https://github.com/apache/iceberg/pull/9301#discussion_r1428070146
##
orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java:
##
@@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator(
try {
re
deniskuzZ commented on code in PR #9301:
URL: https://github.com/apache/iceberg/pull/9301#discussion_r1428070146
##
orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java:
##
@@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator(
try {
re
pvary commented on PR #9216:
URL: https://github.com/apache/iceberg/pull/9216#issuecomment-1857969574
I was originally not able to run the tests multiple times using IntelliJ,
but found some issues, and fixed them.
Now I have a test which I was able to run 1000 times from IntelliJ without
pvary commented on PR #9302:
URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1857964575
Could this be that the following error is caused by this change:
```
# Danger! Multiple jars contain identically named classes. This may cause
different behaviour depending on classpat
nastra commented on code in PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#discussion_r1428012608
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:
##
@@ -948,6 +950,17 @@ public static
org.apache.spark.sql.catalyst.TableIdentifier toV1Tabl
ajantha-bhat commented on PR #9298:
URL: https://github.com/apache/iceberg/pull/9298#issuecomment-1857854433
@nastra, @Fokko: PR is ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
ajantha-bhat commented on PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1857853254
@aokolnychyi: I have handled all the new comments. Thanks again for the
review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
ajantha-bhat commented on code in PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427955084
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -946,15 +955,25 @@ private Builder(TableMetadata base) {
this.previousFileLocation = base
ajantha-bhat commented on code in PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427954785
##
api/src/main/java/org/apache/iceberg/PartitionStatisticsFile.java:
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
ajantha-bhat commented on code in PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427926385
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1288,6 +1307,24 @@ public Builder suppressHistoricalSnapshots() {
return this;
}
ajantha-bhat commented on code in PR #8502:
URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427920868
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -946,15 +955,25 @@ private Builder(TableMetadata base) {
this.previousFileLocation = base
ajantha-bhat commented on code in PR #9305:
URL: https://github.com/apache/iceberg/pull/9305#discussion_r1427885903
##
core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java:
##
@@ -38,19 +49,28 @@
public class TestCatalogUtilDropTable extends HadoopTableTes
findepi commented on PR #7844:
URL: https://github.com/apache/iceberg/pull/7844#issuecomment-1857726420
cc @danielcweeks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
JFinis opened a new issue, #9307:
URL: https://github.com/apache/iceberg/issues/9307
### Apache Iceberg version
1.4.2 (latest release)
### Query engine
None
### Please describe the bug 🐞
The spec is incomplete w.r.t. the mapping of Iceberg types to Avro type
nastra commented on issue #9026:
URL: https://github.com/apache/iceberg/issues/9026#issuecomment-1857686284
@MehulBatra yes, this is expected behavior, but sorry that I forgot to close
this issue out.
--
This is an automated message from the Apache Git Service.
To respond to the message,
nastra closed issue #9026: Spark SQL DESCRIBE not showing proper schema on a
branch
URL: https://github.com/apache/iceberg/issues/9026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
Fokko commented on issue #217:
URL: https://github.com/apache/iceberg-python/issues/217#issuecomment-1857609612
The error is being raised when PyArrow tries to filter the data, but this
file should be skipped based on the Iceberg statistics.
--
This is an automated message from the Apache
jasonf20 commented on code in PR #9230:
URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427260537
##
core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java:
##
@@ -895,7 +895,7 @@ private void cleanUncommittedAppends(Set
committed) {
}
Fokko opened a new issue, #217:
URL: https://github.com/apache/iceberg-python/issues/217
### Feature Request / Improvement
Reported on Slack:
https://apache-iceberg.slack.com/archives/C029EE6HQ5D/p1702607510827899 Thanks
for reporting this 👍
Able to reproduce locally:
