Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-05-26 Thread via GitHub
catkins commented on PR #10151: URL: https://github.com/apache/iceberg/pull/10151#issuecomment-2132549187 This would be a great addition, I got a bit caught out in development this week when setting up an iceberg sink on my local docker flink setup, and being confused why the s3 config

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-25 Thread via GitHub
stevenzwu commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1580188262 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-25 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1580019899 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-25 Thread via GitHub
stevenzwu commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1579860068 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-23 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1576011008 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-23 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1576012292 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-23 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1576007535 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-22 Thread via GitHub
stevenzwu commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1575403714 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on PR #10151: URL: https://github.com/apache/iceberg/pull/10151#issuecomment-2066270577 > Quick question: now that flink 1.19 is available in the repo, do we still merge this to 1.18 and then later we port it to 1.19 and all the other versions? I usually try to keep

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1572152325 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkInputFile.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1572150806 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1572151051 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1572147961 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-19 Thread via GitHub
pvary commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1572124567 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571094073 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/FlinkFileIOTest.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571092822 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkInputFile.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571089828 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571086551 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571083046 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-18 Thread via GitHub
rodmeneses commented on code in PR #10151: URL: https://github.com/apache/iceberg/pull/10151#discussion_r1571082650 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkFileIO.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[PR] Flink: FlinkFileIO implementation [iceberg]

2024-04-16 Thread via GitHub
pvary opened a new pull request, #10151: URL: https://github.com/apache/iceberg/pull/10151 FileIO implementation which uses Flink embedded filesystem implementations. This allows the user to use the Flink pluggable FileSystems: