Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
bolkedebruin merged PR #34729: URL: https://github.com/apache/airflow/pull/34729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1783193702 I added it to milestone 2.8.0 since the vote can be closed and already has the needed binding votes. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
bolkedebruin commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1783139423 can I have a (final) review @potiuk @Taragolis @uranusjr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-26 Thread via GitHub
bolkedebruin commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1781038669 Afk at the moment but will do @potiuk . 🙏 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-26 Thread via GitHub
potiuk commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1780955847 You will need to rebase again @bolkedebruin (this time pre-commit should automatically fix things for you for the breeze help images). also I've fixed the k8s_requirements.txt problem d

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-24 Thread via GitHub
jens-scheffler-bosch commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1777859553 > I can consider adding it to the higher level module and export it _also_ from there so you could do `from airflow.io import ObjectStoragePath`. Note that we got rid of th

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-23 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1369692131 ## airflow/providers/amazon/aws/fs/s3.py: ## @@ -0,0 +1,123 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-23 Thread via GitHub
bolkedebruin commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1776620355 > * If the plugin interface is extended does it mean "newer" extended providers like AWS will work on older versions e.g. Airflow 2.7.3 (See also the comment in CWiki ([Comment

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-23 Thread via GitHub
Taragolis commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1369145057 ## airflow/providers/amazon/aws/fs/s3.py: ## @@ -0,0 +1,123 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreemen

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-23 Thread via GitHub
bolkedebruin commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1775323854 @jens-scheffler-bosch @hussein-awala @uranusjr @Taragolis I think we're good now - sqlite test seems unrelated. -- This is an automated message from the Apache Git Service. To re

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367798191 ## airflow/io/store/__init__.py: ## Review Comment: Yes I think it will be extended for example with `dataframe` and potentially `catalog`. -- This is

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367792359 ## tests/system/providers/common/io/example_file_transfer_local_to_s3.py: ## @@ -0,0 +1,90 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367792301 ## airflow/providers/common/io/provider.yaml: ## Review Comment: That's okay to me, but don't consider it part of this PR. It's complex as it is. -- Th

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367792225 ## airflow/io/store/path.py: ## @@ -0,0 +1,680 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See t

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367791852 ## airflow/providers/common/io/operators/file_transfer.py: ## @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contri

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367791730 ## airflow/io/__init__.py: ## @@ -0,0 +1,90 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-21 Thread via GitHub
jens-scheffler-bosch commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367761520 ## airflow/io/store/__init__.py: ## Review Comment: What is the reason that we have a sub-package `store` below `airflow.io` if there are no other