[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1358432899 > A better option will be to add links to installation instructions for those packages that we know might be problematic. For example for Plyvel linking https://plyvel.readthedocs.io/en/latest/installation.html would be fine. NOTE - even Plyvel developer limited those to "Ubuntu and Debian". Yep initially i tried to use that doc, and unfortunately they refers to outdated name of package, from Debian 9 I guess :man_facepalming: https://packages.debian.org/search?suite=bullseye=any=names=libleveldb Ok, lets better see if any kind of issue appear, it is possible that no one actually affected or found solution by their own. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1357412806 > I do not think we can reasonably describe it for each provider separately - especially that many of our users do not use the image at all - they install airflow in their own virtualenv and on different distributions and there different dependencies are needed for those different distributions. We could try to do with simple cases. It is not mandatory I'd rather say it nice to have, if we know that package could be install on Linux (glib-based, not musl) but required install from sources then would be nice to have this information which save some time. >>My provider fails to install and I see those other providers have instructions - surely you should have instruction here and I should not need to solve it myself when I am installing it on . - what are the instructions?" >I think it's a dangerous path to walk. I thought you have some kind template about how many contributors we already have and would be nice if someone who find become a contributor and improve this part 藍 There is always would be someone who want to everything and now. I more focus on users which might be never appear into issues/discussions and slack and find this info by themself (Airflow / providers documentation or in some search engine). > Something that I have thought about before. Maybe - on top of having optimized apache/airflow image and "slim" image we should have "fat" I think this would be nice if it not required a lot of effort from our side. BTW, do we have some statistic about downloads particular image from Docker Hub? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356851154 Oh, I just repeat step which probably most of user do first, just try to install it by `pip` into the regular image. Some of them after would read documentation and found this info, which really easy to find https://www.google.com/search?q=airflow+docker+install+python+packages, other would open issue, discussion, question on SO or into the Slack. Another problem for users what packages they need to install for build it, so that is why I suggest to create per-provider information about each extra/optional dependency (I could do that) > That would be - however - backwards incompatible for x86 users - because installing the provider will not pull azure-service-bus as requirement, so I am not too sure (we could likely come with some mixed approach where we have != aarch64 as "core" requirement and then have an extra to install it without that limitation - but I feel that would be rather cumbersome. I think better to keep compatibility and add as extra section even if it looks dumb. I bet that x86 main users now and for ARM we just have experimental support -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356842242 Hmmm.. I've unable to install it into the regular airflow image ```console ❯ docker run -it --rm apache/airflow:2.5.0-python3.9 bash airflow@7e0bc7525889:/opt/airflow$ uname -a Linux 7e0bc7525889 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 aarch64 GNU/Linux airflow@7e0bc7525889:/opt/airflow$ pip install azure-servicebus Defaulting to user installation because normal site-packages is not writeable Collecting azure-servicebus Downloading azure_servicebus-7.8.1-py3-none-any.whl (210 kB) 210.4/210.4 kB 2.3 MB/s eta 0:00:00 Requirement already satisfied: azure-core<2.0.0,>=1.14.0 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (1.26.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (4.4.0) Requirement already satisfied: six>=1.11.0 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (1.16.0) Requirement already satisfied: isodate>=0.6.0 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (0.6.1) Requirement already satisfied: azure-common~=1.1 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (1.1.28) Requirement already satisfied: msrest<2.0.0,>=0.6.17 in /home/airflow/.local/lib/python3.9/site-packages (from azure-servicebus) (0.7.1) Collecting uamqp<2.0.0,>=1.5.1 Downloading uamqp-1.6.3.tar.gz (4.4 MB) 4.4/4.4 MB 3.7 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: requests>=2.18.4 in /home/airflow/.local/lib/python3.9/site-packages (from azure-core<2.0.0,>=1.14.0->azure-servicebus) (2.28.1) Requirement already satisfied: certifi>=2017.4.17 in /home/airflow/.local/lib/python3.9/site-packages (from msrest<2.0.0,>=0.6.17->azure-servicebus) (2022.9.24) Requirement already satisfied: requests-oauthlib>=0.5.0 in /home/airflow/.local/lib/python3.9/site-packages (from msrest<2.0.0,>=0.6.17->azure-servicebus) (1.3.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/airflow/.local/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.14.0->azure-servicebus) (1.26.13) Requirement already satisfied: charset-normalizer<3,>=2 in /home/airflow/.local/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.14.0->azure-servicebus) (2.1.1) Requirement already satisfied: idna<4,>=2.5 in /home/airflow/.local/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.14.0->azure-servicebus) (3.4) Requirement already satisfied: oauthlib>=3.0.0 in /home/airflow/.local/lib/python3.9/site-packages (from requests-oauthlib>=0.5.0->msrest<2.0.0,>=0.6.17->azure-servicebus) (3.2.2) Building wheels for collected packages: uamqp Building wheel for uamqp (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for uamqp (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [101 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-aarch64-cpython-39 creating build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/mgmt_operation.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/message.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/address.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/errors.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/session.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/receiver.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/utils.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/__init__.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/types.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/connection.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/compat.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/sender.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/constants.py -> build/lib.linux-aarch64-cpython-39/uamqp copying uamqp/client.py -> build/lib.linux-aarch64-cpython-39/uamqp creating build/lib.linux-aarch64-cpython-39/uamqp/authentication copying uamqp/authentication/common.py -> build/lib.linux-aarch64-cpython-39/uamqp/authentication copying uamqp/authentication/cbs_auth.py ->
