Just to share a little bit of battle-stories.
We discovered some ... interesting ... issues ... This was really a bit of
a puzzle why virtualenvs created in the ARM image of Python 3.11 did not
have SSL support :D.
Just to repeat: *main Python was working*, *ONLY*(!) venvs created in the
image, *ONLY*(!!) in ARM image, *ONLY*(!!!) for Python 3.11 did not have
SSL support. Single group of "External Python Decorators" was failing.
That's 1 group of tests failing in 3 of our 2x4x3 matrix of tests.
Sometimes I have doubts about all the overhead of running all combinations
of supported python versions/platforms in all the combinations and running
a full set of tests for them. I was also holding off on making our ARM
images as "experimental" until we were able to run all tests on Public ARM
runners.
But situations like this make it 100% worthy.
I can't imagine trying to find out what's wrong if a user reports this
error out of the sudden:
WARNING: pip is configured with locations that require TLS/SSL, however the
ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'SSLError("Can't connect to HTTPS
URL because the SSL module is not available.")': /simple/cloudpickle/
Finding out that this is this:
* because it is 3.11
* on ARM
* and venv was created with `venv.create()` from the main python
* and the problem is that the system python shared .so libraries from
3.11.2 interfered with 3.11.5 we installed
- would be next to impossible.
For those interested: https://github.com/apache/airflow/pull/55120
J.
On Tue, Sep 2, 2025 at 8:29 AM Amogh Desai <[email protected]> wrote:
> Good job folks!
>
> I would've loved to follow it more closely than I did!
>
> Thanks & Regards,
> Amogh Desai
>
>
> On Sat, Aug 30, 2025 at 8:59 PM Jarek Potiuk <[email protected]> wrote:
>
> > Thanks to Aritra for leading it !!!
> >
> > On Sat, Aug 30, 2025 at 5:27 PM Aritra Basu <[email protected]>
> > wrote:
> >
> > > Woohoo! 🎉
> > > --
> > > Regards,
> > > Aritra Basu
> > >
> > > On Sat, 30 Aug 2025, 8:32 pm Jarek Potiuk, <[email protected]> wrote:
> > >
> > > > The https://github.com/apache/airflow/pull/53770 ("Migrated prod
> > builds
> > > to
> > > > use python built from source") has been merged. This means that our
> > PROD
> > > > images built from main (soon 3.1) will get the latest released Python
> > > > packages directly from the PSF release server and we build Python
> from
> > > > sources. This will give us more control over what is "in" the image
> and
> > > > allow us to switch to the latestst Python patchlevel faster.
> > > >
> > > > We still have lazy consensus running in
> > > > https://lists.apache.org/thread/fp6j1jcx82bok75y087p0445bbl6rzsb,
> but
> > I
> > > > want to use the calmer weekend time to see and fix any teething
> > > problems. -
> > > > we can always revert it if need be.
> > > >
> > > > We do not plan to back-port it to 3.0.
> > > >
> > > > For the next few hours, rebuilding the CI image might be slow because
> > the
> > > > Dockerfiles used trigger cache invalidation at the very start - but
> it
> > > > should get back to faster builds soon.
> > > >
> > > > J
> > > >
> > >
> >
>
