(index of previous posts: https://github.com/sagemath/sage/issues/29705)
Here I will summarize the *technological constraints of modern Python packaging* that a design of modularized distributions must respect. *1. *Two granularities matter for the modularization: The smallest unit is a Python/Cython module. Several modules are packaged together in a distribution package <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#modules-packages-distribution-packages> ("distribution", "pip-installable package"). There are no technical constraints on what modules can go into which distribution. We can partition a package like "sage.misc" and ship the modules from it as parts of 5 different distributions. *2. *Any installation of a distribution in source format ("sdist") first builds a wheel package (which contains the compiled versions of any Cython modules); distributions can run arbitrary code during the build. Then the wheel is installed, which means that a zip file is unpacked into the site-packages directory. There is no such thing as a script that would run at the time of installing a wheel. *3.* Distribution packages declare build dependencies and runtime dependencies <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#dependencies-and-distribution-packages> (on other Python distribution packages). These two can be entirely unrelated to each other. In addition to the runtime dependencies, it is possible to declare extra dependencies – either those necessary for running tests <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#doctest-only-dependencies> or for providing advertised <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#other-runtime-dependencies> extra <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#other-runtime-dependencies> features <https://sagemath-tobias.netlify.app/developer/packaging_sage_library.html#other-runtime-dependencies>. But there is no such thing as an "optional build dependency" or "conditional compilation". The set of modules that form a distribution package is static. There is no place in the name (or metadata) of a wheel package that could indicate different "configurations". *4. *As a result of 3, it is possible to create distributions that contain some modules that cannot be imported because some of their dependencies are missing. That's OK; they can become importable simply by the presence of other distributions, in particular those declared as extra dependencies. All of this is discovered at the time of importing a module; there is no ahead-of-time linking step of any sort. And here are some *desirable criteria* for designing the set of modularized distributions. *5.* Not too fine-grained. As one of the extremes, each of our ~3000 modules could of course be shipped in a separate distribution – guaranteed to allow users to install exactly the modules needed for a specific task, but would obviously leave the user alone with the task of putting together something works. *6. *Each distribution should not depend on too many non-Python dependencies. This is so that when users of a modularized distribution make their first step to contributing to it (perhaps fixing a bug), we do not want to have to tell them "if you want to make this contribution, you either first have to become a real SageMath user and install all of this... or do all your development in our magic cloud environment." *7. *Each distribution should include much fewer Cython modules than the 570 that make up the Sage library. Compilation times matter to users and developers; and remember, there's no such thing as conditional compilation. On the other hand, the number of Python modules shipped by a distribution does not matter all that much. *8. *Discoverability: The modularized distributions should make it easy for users to understand which of them will be needed and should make it convenient to install extra features. At the Sage or IPython prompt, make it convenient for users to discover functionality that has not been installed yet -- both ahead of time and after an exception due to a missing feature is raised. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/97ba1fd5-31a5-4da2-bb97-2bf388a07201n%40googlegroups.com.
