Hi all, I am happy to announce that Texera now supports Python Virtual Environments (PVEs) for Python UDFs.
With this feature, users can create and manage isolated Python environments on their computing units and select a specific environment for individual Python UDF operators. This makes it possible to install and use custom Python packages without affecting the system-wide Python environment or other workflows. Key capabilities include: - Creating and managing virtual environments through the Texera interface. - Installing custom Python dependencies within each environment. - Selecting a virtual environment for Python UDF operators. - Falling back to the default system Python environment when desired. This feature improves reproducibility and flexibility for Python-based workflows, especially those that depend on specialized libraries or conflicting package versions. The feature was developed over several PRs. - https://github.com/apache/texera/pull/4484 - https://github.com/apache/texera/pull/4902 - https://github.com/apache/texera/pull/5035 - https://github.com/apache/texera/pull/5069 - https://github.com/apache/texera/pull/5138 Best regards, Sarah Asad
