This is an automated email from the ASF dual-hosted git repository.
kunwp1 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new df23815633 feat: Add Python Virtual Environment Support: Uninstalling
User Defined Packages (#5035)
df23815633 is described below
commit df238156335a37c2479ead2a56b2c0a843224ff9
Author: Sarah Asad <[email protected]>
AuthorDate: Tue May 12 21:15:03 2026 -0700
feat: Add Python Virtual Environment Support: Uninstalling User Defined
Packages (#5035)
<!--
Thanks for sending a pull request (PR)! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
[Contributing to
Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md)
2. Ensure you have added or run the appropriate tests for your PR
3. If the PR is work in progress, mark it a draft on GitHub.
4. Please write your PR title to summarize what this PR proposes, we
are following Conventional Commits style for PR titles as well.
5. Be sure to keep the PR description updated to reflect all changes.
-->
### What changes were proposed in this PR?
<!--
Please clarify what changes you are proposing. The purpose of this
section
is to outline the changes. Here are some tips for you:
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
3. If it is a refactoring, clarify what has been changed.
3. It would be helpful to include a before-and-after comparison using
screenshots or GIFs.
4. Please consider writing useful notes for better and faster reviews.
-->
This PR is an extension of PR https://github.com/apache/texera/pull/4484
and #4902. Previously, we introduced support for creating Python Virtual
Environments (PVEs) with system-level dependencies preinstalled, along
with support for installing user-defined packages. This PR extends that
functionality by allowing users to uninstall user-installed packages
from their PVEs.
### Any related issues, documentation, discussions?
<!--
Please use this section to link other resources if not mentioned
already.
1. If this PR fixes an issue, please include `Fixes #1234`, `Resolves
#1234`
or `Closes #1234`. If it is only related, simply mention the issue
number.
2. If there is design documentation, please add the link.
3. If there is a discussion in the mailing list, please add the link.
-->
This change is part of ongoing efforts to support environment isolation
and reproducibility within Texera. Related issue includes
https://github.com/apache/texera/issues/4296. This PR closes sub-issue
#4466.
### How was this PR tested?
<!--
If tests were added, say they were added here. Or simply mention that if
the PR
is tested with existing test cases. Make sure to include/update test
cases that
check the changes thoroughly including negative and positive cases if
possible.
If it was tested in a way different from regular unit tests, please
clarify how
you tested step by step, ideally copy and paste-able, so that other
reviewers can
test and check, and descendants can verify in the future. If tests were
not added,
please describe why they were not added and/or why it was difficult to
add.
-->
Tested Manually and PveResourceSpec test file updated.
To test:
On CU click "+" Python Environments.
Input environment name.
Input package name and version.
Click "OK" and wait for pip logs.
To delete click on "Delete Icon" and click "OK"
### Was this PR authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this
PR,
please include the phrase: 'Generated-by: ' followed by the name of the
tool
and its version. If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for
details.
-->
Co-authored using: ChatGPT (OpenAI)
---------
Co-authored-by: Kunwoo (Chris) <[email protected]>
---
.../pythonvirtualenvironment/PveManager.scala | 93 ++++++++++++++++++++++
.../pythonvirtualenvironment/PveResource.scala | 28 +++++++
.../pythonvirtualenvironment/PveResourceSpec.scala | 41 ++++++++++
.../computing-unit-selection.component.html | 33 +++++++-
.../computing-unit-selection.component.ts | 88 +++++++++++++++++++-
.../virtual-environment.service.ts | 9 +++
frontend/src/styles.scss | 59 ++++++++------
7 files changed, 320 insertions(+), 31 deletions(-)
diff --git
a/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveManager.scala
b/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveManager.scala
index 27a3b7be7c..fce636a382 100644
---
a/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveManager.scala
+++
b/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveManager.scala
@@ -341,4 +341,97 @@ object PveManager {
queue.put(s"[user-package] $pkg")
}
}
+
+ /**
+ * Uninstalls a user-installed package from the PVE.
+ * 1. Prevents deletion of system packages
+ * 2. Updates user metadata upon success
+ * 3. Returns status messages
+ */
+ def deletePackages(
+ cuid: Int,
+ packageName: String,
+ pveName: String,
+ isLocal: Boolean
+ ): List[String] = {
+ val python = pythonBinPath(cuid, pveName).toAbsolutePath.toString
+ val metadataPath = cuidDir(cuid, pveName).resolve("user-packages.txt")
+
+ if (!Files.exists(Paths.get(python))) {
+ val msg = s"[PVE][ERR] Python executable not found for PVE: $python"
+ println(msg)
+ return List(msg)
+ }
+
+ val trimmedPackageName = packageName.trim
+ val normalizedPackageName =
trimmedPackageName.split("==")(0).trim.toLowerCase
+
+ val systemPackages =
+ if (Files.exists(getSystemPath(isLocal))) {
+ Files
+ .readAllLines(getSystemPath(isLocal))
+ .asScala
+ .map(_.trim)
+ .filter(line => line.nonEmpty && !line.startsWith("#"))
+ .map(line => line.split("==")(0).trim.toLowerCase)
+ .toSet
+ } else {
+ Set[String]()
+ }
+
+ if (systemPackages.contains(normalizedPackageName)) {
+ return List(
+ s"[PVE][ERR] $trimmedPackageName is a system package and cannot be
deleted."
+ )
+ }
+
+ try {
+ val command = Process(
+ Seq(
+ python,
+ "-u",
+ "-m",
+ "pip",
+ "uninstall",
+ "-y",
+ trimmedPackageName
+ ),
+ None,
+ pipEnv.toSeq: _*
+ )
+
+ val output = scala.collection.mutable.ListBuffer[String]()
+
+ val exitCode = command.!(
+ ProcessLogger(
+ out => {
+ println(s"[pip] $out")
+ output += s"[pip] $out"
+ },
+ err => {
+ System.err.println(s"[pip][ERR] $err")
+ output += s"[pip][ERR] $err"
+ }
+ )
+ )
+
+ if (exitCode == 0) {
+ val updatedPackages = readPackageFile(metadataPath)
+ .filterNot(line => line.split("==")(0).trim.toLowerCase ==
normalizedPackageName)
+ .sorted
+
+ Files.write(metadataPath, updatedPackages.asJava)
+
+ output += s"[pip] uninstall($trimmedPackageName) finished with exit
code $exitCode"
+ output += s"[PVE] Uninstalled $trimmedPackageName successfully"
+ } else {
+ output += s"[PVE][ERR] Failed to uninstall package:
$trimmedPackageName"
+ }
+
+ output.toList
+ } catch {
+ case e: Exception =>
+ List(s"[PVE][ERR] Failed to delete package for cuid=$cuid:
${e.getMessage}")
+ }
+ }
}
diff --git
a/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResource.scala
b/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResource.scala
index 8a6f487529..4d810678cc 100644
---
a/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResource.scala
+++
b/amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResource.scala
@@ -23,6 +23,9 @@ import javax.ws.rs._
import javax.ws.rs.core.MediaType
import scala.jdk.CollectionConverters._
import java.util
+import javax.ws.rs.DELETE
+import javax.ws.rs.PathParam
+import javax.ws.rs.core.Response
@Path("/pve")
@Consumes(Array(MediaType.APPLICATION_JSON))
@@ -85,4 +88,29 @@ class PveResource {
def deleteEnvironments(@PathParam("cuId") cuid: Int): Unit = {
PveManager.deleteEnvironments(cuid)
}
+
+ // --------------------------------------------------
+ // Delete User Installed Package
+ // --------------------------------------------------
+ @DELETE
+ @Path("/{cuid}/{pveName}/packages/{packageName}")
+ def deletePackage(
+ @PathParam("cuid") cuid: Int,
+ @PathParam("pveName") pveName: String,
+ @PathParam("packageName") packageName: String,
+ @QueryParam("isLocal") isLocal: Boolean
+ ): Response = {
+ val messages = PveManager.deletePackages(
+ cuid,
+ packageName,
+ pveName,
+ isLocal
+ )
+
+ if (messages.exists(_.contains("[PVE][ERR]"))) {
+
Response.status(Response.Status.BAD_REQUEST).entity(messages.asJava).build()
+ } else {
+ Response.ok(messages.asJava).build()
+ }
+ }
}
diff --git
a/amber/src/test/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResourceSpec.scala
b/amber/src/test/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResourceSpec.scala
index 10e952c8bd..f8b365c003 100644
---
a/amber/src/test/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResourceSpec.scala
+++
b/amber/src/test/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveResourceSpec.scala
@@ -98,6 +98,47 @@ class PveResourceSpec extends AnyFlatSpec with Matchers with
BeforeAndAfterEach
pve.get.userPackages should contain(packageSpec)
}
+ "PveManager" should "delete a user package and remove it from the PVE
package list" in {
+ PveManager.createNewPve(testCuid, queue, testPveName, isLocal = true)
+
+ val packageName = "colorama"
+ val packageVersion = "0.4.6"
+ val packageSpec = s"$packageName==$packageVersion"
+
+ queue.clear()
+
+ PveManager.installUserPackages(
+ List(packageSpec),
+ testCuid,
+ queue,
+ testPveName,
+ isLocal = true
+ )
+
+ PveManager
+ .getEnvironments(testCuid)
+ .find(_.pveName == testPveName)
+ .get
+ .userPackages should contain(packageSpec)
+
+ val deleteLogs = PveManager.deletePackages(
+ testCuid,
+ packageName,
+ testPveName,
+ isLocal = true
+ )
+
+ deleteLogs.mkString("\n") should not include "[PVE][ERR]"
+ deleteLogs.mkString("\n") should include(s"[PVE] Uninstalled $packageName
successfully")
+
+ val pve = PveManager
+ .getEnvironments(testCuid)
+ .find(_.pveName == testPveName)
+
+ pve should not be empty
+ pve.get.userPackages should not contain packageSpec
+ }
+
"PveManager" should "delete all PVEs for a computing unit" in {
PveManager.createNewPve(testCuid, queue, testPveName, isLocal = true)
diff --git
a/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.html
b/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.html
index 6f16073073..6167d36759 100644
---
a/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.html
+++
b/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.html
@@ -554,17 +554,31 @@
[ngModel]="pkg.version"
[disabled]="true" />
</div>
+ <button
+ nz-button
+ nzType="default"
+ nzShape="circle"
+ nzDanger
+ [ngClass]="{ 'highlighted-btn': pkg.deleteToggle }"
+ (click)="togglePackageDelete(envIndex, pkg)">
+ <i
+ nz-icon
+ nzType="delete"></i>
+ </button>
</div>
</div>
<!-- NEW PACKAGES -->
<div class="new-packages-section">
<div
- class="user-package-header-row"
+ class="package-row user-package-header-row"
*ngIf="pve.newPackages.length > 0">
- <div class="package-column-label">Package</div>
- <label style="visibility: hidden">Op</label>
- <div class="package-column-label">Version</div>
+ <div class="user-package-inputs">
+ <div class="package-column-label">Package</div>
+ <div></div>
+ <div class="package-column-label">Version</div>
+ <div></div>
+ </div>
</div>
<div
@@ -601,6 +615,17 @@
placeholder="Package Version"
[(ngModel)]="pve.newPackages[i].version" />
</div>
+ <button
+ nz-button
+ nzType="default"
+ nzShape="circle"
+ nzDanger
+ [ngClass]="{ 'highlighted-btn': pkg.deleteToggle }"
+ (click)="togglePackageDelete(envIndex, pkg)">
+ <i
+ nz-icon
+ nzType="delete"></i>
+ </button>
</div>
</div>
</div>
diff --git
a/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.ts
b/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.ts
index 1b3dfb2322..a2843a51bf 100644
---
a/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.ts
+++
b/frontend/src/app/workspace/component/power-button/computing-unit-selection.component.ts
@@ -81,12 +81,14 @@ type PveUserPackageRow = {
name: string;
versionOp?: "==" | ">=" | "<=";
version?: string;
+ deleteToggle?: boolean;
};
type PveDraft = {
name: string;
userPackages: PveUserPackageRow[];
newPackages: PveUserPackageRow[];
+ deletingPackages: { name: string; version: string }[];
pipOutput: string;
prettyPipOutput: string;
expanded: boolean;
@@ -732,7 +734,21 @@ export class ComputingUnitSelectionComponent implements
OnInit {
addPackage(index: number): void {
const env = this.pves[index];
- env.newPackages.push({ name: "", version: "", versionOp: undefined });
+ env.newPackages.push({ name: "", version: "", versionOp: undefined,
deleteToggle: false });
+ }
+
+ togglePackageDelete(index: number, pkg: PveUserPackageRow): void {
+ const env = this.pves[index];
+
+ pkg.deleteToggle = !pkg.deleteToggle;
+
+ const version = pkg.version ?? "";
+
+ env.deletingPackages = env.deletingPackages.filter(p => !(p.name ===
pkg.name && p.version === version));
+
+ if (pkg.deleteToggle) {
+ env.deletingPackages.push({ name: pkg.name, version });
+ }
}
addEnvironment(): void {
@@ -740,6 +756,7 @@ export class ComputingUnitSelectionComponent implements
OnInit {
name: "",
userPackages: [],
newPackages: [],
+ deletingPackages: [],
pipOutput: "",
prettyPipOutput: "",
expanded: true,
@@ -776,6 +793,7 @@ export class ComputingUnitSelectionComponent implements
OnInit {
name: pve.pveName,
userPackages: this.parsePackageRows(pve.userPackages),
newPackages: [],
+ deletingPackages: [],
expanded: false,
isInstalling: false,
pipOutput: "",
@@ -949,6 +967,7 @@ export class ComputingUnitSelectionComponent implements
OnInit {
createVirtualEnvironment(index: number): void {
const env = this.pves[index];
const trimmedName = env.name.trim();
+ const isLocal = this.selectedComputingUnit?.computingUnit.type === "local";
if (!/^[a-zA-Z0-9]+$/.test(trimmedName)) {
this.notificationService.error("Environment name must contain only
letters and numbers.");
@@ -956,7 +975,9 @@ export class ComputingUnitSelectionComponent implements
OnInit {
}
if (env.isLocked) {
- this.installUserPackages(index);
+ this.deleteUserPackages(index, () => {
+ this.installUserPackages(index);
+ });
return;
}
@@ -968,7 +989,9 @@ export class ComputingUnitSelectionComponent implements
OnInit {
}
this.runPveWebSocket(index, "create", "Creating virtual environment...\n",
[], () => {
- this.installUserPackages(index);
+ this.deleteUserPackages(index, () => {
+ this.installUserPackages(index);
+ });
});
}
@@ -1019,6 +1042,7 @@ export class ComputingUnitSelectionComponent implements
OnInit {
if (packageArray.length === 0) {
this.pves[index].newPackages = [];
+ this.pves[index].isInstalling = false;
this.refreshUserPackages(index);
return;
}
@@ -1039,4 +1063,62 @@ export class ComputingUnitSelectionComponent implements
OnInit {
};
});
}
+
+ private deleteUserPackages(index: number, onDone?: () => void): void {
+ const cuId = this.selectedComputingUnit!.computingUnit.cuid;
+ const isLocal = this.selectedComputingUnit?.computingUnit.type === "local";
+ const pveName = this.pves[index].name.trim();
+ const packagesToDelete = [...this.pves[index].deletingPackages];
+
+ if (packagesToDelete.length === 0) {
+ onDone?.();
+ return;
+ }
+
+ this.pves[index] = {
+ ...this.pves[index],
+ pipOutput: `${this.pves[index].pipOutput ?? ""}Deleting user
packages...\n`,
+ isInstalling: true,
+ };
+
+ let deleteIndex = 0;
+
+ const deleteNext = (): void => {
+ if (deleteIndex >= packagesToDelete.length) {
+ this.pves[index].deletingPackages = [];
+ this.refreshUserPackages(index);
+ onDone?.();
+ return;
+ }
+
+ const pkg = packagesToDelete[deleteIndex];
+
+ this.workflowPveService
+ .deletePackage(cuId, pveName, pkg.name, isLocal)
+ .pipe(untilDestroyed(this))
+ .subscribe({
+ next: messages => {
+ this.pves[index].pipOutput = `${this.pves[index].pipOutput ??
""}${messages.join("\n")}\n`;
+
+ this.updatePrettyPipOutput(index);
+ this.scrollToBottomOfPipModal(index);
+
+ deleteIndex++;
+ deleteNext();
+ },
+ error: () => {
+ this.pves[index].pipOutput =
+ `${this.pves[index].pipOutput ?? ""}[PVE][ERR] Failed to delete
package: ${pkg.name}\n`;
+
+ this.updatePrettyPipOutput(index);
+ this.scrollToBottomOfPipModal(index);
+
+ deleteIndex++;
+ deleteNext();
+ },
+ });
+ };
+
+ deleteNext();
+ }
}
diff --git
a/frontend/src/app/workspace/service/virtual-environment/virtual-environment.service.ts
b/frontend/src/app/workspace/service/virtual-environment/virtual-environment.service.ts
index 9c7c123df1..7788cba270 100644
---
a/frontend/src/app/workspace/service/virtual-environment/virtual-environment.service.ts
+++
b/frontend/src/app/workspace/service/virtual-environment/virtual-environment.service.ts
@@ -67,6 +67,15 @@ export class WorkflowPveService {
return this.http.delete(`/pve/pves/${cuid}`);
}
+ deletePackage(cuid: number, pveName: string, packageName: string, isLocal:
boolean) {
+ const params = this.buildBaseParams().set("isLocal", isLocal.toString());
+
+ return this.http.delete<string[]>(
+
`/pve/${cuid}/${encodeURIComponent(pveName)}/packages/${encodeURIComponent(packageName)}`,
+ { params }
+ );
+ }
+
getPveWebSocketUrl(cuid: number, pveName: string, isLocal: boolean, action:
string, packages: string[] = []): string {
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
const query = encodeURIComponent(JSON.stringify(packages));
diff --git a/frontend/src/styles.scss b/frontend/src/styles.scss
index 27b2a5362a..5f4a00952a 100644
--- a/frontend/src/styles.scss
+++ b/frontend/src/styles.scss
@@ -55,14 +55,6 @@ img {
padding-bottom: 0;
}
-.ant-select-selector {
- height: 24px !important;
-}
-
-.ant-select-selection-item {
- line-height: 24px !important;
-}
-
.ant-form-item-label {
padding-bottom: 0 !important;
}
@@ -236,6 +228,21 @@ hr {
padding: 14px !important;
}
+ .operator-select {
+ height: 24px;
+ }
+
+ .operator-select nz-select,
+ .operator-select .ant-select,
+ .operator-select .ant-select-selector {
+ height: 24px !important;
+ }
+
+ .operator-select .ant-select-selection-item,
+ .operator-select .ant-select-selection-placeholder {
+ line-height: 22px !important;
+ }
+
.ve-form {
display: flex;
flex-direction: column;
@@ -262,7 +269,7 @@ hr {
.package-row {
display: flex;
- align-items: flex-end;
+ align-items: center;
justify-content: space-between;
gap: 10px;
bottom: 0px;
@@ -281,9 +288,10 @@ hr {
.user-package-inputs {
flex: 1;
display: grid;
- grid-template-columns: 1fr 160px 1fr;
+ grid-template-columns: 1fr 160px 1fr 58px;
gap: 14px;
width: 100%;
+ align-items: center;
}
.field {
@@ -309,11 +317,6 @@ hr {
width: 100%;
}
- .ant-input,
- .ant-select-selector {
- //border-radius: 10px !important;
- }
-
.ant-input[disabled] {
background: #f5f6f8 !important;
border-color: #e6e8ec !important;
@@ -369,22 +372,30 @@ hr {
background: transparent;
}
- .user-package-header-row .package-column-label {
- font-weight: 600;
- }
-
.new-packages-section {
display: flex;
flex-direction: column;
gap: 2px;
}
+ .user-package-inputs button {
+ align-self: center;
+ justify-self: center;
+ }
+
+ .highlighted-btn {
+ background-color: #ff4d4f !important; /* Ant Design red */
+ border-color: #ff4d4f !important;
+ color: white !important;
+ }
.user-package-header-row {
- display: grid;
- grid-template-columns: 1fr 160px 1fr;
- gap: 14px;
- margin-bottom: 0;
- padding: 0;
+ border: none;
+ background: transparent;
+ align-items: center;
+ }
+
+ .user-package-header-row .package-column-label {
+ font-weight: 600;
}
.system-header {
