This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-7277-d8bb0f180fa0ea12f25babd5af5dd24e316c3477 in repository https://gitbox.apache.org/repos/asf/texera.git
commit f3231c8d1b7b664ce6a3f716423b892f6cd329aa Author: Xinyuan Lin <[email protected]> AuthorDate: Mon Aug 3 12:37:01 2026 -0700 refactor(storage): remove dead members from LakeFSStorageClient (#7277) ### What changes were proposed in this PR? `LakeFSStorageClient` carried four members with no production caller. Each was verified dead by a repo-wide grep before deleting: | Removed | Callers before | Why it is safe | | --- | --- | --- | | `stagingApi` | none at all | Private field, unreferenced even inside the class. `StagingApi` arrived through the wildcard `io.lakefs.clients.sdk._`, so no import change is needed. | | `removeFileFromRepo` | none | Every other grep hit is `GitVersionControlLocalFileStorage.removeFileFromRepo(Path, Path)` — a different method, still live. | | `retrieveFileContent` | none | Byte-for-byte duplicate of `getFileFromRepo`, which stays live (`DatasetFileDocument`, `DatasetResource`). | | `withCreateVersion` | 1 test | Only `DatasetResourceSpec`, passing an empty `operations` block. | That one call site moves to the live equivalent. Behaviour is identical: the block it ran was empty, and the old method hardcoded the same `branchName = "main"`. ```diff - val commit = LakeFSStorageClient.withCreateVersion(repoName, "commit all files") {} + val commit = LakeFSStorageClient.createCommit(repoName, "main", "commit all files") ``` `writeFileToRepo` is deliberately **kept**. It has no production caller either, but `DatasetResourceSpec`, `StagedFileCleanupJobSpec` and `LakeFSStorageClientMtimeSpec` all use it to stage a file, and with `stagingApi` gone it is the only public staging entry point left. Also fixes the `getFilePresignedUrl` scaladoc, which was copied from `retrieveFileContent`: it described retrieving file content and documented no return value, on a method that returns a presigned URL `String`. ### Any related issues, documentation, discussions? Closes #7276 Touches the same class as #7273, but not the same files — that PR only names these members in a "left uncovered on purpose" comment, so there is no conflict. Whichever of the two lands second should drop that comment's second bullet, since the members it points at will no longer exist. ### How was this PR tested? Deletion only — no behaviour change, no assertion edits, so the gate is that nothing still refers to these members and the one moved call site still compiles. ``` sbt "WorkflowCore/Test/compile" "FileService/Test/compile" ``` ``` sbt "WorkflowCore/scalafmtCheck" "WorkflowCore/Test/scalafmtCheck" "FileService/Test/scalafmtCheck" "WorkflowCore/scalafix --check" "WorkflowCore/Test/scalafix --check" "FileService/Test/scalafix --check" ``` Both green. The Compile config is included because the deleted members live in main sources, which `WorkflowCore/Test/*` alone would not lint. The LakeFS-backed suites that call `writeFileToRepo` (`DatasetResourceSpec`, `StagedFileCleanupJobSpec`) stand up postgres / MinIO / LakeFS via testcontainers and need a Docker daemon, which was not available on this machine; CI runs them. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 5) --- .../core/storage/util/LakeFSStorageClient.scala | 44 ++-------------------- .../service/resource/DatasetResourceSpec.scala | 2 +- 2 files changed, 4 insertions(+), 42 deletions(-) diff --git a/common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClient.scala b/common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClient.scala index e79bf63b1d..d46fe8341e 100644 --- a/common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClient.scala +++ b/common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClient.scala @@ -66,7 +66,6 @@ object LakeFSStorageClient extends LazyLogging { private lazy val branchesApi: BranchesApi = new BranchesApi(apiClient) private lazy val commitsApi: CommitsApi = new CommitsApi(apiClient) private lazy val refsApi: RefsApi = new RefsApi(apiClient) - private lazy val stagingApi: StagingApi = new StagingApi(apiClient) private lazy val experimentalApi: ExperimentalApi = new ExperimentalApi(apiClient) private lazy val healthCheckApi: HealthCheckApi = new HealthCheckApi(apiClient) @@ -159,50 +158,13 @@ object LakeFSStorageClient extends LazyLogging { } /** - * Removes a file from the repository (similar to Git rm). - * - * @param repoName Repository name. - * @param branch Branch name. - * @param filePath Path in the repository to delete. - */ - def removeFileFromRepo(repoName: String, branch: String, filePath: String): Unit = { - objectsApi.deleteObject(repoName, branch, filePath).execute() - } - - /** - * Executes operations and creates a commit (similar to a transactional commit). - * - * @param repoName Repository name. - * @param commitMessage Commit message. - * @param operations File operations to perform before committing. - */ - def withCreateVersion(repoName: String, commitMessage: String)( - operations: => Unit - ): Commit = { - operations - val commit = new CommitCreation() - .message(commitMessage) - - commitsApi.commit(repoName, branchName, commit).execute() - } - - /** - * Retrieves file content from a specific commit and path. - * - * @param repoName Repository name. - * @param commitHash Commit hash of the version. - * @param filePath Path to the file in the repository. - */ - def retrieveFileContent(repoName: String, commitHash: String, filePath: String): File = { - objectsApi.getObject(repoName, commitHash, filePath).execute() - } - - /** - * Retrieves file content from a specific commit and path. + * Generates a presigned URL for downloading a file directly from the underlying object store, + * bypassing the LakeFS server. * * @param repoName Repository name. * @param commitHash Commit hash of the version. * @param filePath Path to the file in the repository. + * @return A time-limited presigned URL pointing at the object's physical address. */ def getFilePresignedUrl(repoName: String, commitHash: String, filePath: String): String = { objectsApi.statObject(repoName, commitHash, filePath).presign(true).execute().getPhysicalAddress diff --git a/file-service/src/test/scala/org/apache/texera/service/resource/DatasetResourceSpec.scala b/file-service/src/test/scala/org/apache/texera/service/resource/DatasetResourceSpec.scala index 7ca1e9429c..4cca045bf3 100644 --- a/file-service/src/test/scala/org/apache/texera/service/resource/DatasetResourceSpec.scala +++ b/file-service/src/test/scala/org/apache/texera/service/resource/DatasetResourceSpec.scala @@ -3351,7 +3351,7 @@ class DatasetResourceSpec LakeFSStorageClient.retrieveUncommittedObjects(repoName).size shouldEqual totalFiles // after commit: 110 files should appear as committed objects - val commit = LakeFSStorageClient.withCreateVersion(repoName, "commit all files") {} + val commit = LakeFSStorageClient.createCommit(repoName, "main", "commit all files") LakeFSStorageClient.retrieveObjectsOfVersion(repoName, commit.getId).size shouldEqual totalFiles } }
