bamaer opened a new issue, #8955:
URL: https://github.com/apache/hop/issues/8955
### Apache Hop version?
2.20.0-SNAPSHOT
### Java version?
21
### Operating system
macOS
### What happened?
Updating a zip that already exists fails when the zip is on a VFS file
system, such as a named S3 connection (`mys3://bucket/folder/archive.zip`).
This affects:
- the **Zip File** transform with "Overwrite zip entry" enabled
- the **Zip file** action with "If zip file exists: Append"
Creating a new zip on the same connection works in both.
In both plugins, the update renames the existing zip to a temporary file and
reads it back with `java.io.File` (`getFile()`, `File.createTempFile`,
`File.renameTo`, `FileInputStream`) instead of VFS. A `java.io.File` cannot
point at a VFS URI.
#### Zip File transform
2.18.1 (2.19.0 has the same code):
```
Zip file.0 - Because of an error, this transform can't continue:
Error creating/updating zip file!
URI scheme is not "file"
```
SNAPSHOT-20261013 (after #8705, which also catches
`IllegalArgumentException` in `getFile()`):
```
Zip file.0 - Error creating/updating zip file!
No such file or directory
```
`getFile()` now falls back to `new
File("mys3:///bucket/folder/archive.zip")`, and `File.createTempFile` is called
on its parent, a local folder `mys3:/bucket/folder` that does not exist.
On 2.18.1, local paths fail as well, with `URI is not absolute`, unless the
path contains a character that is invalid in a URI, such as a space. #8705
fixed the local case on SNAPSHOT.
#### Zip file action
2.18.1: the rename fails and is only logged (`could not rename the file
...`). The action then writes a new zip that holds only the new files and fails
with `Unable to delete temporary file`. **The entries already in the zip are
lost.**
SNAPSHOT-20261013: fails with `java.io.IOException: No such file or
directory` from `File.createTempFile`. The existing zip is left unchanged.
#### Steps to reproduce
1. Create an S3 connection, for example `mys3`, and a bucket. An
S3-compatible store works too, for example SeaweedFS with a custom endpoint and
path-style access.
2. Create a pipeline:
- **Get file names**: two local files, `a.txt` and `b.txt`
- **Get variables**: field `zip_filename` =
`mys3://bucket/folder/archive.zip`
- **Zip file**: source filename field `filename`, target filename field
`zip_filename`, "Overwrite zip entry" and "Create parent folder" enabled
3. Run the pipeline. Row 1 creates `archive.zip`. Row 2 fails with the error
above. Once `archive.zip` exists, every following run fails on row 1.
For the action: a workflow with two **Zip file** actions on the same
`mys3://bucket/folder/archive.zip`, "If zip file exists: Append". The first one
zips `a.txt` and the second one zips `b.txt`. The second action fails as
described above.
#### Expected behaviour
The existing zip is updated through VFS and holds `a.txt` and `b.txt`, on
every VFS file system as well as locally. If the update fails, the original zip
is left intact.
#### Related problems in the same code
- **"Overwrite zip entry" does not overwrite.** The transform copies the
existing entries first and skips a new file whose name is already in the zip. A
zip that holds `a.txt` keeps its old `a.txt` after zipping a new `a.txt` with
this option enabled (verified on SNAPSHOT-20261013 with a local zip). From the
code, the action uses the same order when appending.
- **"Keep source folder" uses the full URI as the entry name.** With a VFS
source, the transform stores entries as `mys3:///bucket/in/src.txt`.
- **The transform can lose the original zip.** From the code: the original
zip is renamed to the temporary file before the new zip is written, and the
temporary file is deleted in a `finally` block. If writing fails, the only
remaining copy is deleted. The action does not delete the temporary file on
failure, which leaves the original under the temporary name.
#### Proposed fix
In both plugins:
1. Write the updated zip to a temporary sibling `FileObject`.
2. Copy the existing entries from `HopVfs.getInputStream(zipFile)`, then add
the new files.
3. On success, replace the original with `HopVfs.moveFile`. On failure,
delete the temporary file and leave the original untouched.
This removes `getFile()`, `File.createTempFile`, `File.renameTo` and
`FileInputStream` from the update path. Unit tests can use a `ram://` target.
### Issue Priority
Priority: 2
### Issue Component
Component: Transforms, Component: Actions, Component: VFS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
