The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-6811-6de37efa301b4635e3f5189d8872b7b450dca982 has failed. Run started by GitHub user aglinxinyuan (triggered by aglinxinyuan).
Head commit for run: 72522b0637ff496ae1041506b5142295589befb9 / roshiiiiz <[email protected]> feat(operator): provide user-friendly error message when binary file scan hits memory limit (#6811) <!-- Thanks for sending a pull request (PR)! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: [Contributing to Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) 2. Ensure you have added or run the appropriate tests for your PR 3. If the PR is work in progress, mark it a draft on GitHub. 4. Please write your PR title to summarize what this PR proposes, we are following Conventional Commits style for PR titles as well. 5. Be sure to keep the PR description updated to reflect all changes. --> ### What changes were proposed in this PR? <!-- Please clarify what changes you are proposing. The purpose of this section is to outline the changes. Here are some tips for you: 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, you can clarify why it is a bug. 3. If it is a refactoring, clarify what has been changed. 3. It would be helpful to include a before-and-after comparison using screenshots or GIFs. 4. Please consider writing useful notes for better and faster reviews. --> This PR improves the user experience for the File Scan operator's in-memory read path by gracefully catching natural Java memory limits and surfacing a helpful UI error message, rather than allowing the worker JVM to crash. **Why is it needed?** Previously, when users mistakenly attempted to read massive files using the `binary` attribute type (instead of the streaming `large binary` type), `ByteArrayOutputStream` would attempt to allocate the entire file into memory. If the file exceeded the JVM's available heap space or the maximum Java array size, it triggered an unhandled `OutOfMemoryError` or `IllegalArgumentException`, causing the `computing-unit-master` to lock up or crash entirely without reporting a user-friendly error to the frontend UI. **What was changed:** - Wrapped the stream reader in `FileScanUtils.safeToByteArray` with a `try-catch` block. - Intercepts natural `OutOfMemoryError` and `IllegalArgumentException` thrown by the JVM or `ByteArrayOutputStream`. - Throws a clean, user-friendly `RuntimeException` directly to the frontend directing the user to use the `large binary` attribute type instead for massive files. *(Note: Based on maintainer feedback, an initial hardcoded size threshold approach was dropped in favor of this cleaner architectural approach that relies on natural JVM limits).* ### Any related issues, documentation, discussions? <!-- Please use this section to link other resources if not mentioned already. 1. If this PR fixes an issue, please include `Fixes #1234`, `Resolves #1234` or `Closes #1234`. If it is only related, simply mention the issue number. 2. If there is design documentation, please add the link. 3. If there is a discussion in the mailing list, please add the link. --> Closes #3271 ### How was this PR tested? <!-- If tests were added, say they were added here. Or simply mention that if the PR is tested with existing test cases. Make sure to include/update test cases that check the changes thoroughly including negative and positive cases if possible. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. --> **Manual Verification:** 1. Uploaded an 8.9 GB CSV test file to the workspace. 2. Created a workflow with the `File Scan` operator configured to use the `binary` attribute type (which intentionally attempts to load the entire file into memory). 3. Ran the workflow. **Result:** The workflow caught the JVM's `OutOfMemoryError` when attempting the massive allocation, safely aborted the thread, and successfully threw the custom user-friendly error message in the UI without crashing the server. **Automated Tests:** - Updated the mock in `FileScanUtilsSpec.scala` to simulate a natural `OutOfMemoryError` being thrown during stream reading to prove the `catch` block reliably intercepts it and translates it to the user-friendly exception. ### Was this PR authored or co-authored using generative AI tooling? <!-- If generative AI tooling has been used in the process of authoring this PR, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> Generated-by: Antigravity (DeepMind) --------- Co-authored-by: probe <probe@x> Co-authored-by: Yicong Huang <[email protected]> Co-authored-by: Xinyuan Lin <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/31672952126 With regards, GitHub Actions via GitBox
