goutamadwant opened a new pull request, #13249:
URL: https://github.com/apache/seatunnel/pull/13249

   <!--
   
   Thank you for contributing to SeaTunnel! Please make sure that your code 
changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   ## Contribution Checklist
     - Make sure that the pull request corresponds to a [GITHUB 
issue](https://github.com/apache/seatunnel/issues).
     - Name the pull request in the form "[Feature] [component] Title of the 
pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
     - Minor fixes should be named following this pattern: `[hotfix] [docs] Fix 
typo in README.md doc`.
   -->
   
   ### Purpose of this pull request
   
   <!-- Describe the purpose of this pull request. For example: This pull 
request adds checkstyle plugin.-->
   
   Prepare Python in Apache SeaTunnel E2E engine images before starting runtime 
containers.
   
   Fixes #13248.
   
   **Depends on #13247. Merge the G1 correction first.** The Python-only Java 
17 runs completed their jobs but failed the existing cleanup assertion on 
native G1 workers. The successful Java 17 results below combine both patches. 
This branch contains only the Python image-preparation change.
   
   - Replace runtime package installation in the existing Python 
source/transform suites with a shared test-image helper.
   - Preserve the selected engine image, platform, original USER and inherited 
JDK/startup configuration. Existing Python gates and executable allowlists 
remain unchanged.
   - Install only when Python is absent, propagate preparation failures, cache 
successful preparation per source image within the test JVM, and use normal 
Testcontainers temporary-image cleanup.
   - Add an offline/configuration regression to existing PythonIT and extend 
Spark provider-policy coverage.
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   <!--
   Note that it means *any* user-facing change including all aspects such as 
the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes 
- provide the console output, description and/or an example to show the 
behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to 
the released SeaTunnel versions or within the unreleased branches such as dev.
   If no, write 'No'.
   If you are adding/modifying connector documents, please follow our new 
specifications: https://github.com/apache/seatunnel/issues/4544.
   -->
   
   
   No production user-facing change.
   
   Before: start engine container, install Python from mirrors, then run the 
test. A runtime package DNS/mirror failure prevents any job from running.
   
   After: prepare the engine-derived image with Python, start the runtime 
container, verify the interpreter and run the existing job. Python execution no 
longer needs runtime package-mirror access.
   
   No public API, option, dependency, security policy or data-format changes. 
The Java 11 Zeta default, Flink 1.13 skip, engine selection and container 
lifecycles are preserved. Initial preparation still needs mirrors when Python 
is absent. No container reuse is added.
   
   ### How was this patch tested?
   
   <!--
   If tests were added, say they were added here. Please make sure to add some 
test cases that check the changes thoroughly including negative and positive 
cases if possible.
   If it was tested in a way different from regular unit tests, please clarify 
how you tested step by step, ideally copy and paste-able, so that other 
reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why 
it was difficult to add.
   If you are adding E2E test cases, maybe refer to 
https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/resources/mysqlcdc_to_mysql.conf,
 here is a good example.
   -->
   
   
   - Reproduced the original Java 11 setup failure with runtime networking 
disabled: package DNS errors, `Unable to locate package python3`, exit 100. The 
same base image lacked Python with normal networking. This does not identify 
the cause of any particular hosted DNS failure.
   - Java 11 Zeta: source 3/3 and transform 4/4 passed.
   - Java 17 Zeta with #13247: source 3/3 and transform 4/4 passed.
   - Staged Flink 1.20 AMD64 image under emulation, actual Java 11: transform 
4/4 passed.
   - These 18 passing project JUnit cases comprise 16 engine jobs and 2 
offline/configuration regressions. They ran through a standalone JUnit Platform 
launcher with lifecycle callbacks, not Maven Failsafe. Validation-only Java 17 
image and isolated mount substitutions were restored before commit.
   - Offline regression checks Python JSON output, inherited image metadata, 
equal baseline/prepared Java versions and availability of `jps`, `jstack` and 
`jmap`. Java 11/17 helper controls passed. A Spark 3.5 image control preserved 
nonroot USER/configuration and executed Python without runtime networking.
   - Three focused Spark provider-policy cases, scoped formatting and build 
verification passed. Existing source assertions retain the 
three-row/ID-range/nonnull-name checks, without claiming exact per-row equality 
for all values.
   
   Pending: full hosted matrix, official Flink 1.15/1.18 runtime images and 
Spark engine jobs. The staged Flink run and Spark image control do not 
establish those results. Hosted CI is pending.
   
   ### Check list
   
   * [ ] If any new Jar binary package adding in your PR, please add License 
Notice according
     [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
   * [ ] If necessary, please update the documentation to describe the new 
feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If necessary, please update `incompatible-changes.md` to describe the 
incompatibility caused by this PR.
   * [ ] If you are contributing the connector code, please check that the 
following files are updated:
     1. Update 
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
 and add new connector information in it
     2. Update the pom file of 
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
     3. Add ci label in 
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
     4. Add e2e testcase in 
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
     5. Update connector 
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
   
   Checklist applicability: no new JARs, connector registration, user 
documentation or incompatible changes are introduced.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to