goutamadwant opened a new issue, #12319:
URL: https://github.com/apache/seatunnel/issues/12319
### Search before asking
- [x] I searched existing issues and pull requests and found no matching
array-length fix in `LengthValidationRule`.
### What happened
DataValidator documents LENGTH validation for arrays, but
`LengthValidationRule` handles only byte arrays directly. Other arrays fall
through to `value.toString().length()`, which measures their Java identity
string rather than their elements.
As a result, a valid two-element `Integer[]` fails `exact_length = 2`.
Through DataValidatorTransform, this can fail a job, silently skip a valid row,
or route it to the error table, depending on the configured policy.
### SeaTunnel Version
Reproduced against unreleased `3.0.0-SNAPSHOT`, commit
`0d9f9e2303da0b40b20e55fd341d564757e86dc0`. Current `dev` at
`f4a9665e8457239dd30c04a8738b506fa3fce180` leaves the affected implementation
unchanged. Released-version coverage has not been established.
### SeaTunnel Config and reproduction
The reproduction uses a typed `items ARRAY<INT>` field containing `new
Integer[] {10, 20}` and invokes DataValidatorTransform directly with these
options:
```hocon
field_rules = [
{
field_name = "items"
rule_type = "LENGTH"
exact_length = 2
}
]
row_error_handle_way = "FAIL"
```
The underlying rule can also be reproduced directly:
```java
new LengthValidationRule(2)
.validate(new Integer[] {10, 20}, ArrayType.INT_ARRAY_TYPE, null)
.isValid();
```
Expected: `true`. Actual before the fix: `false`.
Repeat the transform-level case with `SKIP` and `ROUTE_TO_TABLE`: the same
valid array row is incorrectly skipped or routed. The regression tests also
verify that genuinely invalid arrays still follow the configured policy after
the fix.
### Running Command
The accompanying regression class is
[DataValidatorArrayLengthTest.java](https://github.com/goutamadwant/seatunnel/blob/21cceffb710ec8a434045f6b4b5ee0a8d779bd46/seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/validator/DataValidatorArrayLengthTest.java).
Apply only that test file to the reproduction revision, leaving production
code unchanged, and run:
```sh
./mvnw -B -pl seatunnel-transforms-v2 -am package \
-Dtest=DataValidatorArrayLengthTest \
-Dsurefire.failIfNoSpecifiedTests=false -Dskip.spotless=true -Dskip.ui=true
```
Repeat with Java 8 and Java 11 selected through `JAVA_HOME`. Run the same
command with the production fix applied to verify the after behavior.
### Error Exception
The LENGTH error reports the Java identity-string length rather than the
array's element count. That number depends on the array class and identity
string; it is not a stable element count. With the FAIL policy,
DataValidatorTransform throws `TransformException` for the valid two-element
row.
On both Java 8 and Java 11, the 23 regression cases against unchanged
production code produced 17 assertion failures and one production validation
exception for a valid array. With the fix, all 23 cases pass on both JDKs.
### Before and after
- Before: non-byte arrays are validated using their identity-string length.
- After: arrays are detected with `getClass().isArray()` and measured using
`Array.getLength`.
- The count is the outer array length, including null elements. Nested
arrays are not traversed or flattened.
- Existing byte-array, null, String UTF-16 length, Collection size, fallback
handling, custom messages and invalid-row policies are preserved.
### Advantages
- Keeps valid array-valued records in the normal pipeline.
- Rejects genuinely invalid element counts consistently across object and
primitive arrays.
- Uses an existing JDK operation with constant-time outer-length lookup,
without traversing elements or adding dependencies.
### Breaking changes and migration
This is an observable validation correction: some array-valued rows will be
accepted or rejected differently. Review limits adjusted to the old behavior
and check previously skipped/routed data before replay. There are no new
options, changed defaults, schemas or state-format changes. English and Chinese
documentation and upgrade notes accompany the fix.
### Validation
After rebasing onto `f4a9665e8457`, all 39 DataValidator tests and
formatting checks passed on Java 8 and Java 11. Production and test changes are
unchanged from the broader validation below.
The 23 new regression cases cover boxed, primitive, nested, empty and null
arrays, min/max/exact boundaries, custom errors and all three transform-level
error policies. The full 1,180-case transform suite passed on Java 8 and Java
11 before rebasing the patch. Java 11 root `./mvnw -q -DskipTests verify` also
passed, including distribution packaging, on the original base.
An initial broader Java 8 run encountered an unrelated Python
executable-path/allowlist mismatch. Putting the real interpreter directory
first on PATH resolved it without changing the allowlist. The retry reran all
transform tests; only an already-passing dependency date/time performance
method was omitted. Java 11 had no test exclusions. No Docker-backed E2E or
deployed workload result is claimed.
### Zeta or Flink or Spark Version
Direct DataValidatorTransform tests with typed SeaTunnel rows. No external
engine cluster was used.
### Java or Scala Version
Java 8 and Java 11. No Scala changes.
### Are you willing to submit PR?
- [x] Yes, I am willing to submit a PR.
### Code of Conduct
- [x] I agree to follow the project's Code of Conduct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
