The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-7568-76ad40d5a0013a8f9cf610d0b33d0e9254042342 has failed. Run started by GitHub user aglinxinyuan (triggered by aglinxinyuan).
Head commit for run: 792b5b0b74fe41c689d062418f86d15ce6a50609 / Kary Zheng <[email protected]> feat(csv-scan): keep a numeric column's type when one of its cells is blank (#7568) ### What changes were proposed in this PR? Schema inference and execution disagreed about what a blank cell is, and the schema side was the one that lost information. Inference set `nullValue("")` on its parser, so a blank read as an empty string: `tryParseDouble("")` fails, `tryParseBoolean("")` fails, and `inferField` lands on `tryParseString()`. One empty cell was enough to type a whole numeric column as STRING. Execution builds its parser without `nullValue`, so the same blank read as null there, which is what `AttributeTypeUtils.parseField` is written to pass through. The effect reaches well past the scan. Every downstream operator that does arithmetic on such a column then receives strings and fails on rows whose values are perfectly good numbers, not on the blank one. Hugging Face Iris Logistic Regression on a three-row file fails at the first row, where numpy is handed `array([['2.6', '0.75']], dtype='<U32')`. Dropping the setting leaves both sides reading a blank as null, and `tryParseDouble(null)` already answers DOUBLE, so the column keeps the type its values give it. One corner changes with it: a column that is blank in every sampled row now infers as INTEGER rather than STRING. Its values are null either way, so this renames the empty rather than reinterpreting anything. ### Any related issues, documentation, discussions? Closes #7550 ### How was this PR tested? `CSVScanSourceOpDescSpec` gains a case that writes a two-column file whose numeric column is blank on one row and asserts the inferred type is DOUBLE. It fails on the previous behavior, 16 passed / 1 failed before the change and 17 / 0 after. Since this touches inference every scan goes through, the whole module was run as well: 2187 passed, 0 failed. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/31673962124 With regards, GitHub Actions via GitBox
