bamaer commented on PR #9365:
URL: https://github.com/apache/hop/pull/9365#issuecomment-6711848318

   Thanks @leehaut, the Vector parsing part looks good.
   
   **Blocker**
   
   - `TextFileOutput.java:695`: with the `v.isString()` guard gone, every type 
is now enclosed when its text contains the separator or enclosure, not only 
Vector. With default settings (enclosure `"`):
     - JSON field, `;` separator: `{"a" : 1}` is now written as `"{""a"" : 
1}"`. That breaks JSON / NDJSON files written with Text File Output.
     - Binary field whose bytes contain `"` or `;`: the content is now wrapped 
in quotes and the `"` bytes are doubled, so the binary file written is 
corrupted.
     - Number / Date whose format contains the separator (`1,234.50`, `Jan 1, 
1970`) are now quoted too, which changes existing output on upgrade.
   
     Suggested fix: keep the new branch to String and Vector:
     ```java
     } else if ((v.isString() || v.getType() == IValueMeta.TYPE_VECTOR)
         && !meta.isEnclosureFixDisabled()
         && containsSeparatorOrEnclosure(str, data.binarySeparator, 
data.binaryEnclosure)) {
     ```
     and build the test helper's value meta with `IValueMeta.TYPE_VECTOR` 
instead of `TYPE_NUMBER`. With that change `TextFileOutputTest` passes and 
JSON, Binary, Number and Date output is the same as on main.
   
   **Should fix**
   
   - Please add an integration test for the round trip from the issue in 
`integration-tests/transforms`, e.g. `0126-vector-csv-round-trip.hpl` plus 
`main-0126-vector-csv-round-trip.hwf`. Write Vector rows with Text File Output 
(separator `,`, enclosure `"`), read the file back with CSV File Input as 
Vector, and compare against a data set. 
`0118-boolean-format-text-file-output.hpl` / 
`0118-boolean-format-read-back.hpl` do the same for Booleans.
   
   Verified: built the PR and ran `TextFileOutputTest` and 
`ValueMetaVectorTest` (all pass, checkstyle clean). The output comparison above 
is from a scratch unit test against main and this PR. Integration tests not run.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to