mattcasters opened a new pull request, #8715:
URL: https://github.com/apache/hop/pull/8715
This pull request works through the open items under umbrella #8711. JSON
Input keeps its current defaults. Enhanced JSON Output writes the file as rows
arrive, and newline-delimited JSON is a separate file option that stays off
until it is selected.
Integration workflows `0018-enhanced-json-output-streaming` and
`0019-enhanced-json-output-grouped-file` passed against the client assembly
built from this branch. The streaming run wrote one million rows in a hop-run
limited to `-Xmx128m`. The file is about 127MB, so the run catches a document
kept in memory. One unlimited group is still held until that group ends. A
split file counts complete items: rows when there is no group key, and groups
when there is one. A group is not cut across files.
**#4644.** A JSON-typed source field was already read through the JSON field
iterator. The manual and a test now record that path.
**#4907.** Six-digit timestamp fractions already convert when the mask
matches the value. A required offset that is missing from the text fails with a
conversion error. The timestamp format examples in the manual match that
behavior. The reported dialog freeze was not reproduced here.
**#8593.** The locale fix for negative numbers is already on main
(`18d16444b3`, PR #8639). No JSON-specific change was added.
**#2572.** The Italian compatibility tooltip no longer mentions a Hop 4.3.0
mode. It describes forcing a JSON array for a single value, which is what the
option does.
**#4395.** Object keys are not array indexes. The manual describes quoted
key names and `keys()`, and does not add a positional object syntax.
**#4373.** A JSON null stays null when empty strings differ from null. The
same correction is also in PR #8691.
**#2570.** A missing or renamed group key is named during metadata
discovery, pipeline check, and the first row. A failed lookup does not clear
the incoming row metadata.
**#8444.** `resolveJsonPaths` stays on, so existing `${PARAM_PATH}`
extraction paths still resolve. Turning it off on the JSONPath tab leaves
`$[0]` literal. File and URL names still resolve. The choice is saved with the
pipeline.
**#2929.** The file writer from PR #8713 is included here, with two
corrections: the output row is the same width as its metadata, and disposal
closes an open JSON generator before the file writer. PR #8713 should not be
merged as well. This branch is the copy that was integration-tested.
**#2569.** The attached pipeline is a grouped, pretty-printed file with a
block name, a key alias, and mixed one-row and many-row groups. A unit test
checks the parsed document. Workflow 0019 already covers grouping, a block
name, and a key alias.
**#5167 and #5313.** Group Key has a Get Fields button. It adds upstream
fields that are not already on the Fields tab or the Group Key tab, and it
keeps existing element names. Cancel does not save the new rows. The element
name is the property name in the file. It does not rename the field passed to
the next transform. A key that should also appear inside the generated fragment
is selected on the Fields tab as well.
**#5953.** Field retention when an upstream lookup fails stays in draft PR
#8588. That change is not copied here.
**#4344.** Newline-delimited JSON is off unless the File format checkbox is
selected. File output then writes one compact JSON value and a line feed for
each input row, or for each completed group. A block name wraps that line.
There is no outer document array. Newlines inside string values are escaped.
Pretty Print and any encoding other than UTF-8 are refused before a file is
opened. Appending to a nonempty file that does not already end in LF fails
before any bytes are added. Ordinary Append still concatenates complete JSON
documents. The checkbox is saved from the dialog, Cancel leaves it unchanged,
and it is disabled for output-value-only mode.
**#7995.** json-simple is unchanged. Removing it needs a decision about
public `JSONObject` signatures.
`.gitignore` also ignores local `graphify-out/` folders.
addresses #8711
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [ ] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
