[
https://issues.apache.org/jira/browse/BEAM-10220?focusedWorklogId=443523&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-443523
]
ASF GitHub Bot logged work on BEAM-10220:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Jun/20 01:49
Start Date: 10/Jun/20 01:49
Worklog Time Spent: 10m
Work Description: reubenvanammers commented on a change in pull request
#11955:
URL: https://github.com/apache/beam/pull/11955#discussion_r437814700
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowJson.java
##########
@@ -185,18 +186,25 @@ private RowJsonDeserializer(Schema schema) {
this.schema = schema;
}
+ public RowJsonDeserializer allowMissingFields(Boolean allowMissing){
+ this.allowMissingFields = allowMissing;
+ return this;
+ }
+
@Override
public Row deserialize(JsonParser jsonParser, DeserializationContext
deserializationContext)
throws IOException {
// Parse and convert the root object to Row as if it's a nested field
with name 'root'
return (Row)
extractJsonNodeValue(
- FieldValue.of("root", FieldType.row(schema),
jsonParser.readValueAsTree()));
+ FieldValue.of("root", FieldType.row(schema),
jsonParser.readValueAsTree(), allowMissingFields));
}
+
+
private static Object extractJsonNodeValue(FieldValue fieldValue) {
- if (!fieldValue.isJsonValuePresent()) {
Review comment:
I initially thought about this, then changed to putting it into the
FieldValue because extractJsonNodeValue was static. Now that you mention it, I
believe it is probably neater to just check this.allowMissing fields and change
the method to non-static.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 443523)
Time Spent: 1h 40m (was: 1.5h)
> Add support for implicit nulls in de/serializing JSON
> -----------------------------------------------------
>
> Key: BEAM-10220
> URL: https://issues.apache.org/jira/browse/BEAM-10220
> Project: Beam
> Issue Type: New Feature
> Components: extensions-java-json
> Reporter: Reuben van Ammers
> Priority: P3
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently, RowJson.java doesn't have support for deserialising JSON with
> fields missing - it fails, even if the beam schema is nullable. This is often
> a problem for reading JSON as nulls are very commonly represented by omitting
> the field. Additionally, for this reason, not writing nulls is a reasonable
> when converting from Beam rows to JSON.
> This would help to allow https://issues.apache.org/jira/browse/BEAM-7624 to
> be implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
