[
https://issues.apache.org/jira/browse/BEAM-10220?focusedWorklogId=446010&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446010
]
ASF GitHub Bot logged work on BEAM-10220:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jun/20 18:46
Start Date: 15/Jun/20 18:46
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on a change in pull request
#11955:
URL: https://github.com/apache/beam/pull/11955#discussion_r440372485
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowJson.java
##########
@@ -185,18 +186,25 @@ private RowJsonDeserializer(Schema schema) {
this.schema = schema;
}
+ public RowJsonDeserializer allowMissingFields(Boolean allowMissing){
+ this.allowMissingFields = allowMissing;
+ return this;
+ }
+
@Override
public Row deserialize(JsonParser jsonParser, DeserializationContext
deserializationContext)
throws IOException {
// Parse and convert the root object to Row as if it's a nested field
with name 'root'
return (Row)
extractJsonNodeValue(
- FieldValue.of("root", FieldType.row(schema),
jsonParser.readValueAsTree()));
+ FieldValue.of("root", FieldType.row(schema),
jsonParser.readValueAsTree(), allowMissingFields));
}
+
+
private static Object extractJsonNodeValue(FieldValue fieldValue) {
- if (!fieldValue.isJsonValuePresent()) {
Review comment:
Ah yeah I didn't notice it was static. I think making it non-static is
the right call, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 446010)
Time Spent: 3h 50m (was: 3h 40m)
> Add support for implicit nulls in de/serializing JSON
> -----------------------------------------------------
>
> Key: BEAM-10220
> URL: https://issues.apache.org/jira/browse/BEAM-10220
> Project: Beam
> Issue Type: New Feature
> Components: extensions-java-json
> Reporter: Reuben van Ammers
> Assignee: Reuben van Ammers
> Priority: P3
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Currently, RowJson.java doesn't have support for deserialising JSON with
> fields missing - it fails, even if the beam schema is nullable. This is often
> a problem for reading JSON as nulls are very commonly represented by omitting
> the field. Additionally, for this reason, not writing nulls is a reasonable
> when converting from Beam rows to JSON.
> This would help to allow https://issues.apache.org/jira/browse/BEAM-7624 to
> be implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
