The deprecated method is not going to be removed anytime soon so I wouldn't
worry about it being removed.
If you really want to use non-deprecated methods, then the
TableRowJsonCoder uses the StringUtf8Coder to parse strings so it is
looking for a nested encoding using the StringUtf8Coder encoding. So
something like this:
ByteArrayOutputStream baos = new ...
StringUtf8Coder.of().encode(jsonString, baos);
TableRow row = TableRowJsonCoder.of().decode(new
ByteArrayInputStream(baos.toByteArray()));
But why use a coder at all? TableRowJsonCoder is a thin wrapper around
using Jackson's ObjectMapper to perform the conversion. So you could do
something like:
ObjectMapper mapper = new
ObjectMapper().disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
TableRow row = mapper.readValue(strValue, TableRow.class);
On Wed, Jul 8, 2020 at 7:57 AM Lars Almgren Schwartz <lars.almg...@tink.com>
wrote:
> Hey,
>
> Don't know if it's the official way but we have written our own proto to
> BigQuery converter which works pretty well.
>
> public static TableRow convertEventToTableRow(TableRow tableRow, Message
> event) {
> Map<Descriptors.FieldDescriptor, Object> fields = event.getAllFields();
> for (Descriptors.FieldDescriptor field : fields.keySet()) {
> tableRow = mapToBigQueryField(tableRow, field, fields.get(field));
> }
>
> return tableRow;
> }
>
> private static TableRow mapToBigQueryField(
> TableRow tableRow, Descriptors.FieldDescriptor field, Object value) {
> Descriptors.FieldDescriptor.JavaType fieldType = field.getJavaType();
> switch (fieldType) {
> case INT:
> case LONG:
> case FLOAT:
> case DOUBLE:
> case BOOLEAN:
> return tableRow.set(field.getName(), value);
> case BYTE_STRING:
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value,
> x ->
> Base64.getEncoder()
> .encodeToString(
> ((ByteString)
> x).toByteArray())));
> } else {
> return tableRow.set(
> field.getName(),
> Base64.getEncoder().encodeToString(((ByteString)
> value).toByteArray()));
> }
> case ENUM:
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(), processRepeatedField(value, x ->
> x.toString()));
> } else {
> return tableRow.set(field.getName(), value.toString());
> }
> case STRING:
> if (isUUIDField(field.getName())) {
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value, x ->
> UUIDUtil.getBase64FromUUID((String) x)));
> } else {
> return tableRow.set(
> field.getName(),
> UUIDUtil.getBase64FromUUID((String) value));
> }
> } else {
> return tableRow.set(field.getName(), value);
> }
> case MESSAGE:
> switch (field.getMessageType().getFullName()) {
> // Map well known message types that we have a specific
> mapping for.
> case "google.protobuf.Timestamp":
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value,
> x ->
>
> com.google.cloud.Timestamp.fromProto(
> (Timestamp) x)
> .toString()));
> } else {
> return tableRow.set(
> field.getName(),
>
> com.google.cloud.Timestamp.fromProto((Timestamp) value)
> .toString());
> }
> case "xxx.xxx.ExactNumber":
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value, x ->
> NumberUtils.toString((ExactNumber) x)));
> } else {
> return tableRow.set(
> field.getName(),
> NumberUtils.toString((ExactNumber) value));
> }
> case "google.protobuf.UInt64Value":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.UInt64Value.getDefaultInstance())) {
> value = 0;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((UInt64Value)
> value).getValue());
> }
> case "google.protobuf.Int32Value":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.Int32Value.getDefaultInstance())) {
> value = 0;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((Int32Value)
> value).getValue());
> }
> case "google.protobuf.DoubleValue":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.DoubleValue.getDefaultInstance())) {
> value = 0;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((DoubleValue)
> value).getValue());
> }
> case "google.protobuf.FloatValue":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.FloatValue.getDefaultInstance())) {
> value = 0;
>
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((FloatValue)
> value).getValue());
> }
> case "google.protobuf.Int64Value":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.Int64Value.getDefaultInstance())) {
> value = 0;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((Int64Value)
> value).getValue());
> }
> case "google.protobuf.UInt32Value":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.UInt32Value.getDefaultInstance())) {
> value = 0;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((UInt32Value)
> value).getValue());
> }
> case "google.protobuf.BoolValue":
> if (field.hasDefaultValue()) break;
> else if
> (value.equals(com.google.protobuf.BoolValue.getDefaultInstance())) {
> value = false;
> return tableRow.set(field.getName(), value);
> } else {
> return tableRow.set(field.getName(), ((BoolValue)
> value).getValue());
> }
> case "google.protobuf.StringValue":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.StringValue.getDefaultInstance())) {
> value = "";
> return tableRow.set(field.getName(), value);
> } else if (isUUIDField(field.getName())) {
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value,
> x ->
>
> UUIDUtil.getBase64FromUUID(
> ((StringValue)
> x).getValue())));
> } else {
> return tableRow.set(
> field.getName(),
> UUIDUtil.getBase64FromUUID(
> ((StringValue)
> value).getValue()));
> }
> } else if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(value, x ->
> ((StringValue) x).getValue()));
> } else {
> return tableRow.set(field.getName(), ((StringValue)
> value).getValue());
> }
> case "google.protobuf.BytesValue":
> if (field.hasDefaultValue()) {
> break;
> } else if (value.equals(
>
> com.google.protobuf.BytesValue.getDefaultInstance())) {
> value = ByteString.EMPTY;
> return tableRow.set(field.getName(), value);
> } else if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value,
> x ->
> Base64.getEncoder()
> .encodeToString(
> ((BytesValue)
> x)
>
> .getValue()
>
> .toByteArray())));
> } else {
> return tableRow.set(
> field.getName(),
> Base64.getEncoder()
> .encodeToString(
> ((BytesValue)
> value).getValue().toByteArray()));
> }
> default:
> if (field.isRepeated()) {
> return tableRow.set(
> field.getName(),
> processRepeatedField(
> value,
> x ->
> convertEventToTableRow(
> new TableRow(),
> (Message) x)));
> } else {
> return tableRow.set(
> field.getName(),
> convertEventToTableRow(new TableRow(),
> (Message) value));
> }
> }
> default:
> throw new IllegalArgumentException(
> field.getFullName() + " has an unsupported type " +
> field.getType());
> }
> }
>
>
> On Wed, Jul 8, 2020 at 4:40 PM Kaymak, Tobias <tobias.kay...@ricardo.ch>
> wrote:
>
>> As a workaround I am currently using the following code to generate a
>> TableRow object from a Java Protobuf class - as I am facing a problem with
>> Beam schemas (
>> https://www.mail-archive.com/user@beam.apache.org/msg05799.html).
>>
>> It relies on the TableRowJsonCoder:
>>
>> String json = JsonFormat.printer().omittingInsignificantWhitespace()
>> .preservingProtoFieldNames().print(article.toBuilder());
>> InputStream inputStream = new
>> ByteArrayInputStream(json.getBytes(StandardCharsets.UTF_8));
>>
>> TableRow tableRow = TableRowJsonCoder.of().decode(inputStream,
>> Coder.Context.OUTER);
>>
>> However, the usage of Coder.Context is deprecated - I've tried to simply
>> use the decode(), but that defaults to Context.NESTED.
>>
>> What is the correct way of doing this?
>>
>> Best,
>> Tobi
>>
>
