[
https://issues.apache.org/jira/browse/BEAM-8364?focusedWorklogId=448641&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-448641
]
ASF GitHub Bot logged work on BEAM-8364:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jun/20 18:43
Start Date: 19/Jun/20 18:43
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on a change in pull request
#11981:
URL: https://github.com/apache/beam/pull/11981#discussion_r442993899
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaCoder.java
##########
@@ -140,7 +140,14 @@ private void verifyDeterministic(Schema schema)
@Override
public boolean consistentWithEquals() {
- return true;
+ // We can't guarantee that the user type T has a good equals method, so we
assume the coder is
+ // not consistent with equals (BEAM-8364).
+ return false;
+ }
Review comment:
My only concern with that is if a user's type doesn't have equals
defined they'll just get undefined behavior, and not a useful error message.
Could we reasonably detect when a user type doesn't meet that requirement and
throw an error (or log a warning and have this function return false)?
I think it's technically possible, my first draft of this PR looked at the
typeDescriptor and only returned true when the class had equals defined, but I
was concerned it was brittle since we don't know the user-defined equals is
actually a good one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 448641)
Time Spent: 0.5h (was: 20m)
> SchemaCoder inconsistent equality behavior for POJO
> ---------------------------------------------------
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
> Affects Versions: 2.16.0
> Reporter: Neville Li
> Assignee: Brian Hulette
> Priority: P3
> Labels: beam-fixit
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder<Pojo> coder = SchemaCoder.<Pojo>of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
