cameronlee314 commented on a change in pull request #1426:
URL: https://github.com/apache/samza/pull/1426#discussion_r486568101
##########
File path:
samza-core/src/main/java/org/apache/samza/serializers/IntermediateMessageSerde.java
##########
@@ -67,7 +67,20 @@ public IntermediateMessageSerde(Serde userMessageSerde) {
public Object fromBytes(byte[] bytes) {
try {
final Object object;
- final MessageType type = MessageType.values()[bytes[0]];
+ final MessageType type;
+ try {
+ type = MessageType.values()[bytes[0]];
+ } catch (ArrayIndexOutOfBoundsException e) {
+ // The message type was introduced in samza 0.13.1. For samza 0.13.0
or older versions, the first byte of
+ // MessageType doesn't exist in the bytes. Thus, upgrading from those
versions will get this exception.
+ // There are two ways to solve this issue:
+ // a) Reset checkpoint or clean all messages for the intermediate topic
Review comment:
1) Could you please add this point to the PR description as well?
2) "Reset checkpoint" means "reset to newest message in the stream", right?
Could you please make sure that is clear in the doc/description?
##########
File path:
samza-core/src/test/java/org/apache/samza/serializers/model/serializers/TestIntermediateMessageSerde.java
##########
@@ -126,4 +132,25 @@ public void testEndOfStreamMessageSerde() {
assertEquals(de.getTaskName(), taskName);
assertEquals(de.getVersion(), 1);
}
+
+ @Test(expected = SamzaException.class)
+ public void testUserMessageSerdeException() {
+ Serde<?> mockUserMessageSerde = mock(Serde.class);
+ when(mockUserMessageSerde.fromBytes(anyObject())).then(new
Answer<Object>() {
+ @Override
+ public Object answer(InvocationOnMock invocation) throws Throwable {
+ byte[] bytes = invocation.getArgumentAt(0, byte[].class);
+ if (Arrays.equals(bytes, new byte[]{1, 2})) {
+ throw new SamzaException("User message serde failed to deserialize
this message.");
Review comment:
Since you have control over it, consider using something other than
`SamzaException` for this mock and validation, since `SamzaException` is
something that gets thrown a lot by core Samza classes.
It's just a nice small thing you can do to help make tests a little more
robust, in case someone introduces a bug in the future.
##########
File path:
samza-core/src/main/java/org/apache/samza/serializers/IntermediateMessageSerde.java
##########
@@ -67,7 +67,20 @@ public IntermediateMessageSerde(Serde userMessageSerde) {
public Object fromBytes(byte[] bytes) {
try {
final Object object;
- final MessageType type = MessageType.values()[bytes[0]];
+ final MessageType type;
+ try {
+ type = MessageType.values()[bytes[0]];
+ } catch (ArrayIndexOutOfBoundsException e) {
Review comment:
Is it possible that this serde gets the old message format but
`bytes[0]` is still small enough that there is no
`ArrayIndexOutOfBoundsException`? In that case, you wouldn't see your message
about the upgrade from `0.13.0`.
I think it was good to have this exception message where you had it before
in the outer `catch`, since then you could at least warn users in all cases.
You might also want to mention in the exception message that the error could
just be due to a user serde error, so that users aren't misled into thinking
that their issue is definitely the `0.13.0` upgrade issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
