[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056917#comment-16056917
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


[~shashank734] I think the next release candidate for 1.3.1 will be pushed out 
today. The vote for it will happen on the mailing list, so hopefully you should 
be able to expect it to be released before next week.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056326#comment-16056326
 ] 

Shashank Agarwal commented on FLINK-6844:
-

So what is the release date for version 1.3.1 cause i don't wanna build this 
with master branch and than have to publish libraries like CEP etc. local. 
Everything will change. is there any quick fix which i can apply on 
release-1.3.0 with this commit.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056044#comment-16056044
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


[~shashank734]

I've tested CEP + with Scala collections as the event type (which would then 
let the CEP operator use the {{TraversableSerializer}} internally), with the 
following code:
{code}
object FlinkCEPTest {
  def main(args: Array[String]) {
// set up the streaming execution environment
val env = StreamExecutionEnvironment.getExecutionEnvironment
env.enableCheckpointing(5000)

val source: DataStream[scala.collection.immutable.List[java.lang.String]] = 
env.socketTextStream("localhost", ).map(x => List(x.split(",")(0)))
val pattern = Pattern.begin("start").where(new 
SimpleCondition[List[String]] {
  override def filter(t: List[String]) = t.head.equals("a")
}).times(4).allowCombinations().followedBy("end").where(new 
SimpleCondition[List[String]] {
  override def filter(t: List[String]) = t.head.equals("b")
})

CEP.pattern(source, pattern).select(_.toString()).print()

// execute program
env.execute("Flink CEP test")
  }
}
{code}

I can confirm that this works correctly without any errors in branch 
{{release-1.3}}. Checkpoint + restoring from savepoints works correctly.
As I've mentioned, simply applying the commit for this JIRA onto 
{{release-1.3.0}} may not work, as the whole fix includes other commits as well.

Please let me know if you think otherwise!

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055861#comment-16055861
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


Didn't you mention this error:
{code}
2017-06-20 15:26:25,518 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint 
triggering task Source: Custom File Source (1/1) is not being executed at the 
moment. Aborting checkpoint.
{code}
?

This doesn't seem to be the Kafka source. The checkpoint is failing because the 
source isn't running.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055854#comment-16055854
 ] 

Shashank Agarwal commented on FLINK-6844:
-

[~tzulitai]

I am using kafka as source and there's no issue with that. Actually what I am 
using flink cep this code was working fine with 1.2.0 and 1.2.1 i have updated 
my applications to 1.3.0 

Applications are working where i haven't used CEP, In application i have used 
cep was giving following exception and terminating the checkpointing for all. 

{code}
java.lang.Exception: Could not perform checkpoint 1 for operator 
KeyedCEPPatternOperator -> Map (6/6).
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:550)
at 
org.apache.flink.streaming.runtime.io.BarrierBuffer.notifyCheckpoint(BarrierBuffer.java:378)
at 
org.apache.flink.streaming.runtime.io.BarrierBuffer.processBarrier(BarrierBuffer.java:281)
at 
org.apache.flink.streaming.runtime.io.BarrierBuffer.getNextNonBlocked(BarrierBuffer.java:183)
at 
org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:213)
at 
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:69)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:262)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Could not complete snapshot 1 for operator 
KeyedCEPPatternOperator -> Map (6/6).
at 
org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:406)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.checkpointStreamOperator(StreamTask.java:1157)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.executeCheckpointing(StreamTask.java:1089)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.checkpointState(StreamTask.java:653)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:589)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:542)
... 8 more
Caused by: java.lang.UnsupportedOperationException
at 
org.apache.flink.api.scala.typeutils.TraversableSerializer.snapshotConfiguration(TraversableSerializer.scala:155)
at 
org.apache.flink.api.common.typeutils.CompositeTypeSerializerConfigSnapshot.(CompositeTypeSerializerConfigSnapshot.java:53)
at 
org.apache.flink.api.scala.typeutils.OptionSerializer$OptionSerializerConfigSnapshot.(OptionSerializer.scala:139)
at 
org.apache.flink.api.scala.typeutils.OptionSerializer.snapshotConfiguration(OptionSerializer.scala:104)
at 
org.apache.flink.api.scala.typeutils.OptionSerializer.snapshotConfiguration(OptionSerializer.scala:28)
at 
org.apache.flink.api.common.typeutils.CompositeTypeSerializerConfigSnapshot.(CompositeTypeSerializerConfigSnapshot.java:53)
at 
org.apache.flink.api.java.typeutils.runtime.TupleSerializerConfigSnapshot.(TupleSerializerConfigSnapshot.java:45)
at 
org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.snapshotConfiguration(TupleSerializerBase.java:132)
at 
org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.snapshotConfiguration(TupleSerializerBase.java:39)
at 
org.apache.flink.api.common.typeutils.CompositeTypeSerializerConfigSnapshot.(CompositeTypeSerializerConfigSnapshot.java:53)
at 
org.apache.flink.api.common.typeutils.base.CollectionSerializerConfigSnapshot.(CollectionSerializerConfigSnapshot.java:39)
at 
org.apache.flink.api.common.typeutils.base.ListSerializer.snapshotConfiguration(ListSerializer.java:183)
at 
org.apache.flink.api.common.typeutils.base.ListSerializer.snapshotConfiguration(ListSerializer.java:47)
at 
org.apache.flink.api.common.typeutils.CompositeTypeSerializerConfigSnapshot.(CompositeTypeSerializerConfigSnapshot.java:53)
at 
org.apache.flink.api.common.typeutils.base.MapSerializerConfigSnapshot.(MapSerializerConfigSnapshot.java:38)
at 
org.apache.flink.runtime.state.HashMapSerializer.snapshotConfiguration(HashMapSerializer.java:210)
at 
org.apache.flink.runtime.state.RegisteredKeyedBackendStateMetaInfo.snapshot(RegisteredKeyedBackendStateMetaInfo.java:71)
at 
org.apache.flink.runtime.state.heap.HeapKeyedStateBackend.snapshot(HeapKeyedStateBackend.java:267)
at 
org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:396)
... 13 more
{code}

Than i have applied your patch on release-1.3.0 tag and used that with this 
code still without CEP app is working fine, but CEP app is 

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055836#comment-16055836
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


Yes, so what I mean is, the checkpointing problem you pasted does not infer 
problems with this JIRA.
What source are you using? Could you find out why it isn't being executed while 
the job was running? That's the actual root cause of the checkpoint failure.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055827#comment-16055827
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


The thing is, I can't tell anything from:
{code}
2017-06-20 15:26:25,518 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint 
triggering task Source: Custom File Source (1/1) is not being executed at the 
moment. Aborting checkpoint.
{code}

It's also not related to the CEP operator.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055824#comment-16055824
 ] 

Shashank Agarwal commented on FLINK-6844:
-

I have cross variefied with source code and flink dashboard also i have 
successfully applied the patch. Actually it's not printing any other error 
stack traces. 

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055819#comment-16055819
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


Then I'm pretty sure that you actually haven't applied the patch, actually. The 
{{TraversableSerializer}} should not be throwing 
{{UnsupportedOperationException}} anymore, as you can see in the patch.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055818#comment-16055818
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


[~shashank734]

Can you try https://github.com/apache/flink/tree/release-1.3? The release 
process for the next stable version 1.3.1 is actually in-progress, and will be 
forked from that branch very soon.

Alternatively, you can also show me the exact error trace, so I can see if it's 
actually fixed in other commits already.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055816#comment-16055816
 ] 

Shashank Agarwal commented on FLINK-6844:
-

There is no more stack traces it's printing after applying the patch May be 
it's due to you have removed

{code}
throw new UnsupportedOperationException()
{code}

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055813#comment-16055813
 ] 

Shashank Agarwal commented on FLINK-6844:
-

[~tzulitai] 

I have applied this patch on 
https://github.com/apache/flink/tree/release-1.3.0. Latest 1.3.0 Release. I 
need stable version. 

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055809#comment-16055809
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


For example, there were some other bugs specific to the serializers in the 
{{KeyedCEPPatternOperator}}. I don't think simply applying the mentioned 
commits would resolve the problem with CEP checkpointing completely.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055807#comment-16055807
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


btw, please test this on the latest {{release-1.3}}. There were some further 
follow up fixes after the commits mentioned above in the comments.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055806#comment-16055806
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


[~shashank734] do you mean that applying the patch still doesn't work? Could 
you paste the actual checkpoint failure error trace? The one you posted doesn't 
seem to reveal anything.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Shashank Agarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055804#comment-16055804
 ] 

Shashank Agarwal commented on FLINK-6844:
-

[~tzulitai]

Checked with patch not working with KeyedCEPPatternOperator. In commit you are 
not throwing exception so only log is printing no exception. But checkpointing 
is not working it was working fine in 1.2.1

{code}
2017-06-20 15:26:25,518 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint 
triggering task Source: Custom File Source (1/1) is not being executed at the 
moment. Aborting checkpoint.
{code}

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1, 1.4.0
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041170#comment-16041170
 ] 

ASF GitHub Bot commented on FLINK-6844:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4081


> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040677#comment-16040677
 ] 

ASF GitHub Bot commented on FLINK-6844:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4081
  
Tested this with a stateful job that uses Scala collections as state.
Merging this ..


> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039071#comment-16039071
 ] 

ASF GitHub Bot commented on FLINK-6844:
---

GitHub user tzulitai opened a pull request:

https://github.com/apache/flink/pull/4081

[FLINK-6844] [scala] Implement compatibility methods for 
TraversableSerializer

The `TraversableSerializer` is used for Scala collections, and therefore 
may take part in checkpointing and should have the compatibility methods 
implemented.

I'm also currently trying out whether or not it makes sense / is easily 
possible to have some test base that guards against these kind of issues. 
Otherwise, having one per-serializer might not make sense.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tzulitai/flink FLINK-6844

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4081.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4081


commit 3fbf693610520a983f68a0091057a0af44108834
Author: Tzu-Li (Gordon) Tai 
Date:   2017-06-05T18:52:57Z

[FLINK-6844] [scala] Implement compatibility methods for 
TraversableSerializer




> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[mingleizhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038324#comment-16038324
 ] 

mingleizhang commented on FLINK-6844:
-

:P [~tzulitai] Yes. I will review that when the patch is available. 

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Tzu-Li (Gordon) Tai (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038288#comment-16038288
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-6844:


[~mingleizhang] I actually already have a fix ready for this, the PR is coming 
up.
Would be also be helpful if you'd like to review that!

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[mingleizhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037942#comment-16037942
 ] 

mingleizhang commented on FLINK-6844:
-

I will look into this issue those days.

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6844) TraversableSerializer should implement compatibility methods

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036083#comment-16036083
 ] 

Ted Yu commented on FLINK-6844:
---

This was first reported by Mahesh:

http://search-hadoop.com/m/Flink/VkLeQDkhbf1oulbz1?subj=Flink+1+3+Checkpointing+failing

> TraversableSerializer should implement compatibility methods
> 
>
> Key: FLINK-6844
> URL: https://issues.apache.org/jira/browse/FLINK-6844
> Project: Flink
>  Issue Type: Bug
>  Components: Type Serialization System
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Blocker
>  Labels: flink-rel-1.3.1-blockers
> Fix For: 1.3.1
>
>
> The {{TraversableSerializer}} may be used as a serializer for managed state 
> and takes part in checkpointing, therefore should implement the compatibility 
> methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)