[jira] [Closed] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr closed FLINK-27813. - Resolution: Won't Fix > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat > > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat > > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat > > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat > > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat > org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat > java.base/java.lang.Thread.run(Unknown Source)\nCaused by: > org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: > Time interval unit label 'm' does not match any of the recognized units: > DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | > minutes), SECONDS: (s | sec | secs | second | seconds),
[jira] [Commented] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543624#comment-17543624 ] Oleksandr commented on FLINK-27813: --- [~galenwarren] I found the issue in the configuration in module.yaml we used such a config in 3.1.0: {code:java} kind: io.statefun.endpoints.v2/http spec: functions: ua.test.execution/* urlPathTemplate: http://test-{{ $.Values.global.platformEnvironment }}.svc.cluster.local:8080/v1/functions transport: type: io.statefun.transports.v1/async timeouts: call: 6m{code} now in 3.2.0 - *6m* should be *6min* > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat > > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat > > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat > > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat > > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat
[jira] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813 ] Oleksandr deleted comment on FLINK-27813: --- was (Author: JIRAUSER290114): We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint: https://minio.test s3.path.style.access: true s3.access-key: *** s3.secret-key: *** state.backend.incremental: true jobmanager.memory.process.size: 1g taskmanager.memory.process.size: 4g }{code} > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > >
[jira] [Comment Edited] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543616#comment-17543616 ] Oleksandr edited comment on FLINK-27813 at 5/29/22 11:13 AM: - We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint: https://minio.test s3.path.style.access: true s3.access-key: *** s3.secret-key: *** state.backend.incremental: true jobmanager.memory.process.size: 1g taskmanager.memory.process.size: 4g }{code} was (Author: JIRAUSER290114): [~galenwarren] We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint:
[jira] [Comment Edited] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543616#comment-17543616 ] Oleksandr edited comment on FLINK-27813 at 5/29/22 11:06 AM: - [~galenwarren] We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint: https://minio.test s3.path.style.access: true s3.access-key: *** s3.secret-key: *** state.backend.incremental: true jobmanager.memory.process.size: 1g taskmanager.memory.process.size: 4g }{code} was (Author: JIRAUSER290114): We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint:
[jira] [Comment Edited] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543616#comment-17543616 ] Oleksandr edited comment on FLINK-27813 at 5/29/22 11:06 AM: - We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} flink-conf.yaml {code:java} { blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 taskmanager.numberOfTaskSlots: 2 execution.checkpointing.interval: 1min execution.checkpointing.mode: EXACTLY_ONCE execution.checkpointing.max-concurrent-checkpoints: 1 execution.checkpointing.min-pause: 6 execution.checkpointing.timeout: 120 execution.checkpointing.externalized-checkpoint-retention:RETAIN_ON_CANCELLATION state.backend: rocksdb state.backend.async: true state.checkpoints.num-retained: 3 state.savepoints.dir: s3://savepoints-dev/savepoints state.checkpoints.dir: s3://checkpoints-dev/checkpoints s3.endpoint: https://minio.test s3.path.style.access: true s3.access-key: *** s3.secret-key: *** state.backend.incremental: true jobmanager.memory.process.size: 1g taskmanager.memory.process.size: 4g }{code} was (Author: JIRAUSER290114): We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} >
[jira] [Commented] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543616#comment-17543616 ] Oleksandr commented on FLINK-27813: --- We have job manager running that is configured via module.yaml(egress and ingress for kafka topics). Our stateful functions are on a separate remote service built on top of Spring boot with controller: {code:java} @RequestMapping("/v1/functions") @RequiredArgsConstructor public class FunctionRouteController { private final RequestReplyHandler handler; @PostMapping(consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE) public CompletableFuture onRequest(@RequestBody byte[] request) { return handler .handle(Slices.wrap(request)) .thenApply(Slice::toByteArray); } } {code} where actually used 'org.apache.flink.statefun.sdk.java.handlerConcurrentRequestReplyHandler' and we just define beans: {code:java} @Bean public StatefulFunctions statefulFunctions(StatefulFunctions autoRegisteredStatefulFunctions) { // obtain a request-reply handler based on the spec above StatefulFunctions statefulFunctions = new StatefulFunctions() .withStatefulFunction(fallbackFn()) .withStatefulFunction(transactionFn()); autoRegisteredStatefulFunctions.functionSpecs().values().forEach(statefulFunctions::withStatefulFunction); return statefulFunctions; } @Bean public RequestReplyHandler requestReplyHandler(StatefulFunctions statefulFunctions) { return statefulFunctions.requestReplyHandler(); } {code} statefulFunctions.requestReplyHandler() is : {code:java} public RequestReplyHandler requestReplyHandler() { return new ConcurrentRequestReplyHandler(this.specs); } {code} > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > >
[jira] [Commented] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543094#comment-17543094 ] Oleksandr commented on FLINK-27813: --- [~galenwarren] we have such specs for example: {code:java} public static final ValueSpec ORIGINAL_CALLER = ValueSpec.named("originalCaller") .thatExpireAfterWrite(ChronoUnit.MONTHS.getDuration()) .withUtf8StringType(); public static final ValueSpec CALLER_ID = ValueSpec.named("callerId") .thatExpireAfterWrite(ChronoUnit.MONTHS.getDuration()) .withUtf8StringType(); public static final ValueSpec RESPONSE_STORAGE = ValueSpec.named("SightResponse") .thatExpireAfterWrite(ChronoUnit.MONTHS.getDuration()) .withCustomType(SightValidationResponse.TYPE); public static final ValueSpec ACT_RESPONSE_STORAGE = ValueSpec.named("ActResponse") .thatExpireAfterWrite(ChronoUnit.MONTHS.getDuration()) .withCustomType(ProtobufTypes.sepAckType()); public static final ValueSpec REQUEST_ATTEMPT = ValueSpec.named("attempt") .thatExpireAfterWrite(ChronoUnit.HOURS.getDuration()) .withIntType(); public static final ValueSpec RETRYABLE_RUNNER_VALUE_SPEC = ValueSpec .named("retryableRunner") .withCustomType(RetryableRunner.TYPE); public static final ValueSpec VALIDATION_IN_PROGRESS = ValueSpec.named("MonitoringValidationInProgress") .thatExpireAfterWrite(ChronoUnit.WEEKS.getDuration()) .withBooleanType();{code} > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration from > flink-statefun:3.1.1-java11 > to > flink-statefun:3.2.0-java11 > > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat > >
[jira] [Updated] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr updated FLINK-27813: -- Description: Issue was met after migration from flink-statefun:3.1.1-java11 to flink-statefun:3.2.0-java11 {code:java} ts.execution-paymentManualValidationRequestEgress-egress, Sink: ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with failure cause: java.lang.IllegalStateException: Unable to parse Netty transport spec.\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat java.base/java.lang.Thread.run(Unknown Source)\nCaused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: Time interval unit label 'm' does not match any of the recognized units: DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | minutes), SECONDS: (s | sec | secs | second | seconds), MILLISECONDS: (ms | milli | millis | millisecond | milliseconds), MICROSECONDS: (µs | micro | micros | microsecond | microseconds), NANOSECONDS: (ns | nano | nanos | nanosecond | nanoseconds) (through reference chain: org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec[\"timeouts\"]->org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec$Timeouts[\"call\"])\n\tat org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:390)\n\tat org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:349)\n\tat
[jira] [Commented] (FLINK-25197) Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackson2
[ https://issues.apache.org/jira/browse/FLINK-25197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17542896#comment-17542896 ] Oleksandr commented on FLINK-25197: --- Hi, [~galenwarren] could your fix affect this issue - https://issues.apache.org/jira/browse/FLINK-27813 ? > Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type > `java.time.Duration` not supported by default: add Module > "org.apache.flink.shaded.jackson2.com.fasterxml.jackson.datatype:jackson-datatype-jsr310" > to enable handling > --- > > Key: FLINK-25197 > URL: https://issues.apache.org/jira/browse/FLINK-25197 > Project: Flink > Issue Type: Bug > Components: Stateful Functions >Affects Versions: statefun-3.1.0 >Reporter: Galen Warren >Assignee: Galen Warren >Priority: Major > Labels: pull-request-available > Fix For: statefun-3.2.0, statefun-3.1.2 > > > When using RequestReplyFunctionBuilder to build a stateful functions job, the > job fails at runtime with: > Java 8 date/time type `java.time.Duration` not supported by default: add > Module > "org.apache.flink.shaded.jackson2.com.fasterxml.jackson.datatype:jackson-datatype-jsr310" > to enable handling > It appears this is because, in > [RequestReplyFunctionBuilder::transportClientPropertiesAsObjectNode|https://github.com/apache/flink-statefun/blob/b4ba9547b8f0105a28544fd28a5e0433666e9023/statefun-flink/statefun-flink-datastream/src/main/java/org/apache/flink/statefun/flink/datastream/RequestReplyFunctionBuilder.java#L127], > a default instance of ObjectMapper is used to serialize the client > properties, which now include a java.time.Duration. There is a > [StateFunObjectMapper|https://github.com/apache/flink-statefun/blob/master/statefun-flink/statefun-flink-common/src/main/java/org/apache/flink/statefun/flink/common/json/StateFunObjectMapper.java] > class in the project that has customized serde support, but it is not used > here. > The fix seems to be to: > * Use an instance of StateFunObjectMapper to serialize the client properties > in RequestReplyFunctionBuilder > * Modify StateFunObjectMapper to both serialize and deserialize instances of > java.time.Duration (currently, only deserialization is supported) > I've made these changes locally and it seems to fix the problem. Would you be > interested in a PR? Thanks. > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr updated FLINK-27813: -- Description: Issue was met after migration from flink-statefun:3.1.1-java11 to flink-statefun:3.2.0-java8 {code:java} ts.execution-paymentManualValidationRequestEgress-egress, Sink: ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with failure cause: java.lang.IllegalStateException: Unable to parse Netty transport spec.\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat java.base/java.lang.Thread.run(Unknown Source)\nCaused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: Time interval unit label 'm' does not match any of the recognized units: DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | minutes), SECONDS: (s | sec | secs | second | seconds), MILLISECONDS: (ms | milli | millis | millisecond | milliseconds), MICROSECONDS: (µs | micro | micros | microsecond | microseconds), NANOSECONDS: (ns | nano | nanos | nanosecond | nanoseconds) (through reference chain: org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec[\"timeouts\"]->org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec$Timeouts[\"call\"])\n\tat org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:390)\n\tat org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:349)\n\tat
[jira] [Updated] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr updated FLINK-27813: -- Attachment: screenshot-1.png > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > Attachments: screenshot-1.png > > > Issue was met after migration to 3.2.0 > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat > > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat > > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat > > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat > > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat > org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat > java.base/java.lang.Thread.run(Unknown Source)\nCaused by: > org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: > Time interval unit label 'm' does not match any of the recognized units: > DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | > minutes), SECONDS: (s | sec | secs | second | seconds), MILLISECONDS: (ms | > milli | millis | millisecond |
[jira] [Updated] (FLINK-27813) java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0
[ https://issues.apache.org/jira/browse/FLINK-27813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr updated FLINK-27813: -- Summary: java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 (was: java.lang.IllegalStateException: afte migration from statefun 3.1.1 to 3.2.0) > java.lang.IllegalStateException: after migration from statefun-3.1.1 to 3.2.0 > - > > Key: FLINK-27813 > URL: https://issues.apache.org/jira/browse/FLINK-27813 > Project: Flink > Issue Type: Bug > Components: API / State Processor >Affects Versions: statefun-3.2.0 >Reporter: Oleksandr >Priority: Blocker > > Issue was met after migration to 3.2.0 > {code:java} > ts.execution-paymentManualValidationRequestEgress-egress, Sink: > ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: > ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 > (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with > failure cause: java.lang.IllegalStateException: Unable to parse Netty > transport spec.\n\tat > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat > > org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat > > org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat > > org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat > > org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat > > org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat > > org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat > > org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat > > org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat > > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat > > org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat > > org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat > > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat > > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat > > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat > > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat > > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat > org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat > java.base/java.lang.Thread.run(Unknown Source)\nCaused by: > org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: > Time interval unit label 'm' does not match any of the recognized units: > DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | > minutes), SECONDS:
[jira] [Created] (FLINK-27813) java.lang.IllegalStateException: afte migration from statefun 3.1.1 to 3.2.0
Oleksandr created FLINK-27813: - Summary: java.lang.IllegalStateException: afte migration from statefun 3.1.1 to 3.2.0 Key: FLINK-27813 URL: https://issues.apache.org/jira/browse/FLINK-27813 Project: Flink Issue Type: Bug Components: API / State Processor Affects Versions: statefun-3.2.0 Reporter: Oleksandr Issue was met after migration to 3.2.0 {code:java} ts.execution-paymentManualValidationRequestEgress-egress, Sink: ua.aval.payments.execution-norkomExecutionRequested-egress, Sink: ua.aval.payments.execution-shareStatusToManualValidation-egress) (1/1)#15863 (98a1f6bef9a435ef9d0292fefeb64022) switched from RUNNING to FAILED with failure cause: java.lang.IllegalStateException: Unable to parse Netty transport spec.\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.parseTransportSpec(NettyRequestReplyClientFactory.java:56)\n\tat org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplyClientFactory.createTransportClient(NettyRequestReplyClientFactory.java:39)\n\tat org.apache.flink.statefun.flink.core.httpfn.HttpFunctionProvider.functionOfType(HttpFunctionProvider.java:49)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:70)\n\tat org.apache.flink.statefun.flink.core.functions.PredefinedFunctionLoader.load(PredefinedFunctionLoader.java:47)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.load(StatefulFunctionRepository.java:71)\n\tat org.apache.flink.statefun.flink.core.functions.StatefulFunctionRepository.get(StatefulFunctionRepository.java:59)\n\tat org.apache.flink.statefun.flink.core.functions.LocalFunctionGroup.enqueue(LocalFunctionGroup.java:48)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.enqueue(Reductions.java:153)\n\tat org.apache.flink.statefun.flink.core.functions.Reductions.apply(Reductions.java:148)\n\tat org.apache.flink.statefun.flink.core.functions.FunctionGroupOperator.processElement(FunctionGroupOperator.java:90)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)\n\tat org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)\n\tat org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.sendDownstream(FeedbackUnionOperator.java:180)\n\tat org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)\n\tat org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)\n\tat org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)\n\tat org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)\n\tat org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)\n\tat org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)\n\tat org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)\n\tat org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)\n\tat org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)\n\tat org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)\n\tat java.base/java.lang.Thread.run(Unknown Source)\nCaused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: Time interval unit label 'm' does not match any of the recognized units: DAYS: (d | day | days), HOURS: (h | hour | hours), MINUTES: (min | minute | minutes), SECONDS: (s | sec | secs | second | seconds), MILLISECONDS: (ms | milli | millis | millisecond | milliseconds), MICROSECONDS: (µs | micro | micros | microsecond | microseconds), NANOSECONDS: (ns | nano | nanos | nanosecond | nanoseconds) (through reference chain: org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec[\"timeouts\"]->org.apache.flink.statefun.flink.core.nettyclient.NettyRequestReplySpec$Timeouts[\"call\"])\n\tat