Hi, We are having the same issue internally and this is not a new issue. Back in April I created and tried to fix the issue with https://jira.onap.org/browse/SO-2797 (it looks very similar but might not be related).
The workaround for us is also to restart bpmn-infra pod but this is not acceptable for production usage. I created a new Jira so we can track it here https://jira.onap.org/browse/SO-3270 You can add comment if I miss anything Thanks, Sebastien From: <onap-discuss@lists.onap.org> on behalf of "Lasse Kaihlavirta via lists.onap.org" <l.kaihlavirt=partner.samsung....@lists.onap.org> Reply-To: "onap-discuss@lists.onap.org" <onap-discuss@lists.onap.org>, "l.kaihlav...@partner.samsung.com" <l.kaihlav...@partner.samsung.com> Date: Tuesday, August 4, 2020 at 11:08 To: "onap-discuss@lists.onap.org" <onap-discuss@lists.onap.org> Subject: [EXT][onap-discuss] [Frankfurt] [SO] DelegateExecution exception in SO BPMN pod in VNF instantiation Hi, let’s get back to this with a better header (looks like I didn’t have any proper subject last time) The mysterious exception about DelegateExecutionImpl mentioned in the original mail, i.e. 2020-08-04T07:31:58.813Z|bc79ec63-71f4-49d9-b476-21dd0bfc43b9|camundaTaskExecutor-3|WorkflowActionBB||||ERROR|500||Runtime error : Error while evaluating expression: ${AbstractCDSProcessingBBUtils.constructExecutionServiceInputObject(InjectExecution.execute(execution, execution.getVariable("gBuildingBlockExecution")))}. Cause: Cannot coerce from class org.onap.so.bpmn.common.DelegateExecutionImpl to interface org.camunda.bpm.engine.delegate.DelegateExecution started resurfacing randomly last week in our Frankfurt deployments when trying to instantiate a VNF via SO after being absent for a few weeks. We can add a few observations now: * most of our deployments do not have this problem, but when they do, it happens consistently in VNF instantiation on that specific deployment (there shouldn’t be anything different between the working and failing envs; one day the same environment works, next night it’s redeployed and starts failing, next night it’s redeployed again with the same package and works again) * SO pbmn pod restart fixes the problem * there doesn’t seem to be any hints whatsoever in the SO pbmn pod logs regarding the source of the failure; while this is clearly caused some sort of timing issue in the pod startup, the actual startup logs of a working and failing pod seem to be exactly identical (!?) We therefore have a simple workaround to deal with this whenever we encounter the issue now, but we still welcome any suggestions for further troubleshooting in the hope of being able to avoid this altogether, br, Lasse From: onap-discuss@lists.onap.org <onap-discuss@lists.onap.org> On Behalf Of Lasse Kaihlavirta via lists.onap.org Sent: tiistai 23. kesäkuuta 2020 12.01 To: onap-discuss@lists.onap.org Subject: Re: [onap-discuss] #so #frankfurt Hi, …and last night’s deployment was even more broken; SO bpmn was full of database errors, and the gist of them was Caused by: java.sql.SQLException: CREATE command denied to user 'so_user'@'10-42-3-86.so-bpmn-infra.onap.svc.cluster.local' for table 'ACT_GE_PROPERTY' …and while the pods were up and running (despite having restarted 24 times by then), no requests were accepted. In comparison, we executed the exact same instantiation flow using images from last Tuesday and it was fully successful. This seems to indicate that we are having some issue with SO initialization job in the latest packages that result in different (hard-to-debug) failures in different deployments. Is it really the case that no-one else has experienced similar problems with the latest Frankfurt packages this week? br, Lasse From: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> <onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>> On Behalf Of Lasse Kaihlavirta via lists.onap.org Sent: maanantai 22. kesäkuuta 2020 17.34 To: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: [onap-discuss] #so #frankfurt Hi, I ran into the following exception in SO bpmn on our latest Frankfurt deployment from this morning when trying to instantiate our internal VNF using CSAR that worked fine last Thursday: 2020-06-22T09:40:51.442Z|482023e8-a1fe-4825-8ba9-78b12aad6e4c|camundaTaskExecutor-3|WorkflowActionBB||||ERROR|500||Runtime error : Error while evaluating expression: ${AbstractCDSProcessingBBUtils.constructExecutionServiceInputObject(InjectExecution.execute(execution, execution.getVariable("gBuildingBlockExecution")))}. Cause: Cannot coerce from class org.onap.so.bpmn.common.DelegateExecutionImpl to interface org.camunda.bpm.engine.delegate.DelegateExecution We've never seen anything like this before, and it's hard to see what can have caused this even after initial investigation. Any suggestions? br, Lasse [http://ext.w1.samsung.net/mail/ext/v1/external/status/update?userid=l.kaihlavirt&do=bWFpbElEPTIwMjAwNjIzMDkwMDQyZXVjYXMxcDJmZmQ0YmQzMmJlYWI3MDU2NGI3N2ExODgyYjdlYzczMyZyZWNpcGllbnRBZGRyZXNzPW9uYXAtZGlzY3Vzc0BsaXN0cy5vbmFwLm9yZw__] [cid:cafe_image_0@s-core.co.kr] ________________________________ External Email: Please use caution when opening links and attachments / Courriel externe: Soyez prudent avec les liens et documents joints [http://ext.w1.samsung.net/mail/ext/v1/external/status/update?userid=l.kaihlavirt&do=bWFpbElEPTIwMjAwODA0MTUwNzU2ZXVjYXMxcDE5MjQ5ZWQxYTQzNWQzOTZkZDdiNWZhNmUwMGViNWQ5ZiZyZWNpcGllbnRBZGRyZXNzPW9uYXAtZGlzY3Vzc0BsaXN0cy5vbmFwLm9yZw__] -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22191): https://lists.onap.org/g/onap-discuss/message/22191 Mute This Topic: https://lists.onap.org/mt/75988615/21656 Group Owner: onap-discuss+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
