[ https://issues.apache.org/jira/browse/CLOUDSTACK-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13867010#comment-13867010 ]
Alena Prokharchyk commented on CLOUDSTACK-5848: ----------------------------------------------- Add database dumps please, taken before and after the upgrade. This is the minimum requirement for the db upgrade bugs. > "Unable to parse VLAN tag" message when network associated with SRX external > firewall device is rebooted. > ---------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-5848 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5848 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Network Devices > Affects Versions: 4.3.0 > Environment: upgraded the CS from 3.0.6 patch E to 4.3 > Reporter: manasaveloori > Priority: Critical > Fix For: 4.3.0 > > Attachments: management-server.rar > > > Steps: > 1. Deploy CS 3.0.6 patch E using Xen 6.0.2 HV. > 2. Add SRX device. > 3 . Deploy the VMs uing the SRX network. > 4. Enable PF,Static nat rules for the VM using SRX. > 5. Upgrade the CS to 4.3. > 6. Now restart the network associated with SRX or disable static nat etc. > Observing following exceptions in logs: > er root, class super-user --><rpc-reply > xmlns:junos="http://xml.juniper.net/junos/10.4R6/junos"><xnm:warning > xmlns="http://xml.juniper.net/xnm/1.1/xnm" > xmlns:xnm="http://xml.juniper.net/xnm/1.1/xnm"><message>uncommitted changes > will be discarded on exit</message></xnm:warning></rpc-reply> > 2014-01-09 22:27:14,815 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-22:ctx-357d339b) Opened a private configuration. > 2014-01-09 22:27:14,816 ERROR [c.c.n.r.JuniperSrxResource] > (DirectAgent-22:ctx-357d339b) com.cloud.utils.exception.ExecutionException: > Unable to parse VLAN tag: 47 > 2014-01-09 22:27:14,816 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-22:ctx-357d339b) Sending request: <!--Licensed to the Apache > Software Foundation (ASF) under oneor more contributor license agreements. > See the NOTICE filedistributed with this work for additional > informationregarding copyright ownership. The ASF licenses this fileto you > under the Apache License, Version 2.0 (the"License"); you may not use this > file except in compliancewith the License. You may obtain a copy of the > License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by > applicable law or agreed to in writing,software distributed under the License > is distributed on an"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF > ANYKIND, either express or implied. See the License for thespecific language > governing permissions and limitationsunder the > License.--><rpc><close-configuration/></rpc> > 2014-01-09 22:27:14,868 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-22:ctx-357d339b) Checking response: <rpc-reply > xmlns:junos="http://xml.juniper.net/junos/10.4R6/junos"></rpc-reply> > 2014-01-09 22:27:14,868 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-22:ctx-357d339b) Closed private configuration. > 2014-01-09 22:27:14,869 DEBUG [c.c.a.m.DirectAgentAttache] > (DirectAgent-22:ctx-357d339b) Seq 9-550895625: Response Received: > 2014-01-09 22:27:14,869 DEBUG [c.c.a.t.Request] (DirectAgent-22:ctx-357d339b) > Seq 9-550895625: Processing: { Ans: , MgmtId: 6642334695485, via: 9, Ver: > v1, Flags: 10, > [{"com.cloud.agent.api.Answer":{"result":false,"details":"Exception: > com.cloud.utils.exception.ExecutionException\nMessage: Unable to parse VLAN > tag: 47\nStack: com.cloud.utils.exception.ExecutionException: Unable to parse > VLAN tag: 47\n\tat > com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609)\n\tat > > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894)\n\tat > > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)\n\tat > > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)\n\tat > > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830)\n\tat > > com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353)\n\tat > > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)\n\tat > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat > java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat > java.lang.Thread.run(Thread.java:679)\n","wait":0}}] } > 2014-01-09 22:27:14,870 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1 > ctx-50987553) Seq 9-550895625: Received: { Ans: , MgmtId: 6642334695485, > via: 9, Ver: v1, Flags: 10, { Answer } } > 2014-01-09 22:27:14,870 DEBUG [c.c.a.m.AgentManagerImpl] > (Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) Details from executing class > com.cloud.agent.api.routing.SetFirewallRulesCommand: Exception: > com.cloud.utils.exception.ExecutionException > Message: Unable to parse VLAN tag: 47 > Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN > tag: 47 > at > com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830) > at > com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353) > at > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2014-01-09 22:27:14,870 ERROR [c.c.n.ExternalFirewallDeviceManagerImpl] > (Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) External firewall was unable to > apply static nat rules to the SRX appliance in zone zonexen due to: > Exception: com.cloud.utils.exception.ExecutionException > Message: Unable to parse VLAN tag: 47 > Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN > tag: 47 > at > com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830) > at > com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353) > at > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > . > 2014-01-09 22:27:14,871 WARN [c.c.n.f.FirewallManagerImpl] > (Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) Failed to apply firewall rules > due to > com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is > unreachable: External firewall was unable to apply static nat rules to the > SRX appliance in zone zonexen due to: Exception: > com.cloud.utils.exception.ExecutionException > Message: Unable to parse VLAN tag: 47 > Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN > tag: 47 > at > com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912) > at > com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830) > at > com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353) > at > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > . > at > com.cloud.network.ExternalFirewallDeviceManagerImpl.sendFirewallRules(ExternalFirewallDeviceManagerImpl.java:616) > at > com.cloud.network.ExternalFirewallDeviceManagerImpl.applyFirewallRules(ExternalFirewallDeviceManagerImpl.java:573) > at > com.cloud.network.element.JuniperSRXExternalFirewallElement.applyFWRules(JuniperSRXExternalFirewallElement.java:233) > at > com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:569) > at > com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:502) > at > com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:523) > at > com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:643) > at > com.cloud.network.firewall.FirewallManagerImpl.applyIngressFirewallRules(FirewallManagerImpl.java:616) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) > at $Proxy174.applyIngressFirewallRules(Unknown Source) > at > org.apache.cloudstack.api.command.user.firewall.CreateFirewallRuleCmd.execute(CreateFirewallRuleCmd.java:125) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161) > at > com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109) > at > com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:524) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2014-01-09 22:27:14,964 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1 > ctx-50987553) Seq 9-550895626: Sending { Cmd , MgmtId: 6642334695485, via: > 9(200-JuniperSRXFirewall-10.147.40.3), Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":9,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[22,22],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":20,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,655],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":24,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,65],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false}],"accessDetails":{},"wait":0}}] > } > 2014-01-09 22:27:14,966 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1 > ctx-50987553) Seq 9-550895626: Executing: { Cmd , MgmtId: 6642334695485, > via: 9(200-JuniperSRXFirewall-10.147.40.3), Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":9,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[22,22],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":20,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,655],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":24,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,65],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false}],"accessDetails":{},"wait":0}}] > } > 2014-01-09 22:27:14,967 DEBUG [c.c.a.m.DirectAgentAttache] > (DirectAgent-332:ctx-e5cfb854) Seq 9-550895626: Executing request > 2014-01-09 22:27:14,996 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-332:ctx-e5cfb854) Sending login request > 2014-01-09 22:27:15,114 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-332:ctx-e5cfb854) Checking login response > 2014-01-09 22:27:15,114 DEBUG [c.c.n.r.JuniperSrxResource] > (DirectAgent-332:ctx-e5cfb854) Sending request: <!--Licensed to the Apache > Software Foundation (ASF) under oneor more contributor license agreements. > See the NOTICE filedistributed with this work for additional > informationregarding copyright ownership. The ASF licenses this fileto you > under the Apache License, Version 2.0 (the"License"); you may not use this > file except in compliancewith the License. You may obtain a copy of the > License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by > applicable law or agreed to in writing,software distributed under the License > is distributed on an"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF > ANYKIND, either express or implied. See the License for thespecific language > governing permissions and limitationsunder the > License.--><rpc><open-configuration><private/></open-configuration></rpc> > 2014-01-09 22:27:16,810 DEBUG [c.c.a.m.AgentManagerImpl] > (AgentManager-Handler-11:null) SeqA 8-162: Processing Seq 8-162: { Cmd , > MgmtId: -1, via: 8, Ver: v1, Flags: 11, > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":4,"_loadInfo":"{\n > \"connections\": []\n}","wait":0}}] } > 2014-01-09 22:27:16,820 DEBUG [c.c.a.m.AgentManagerImpl] > (AgentManager-Handler-11:null) SeqA 8-162: Sending Seq 8-162: { Ans: , > MgmtId: 6642334695485, via: 8, Ver: v1, Flags: 100010, > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } > 2014-01-09 22:27:16,891 DEBUG [c.c.a.ApiServlet] > (catalina-exec-3:ctx-b061c17c) ===START=== 10.252.192.34 -- GET > command=queryAsyncJobResult&jobId=eac > attaching the MS logs -- This message was sent by Atlassian JIRA (v6.1.5#6160)