[jira] [Updated] (CLOUDSTACK-6475) [Automation] communication between cloudstack agent and MS disconnecting continuously
[ https://issues.apache.org/jira/browse/CLOUDSTACK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishan Kavala updated CLOUDSTACK-6475: -- Priority: Major (was: Blocker) [Automation] communication between cloudstack agent and MS disconnecting continuously --- Key: CLOUDSTACK-6475 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6475 Project: CloudStack Issue Type: Bug Security Level: Public(Anyone can view this level - this is the default.) Components: KVM Affects Versions: 4.4.0 Environment: RHEL 6.3 Reporter: Rayees Namathponnan Assignee: Kishan Kavala Fix For: 4.4.0 Attachments: Agent_log.rar, management-server.rar This issue is observed during automation, run. communication between cloudstack agent and ms getting disconnected continuously; observed below error in agent log 2014-04-22 04:08:47,867 INFO [cloud.agent.Agent] (AgentShutdownThread:null) Stopping the agent: Reason = sig.kill 2014-04-22 04:10:41,456 INFO [cloud.agent.AgentShell] (main:null) Agent started 2014-04-22 04:10:41,500 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.4.0-SNAPSHOT 2014-04-22 04:10:41,502 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /etc/cloudstack/agent/agent.properties 2014-04-22 04:10:41,551 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage 2014-04-22 04:10:41,552 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm 2014-04-22 04:10:41,572 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2014-04-22 04:10:41,722 INFO [cloud.agent.Agent] (main:null) id is 0 2014-04-22 04:10:42,501 INFO [kvm.resource.LibvirtComputingResource] (main:null) No libvirt.vif.driver specified. Defaults to BridgeVifDriver. 2014-04-22 04:10:42,590 INFO [cloud.agent.Agent] (main:null) Agent [id = 0 : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host = 10.223.49.195 : port = 8250 2014-04-22 04:10:42,664 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.223.49.195:8250 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) SSL: Handshake done 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) Connected to 10.223.49.195:8250 2014-04-22 04:10:42,941 WARN [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:null) Could not read cpuinfo_max_freq 2014-04-22 04:10:43,158 INFO [cloud.serializer.GsonHelper] (Agent-Handler-1:null) Default Builder inited. 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Set agent id 0 2014-04-22 04:10:43,233 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Startup Response Received: agent id = 0 2014-04-22 04:11:40,925 INFO [cloud.agent.Agent] (Agent-Handler-1:null) Lost connection to the server. Dealing with the remaining commands... 2014-04-22 04:11:42,352 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 0 2014-04-22 04:11:42,368 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 1 2014-04-22 04:11:42,383 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 2 2014-04-22 04:11:42,398 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 3 2014-04-22 04:11:42,413 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 4 2014-04-22 04:11:42,414 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error; reboot the host 2014-04-22 04:11:42,472 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output
[jira] [Updated] (CLOUDSTACK-6475) [Automation] communication between cloudstack agent and MS disconnecting continuously
[ https://issues.apache.org/jira/browse/CLOUDSTACK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Ganesh updated CLOUDSTACK-6475: --- Assignee: Kishan Kavala [Automation] communication between cloudstack agent and MS disconnecting continuously --- Key: CLOUDSTACK-6475 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6475 Project: CloudStack Issue Type: Bug Security Level: Public(Anyone can view this level - this is the default.) Components: KVM Affects Versions: 4.4.0 Environment: RHEL 6.3 Reporter: Rayees Namathponnan Assignee: Kishan Kavala Priority: Blocker Fix For: 4.4.0 Attachments: Agent_log.rar, management-server.rar This issue is observed during automation, run. communication between cloudstack agent and ms getting disconnected continuously; observed below error in agent log 2014-04-22 04:08:47,867 INFO [cloud.agent.Agent] (AgentShutdownThread:null) Stopping the agent: Reason = sig.kill 2014-04-22 04:10:41,456 INFO [cloud.agent.AgentShell] (main:null) Agent started 2014-04-22 04:10:41,500 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.4.0-SNAPSHOT 2014-04-22 04:10:41,502 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /etc/cloudstack/agent/agent.properties 2014-04-22 04:10:41,551 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage 2014-04-22 04:10:41,552 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm 2014-04-22 04:10:41,572 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2014-04-22 04:10:41,722 INFO [cloud.agent.Agent] (main:null) id is 0 2014-04-22 04:10:42,501 INFO [kvm.resource.LibvirtComputingResource] (main:null) No libvirt.vif.driver specified. Defaults to BridgeVifDriver. 2014-04-22 04:10:42,590 INFO [cloud.agent.Agent] (main:null) Agent [id = 0 : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host = 10.223.49.195 : port = 8250 2014-04-22 04:10:42,664 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.223.49.195:8250 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) SSL: Handshake done 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) Connected to 10.223.49.195:8250 2014-04-22 04:10:42,941 WARN [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:null) Could not read cpuinfo_max_freq 2014-04-22 04:10:43,158 INFO [cloud.serializer.GsonHelper] (Agent-Handler-1:null) Default Builder inited. 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Set agent id 0 2014-04-22 04:10:43,233 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Startup Response Received: agent id = 0 2014-04-22 04:11:40,925 INFO [cloud.agent.Agent] (Agent-Handler-1:null) Lost connection to the server. Dealing with the remaining commands... 2014-04-22 04:11:42,352 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 0 2014-04-22 04:11:42,368 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 1 2014-04-22 04:11:42,383 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 2 2014-04-22 04:11:42,398 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 3 2014-04-22 04:11:42,413 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 4 2014-04-22 04:11:42,414 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error; reboot the host 2014-04-22 04:11:42,472 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error:
[jira] [Updated] (CLOUDSTACK-6475) [Automation] communication between cloudstack agent and MS disconnecting continuously
[ https://issues.apache.org/jira/browse/CLOUDSTACK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rayees Namathponnan updated CLOUDSTACK-6475: Attachment: management-server.rar [Automation] communication between cloudstack agent and MS disconnecting continuously --- Key: CLOUDSTACK-6475 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6475 Project: CloudStack Issue Type: Bug Security Level: Public(Anyone can view this level - this is the default.) Components: KVM Affects Versions: 4.4.0 Environment: RHEL 6.3 Reporter: Rayees Namathponnan Priority: Blocker Fix For: 4.4.0 Attachments: management-server.rar This issue is observed during automation, run. communication between cloudstack agent and ms getting disconnected continuously; observed below error in agent log 2014-04-22 04:08:47,867 INFO [cloud.agent.Agent] (AgentShutdownThread:null) Stopping the agent: Reason = sig.kill 2014-04-22 04:10:41,456 INFO [cloud.agent.AgentShell] (main:null) Agent started 2014-04-22 04:10:41,500 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.4.0-SNAPSHOT 2014-04-22 04:10:41,502 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /etc/cloudstack/agent/agent.properties 2014-04-22 04:10:41,551 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage 2014-04-22 04:10:41,552 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm 2014-04-22 04:10:41,572 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2014-04-22 04:10:41,722 INFO [cloud.agent.Agent] (main:null) id is 0 2014-04-22 04:10:42,501 INFO [kvm.resource.LibvirtComputingResource] (main:null) No libvirt.vif.driver specified. Defaults to BridgeVifDriver. 2014-04-22 04:10:42,590 INFO [cloud.agent.Agent] (main:null) Agent [id = 0 : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host = 10.223.49.195 : port = 8250 2014-04-22 04:10:42,664 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.223.49.195:8250 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) SSL: Handshake done 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) Connected to 10.223.49.195:8250 2014-04-22 04:10:42,941 WARN [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:null) Could not read cpuinfo_max_freq 2014-04-22 04:10:43,158 INFO [cloud.serializer.GsonHelper] (Agent-Handler-1:null) Default Builder inited. 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Set agent id 0 2014-04-22 04:10:43,233 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Startup Response Received: agent id = 0 2014-04-22 04:11:40,925 INFO [cloud.agent.Agent] (Agent-Handler-1:null) Lost connection to the server. Dealing with the remaining commands... 2014-04-22 04:11:42,352 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 0 2014-04-22 04:11:42,368 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 1 2014-04-22 04:11:42,383 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 2 2014-04-22 04:11:42,398 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 3 2014-04-22 04:11:42,413 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 4 2014-04-22 04:11:42,414 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error; reboot the host 2014-04-22 04:11:42,472 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error,
[jira] [Updated] (CLOUDSTACK-6475) [Automation] communication between cloudstack agent and MS disconnecting continuously
[ https://issues.apache.org/jira/browse/CLOUDSTACK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rayees Namathponnan updated CLOUDSTACK-6475: Attachment: Agent_log.rar [Automation] communication between cloudstack agent and MS disconnecting continuously --- Key: CLOUDSTACK-6475 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6475 Project: CloudStack Issue Type: Bug Security Level: Public(Anyone can view this level - this is the default.) Components: KVM Affects Versions: 4.4.0 Environment: RHEL 6.3 Reporter: Rayees Namathponnan Priority: Blocker Fix For: 4.4.0 Attachments: Agent_log.rar, management-server.rar This issue is observed during automation, run. communication between cloudstack agent and ms getting disconnected continuously; observed below error in agent log 2014-04-22 04:08:47,867 INFO [cloud.agent.Agent] (AgentShutdownThread:null) Stopping the agent: Reason = sig.kill 2014-04-22 04:10:41,456 INFO [cloud.agent.AgentShell] (main:null) Agent started 2014-04-22 04:10:41,500 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.4.0-SNAPSHOT 2014-04-22 04:10:41,502 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /etc/cloudstack/agent/agent.properties 2014-04-22 04:10:41,551 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage 2014-04-22 04:10:41,552 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm 2014-04-22 04:10:41,572 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2014-04-22 04:10:41,722 INFO [cloud.agent.Agent] (main:null) id is 0 2014-04-22 04:10:42,501 INFO [kvm.resource.LibvirtComputingResource] (main:null) No libvirt.vif.driver specified. Defaults to BridgeVifDriver. 2014-04-22 04:10:42,590 INFO [cloud.agent.Agent] (main:null) Agent [id = 0 : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host = 10.223.49.195 : port = 8250 2014-04-22 04:10:42,664 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.223.49.195:8250 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) SSL: Handshake done 2014-04-22 04:10:42,920 INFO [utils.nio.NioClient] (Agent-Selector:null) Connected to 10.223.49.195:8250 2014-04-22 04:10:42,941 WARN [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:null) Could not read cpuinfo_max_freq 2014-04-22 04:10:43,158 INFO [cloud.serializer.GsonHelper] (Agent-Handler-1:null) Default Builder inited. 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 2014-04-22 04:10:43,227 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Set agent id 0 2014-04-22 04:10:43,233 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Startup Response Received: agent id = 0 2014-04-22 04:11:40,925 INFO [cloud.agent.Agent] (Agent-Handler-1:null) Lost connection to the server. Dealing with the remaining commands... 2014-04-22 04:11:42,352 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 0 2014-04-22 04:11:42,368 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 1 2014-04-22 04:11:42,383 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 2 2014-04-22 04:11:42,398 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 3 2014-04-22 04:11:42,413 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error, retry: 4 2014-04-22 04:11:42,414 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output error; reboot the host 2014-04-22 04:11:42,472 WARN [kvm.resource.KVMHAMonitor] (Thread-4:null) write heartbeat failed: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh: line 131: echo: write error: Input/output