[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

GitHub user serbaut opened a pull request:

https://github.com/apache/cloudstack/pull/981

CLOUDSTACK-8993: DHCP fails with "no address available" when an IP is…

… reused

Repopulate /etc/dhcphosts.txt to remove old entries with the same IP 
address.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/serbaut/cloudstack repopulate-dhcphost

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/981.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #981


commit eb22ca8df0ce15a122ace4c425cff02717e6d63b
Author: Joakim Sernbrant 
Date:   2015-10-26T14:15:19Z

CLOUDSTACK-8993: DHCP fails with "no address available" when an IP is reused

Repopulate /etc/dhcphosts.txt to remove old entries with the same IP 
address.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151418751
  
I tested this manually. I did not manage to get any test environment to 
work but there is some tests in test/systemvm/test_update_config.py that could 
probably be extended to test this. Not sure where to start with running those 
tests though :(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151505153
  
Thanks, I will have another go at running the tests to see if I can write a 
test for it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151652563
  
Getting errors when running test/systemvm/test_update_config.py.

from envassert import file, process, package, user, group, port, 
cron, detect, ip
ImportError: cannot import name ip

Got other errors when setting up devcloud by following 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/devcloud-kvm but I 
managed to work around it.

Exception Occurred Under createLogs :['Traceback (most recent call 
last):\n', '  File 
"/usr/local/lib/python2.7/site-packages/Marvin-4.6.0_SNAPSHOT-py2.7.egg/marvin/marvinLog.py",
 line 157, in createLogs\n(\'LogFolderPath\' in log_cfg.__dict__.keys()) 
and\n', "Attri
buteError: 'list' object has no attribute '__dict__'\n"]

Not sure how to proceed. Are these tests supposed to work at all?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151768463
  
In our environment it was really easy to reproduce.
* Deploy instance 1, got IP 10.7.32.101
* Delete and expunge instance 1
* Deploy instance 2, got IP 100.7.32.101

/etc/dhcphosts.txt now looked like

mac1,10.7.32.101,infinite
mac2,10.7.32.101,infinite

And ndsmasq failed with a warning about duplicates and "no address 
available".

I am still trying to get marvin tests inside devcloud to work. Is there a 
way to reach you if I need assistance. Are the tests i tried to run above 
(test_update_config.py) supposed to work or are the deprecated?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151805357
  
Thanks. I still haven't given up on writing a test for this. Just found one 
issue with running

python2.7 marvin/marvin/deployDataCenter.py -i 
devcloud-kvm/devcloud-kvm-advanced.cfg

That config uses a logger config that doesn't work with marvin.

Now I can't get nose to use python2.7, even when running nosetest2.7:

[root@devcloud-kvm cloudstack]# nosetests-2.7 --with-marvin 
--marvin-config=tools/devcloud-kvm/devcloud-kvm-advanced.cfg -a tags='devcloud'
 Marvin Init Started 

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

 Log Folder Path: /tmp//MarvinLogs//Oct_28_2015_05_10_35_QWNYQ4. 
All logs will be available here 

=== Marvin Init Logging Successful===

 Marvin Init Successful 
Traceback (most recent call last):
  File 
"/root/cloudstack/test/integration/component/find_hosts_for_migration.py", line 
21, in 
from nose.plugins.attrib import attr
ImportError: cannot import name attr
Traceback (most recent call last):
  File 
"/root/cloudstack/test/integration/component/maint/test_bugs.py", line 19, in 

from marvin.cloudstackTestCase import cloudstackTestCase, unittest
ImportError: No module named marvin.cloudstackTestCase
^C  File 
"/root/cloudstack/test/integration/component/maint/test_dedicate_guest_vlan_ranges.py",
 line 26, in 
from nose.plugins.attrib import attr

Is there a better/simpler way to run the tests? Should i not use 
devcloud-kvm? Should I not use advanced networking? Just trying to follow 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/devcloud-kvm

A "make test" that just worked would have been nice :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151843789
  
Yes I have a running zone with advanced networking. Giving up on the tests, 
too much breakage everywhere. I must be doing something wrong.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151855358
  
Wtf Jenkins, no means no.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-151894479
  
Thank you. Sorry I could not write a test for it. Maybe the instructions 
for running tests need to be updated because I ran into all sorts of problems. 
Can't even get Jenkins to execute the tests it seems :(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-152175108
  
Sorry, been busy. Running tests now.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9007 - Write test to check tha...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/1002#issuecomment-152180011
  
Thank you, that would have taken me days to write.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9007 - Write test to check tha...

[serbaut]

Github user serbaut commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1002#discussion_r43384431
  
--- Diff: test/integration/component/test_router_dhcphosts.py ---
@@ -0,0 +1,381 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Import Local Modules
+from nose.plugins.attrib import attr
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackAPI import (stopVirtualMachine,
+  stopRouter,
+  startRouter)
+from marvin.lib.utils import (cleanup_resources,
+  get_process_status)
+from marvin.lib.base import (ServiceOffering,
+ VirtualMachine,
+ Account,
+ ServiceOffering,
+ NATRule,
+ NetworkACL,
+ FireWallRule,
+ PublicIPAddress,
+ NetworkOffering,
+ Network,
+ Router)
+from marvin.lib.common import (get_zone,
+   get_template,
+   get_domain,
+   list_virtual_machines,
+   list_networks,
+   list_configurations,
+   list_routers,
+   list_nat_rules,
+   list_publicIP,
+   list_firewall_rules,
+   list_hosts)
+
+# Import System modules
+import time
+import logging
+
+
+class TestRouterDHCPHosts(cloudstackTestCase):
+
+@classmethod
+def setUpClass(cls):
+
+cls.logger = logging.getLogger('TestRouterDHCPHosts')
+cls.stream_handler = logging.StreamHandler()
+cls.logger.setLevel(logging.DEBUG)
+cls.logger.addHandler(cls.stream_handler)
+
+cls.testClient = super(TestRouterDHCPHosts, cls).getClsTestClient()
+cls.api_client = cls.testClient.getApiClient()
+
+cls.services = cls.testClient.getParsedTestDataConfig()
+# Get Zone, Domain and templates
+cls.domain = get_domain(cls.api_client)
+cls.zone = get_zone(cls.api_client, 
cls.testClient.getZoneForTests())
+cls.services['mode'] = cls.zone.networktype
+cls.template = get_template(
+cls.api_client,
+cls.zone.id,
+cls.services["ostype"]
+)
+
+cls.services["virtual_machine"]["zoneid"] = cls.zone.id
+
+cls.logger.debug("Creating Admin Account for domain %s on zone %s" 
% (cls.domain.id, cls.zone.id))
+# Create an account, network, VM and IP addresses
+cls.account = Account.create(
+cls.api_client,
+cls.services["account"],
+admin=True,
+domainid=cls.domain.id
+)
+
+cls.logger.debug("Creating Service Offering on zone %s" % 
(cls.zone.id))
+cls.service_offering = ServiceOffering.create(
+cls.api_client,
+cls.services["service_offering"]
+)
+
+cls.services["isolated_network_offering"]["egress_policy"] = "true"
+
+cls.logger.debug("Creating Network Offering on zone %s" % 
(cls.zone.id))
+cls.network_offering = NetworkOffering.create(cls.api_client,
+   
cls.services["isolated_network_offering"],
+   conservemode=True)
+
+cls.netwo

[GitHub] cloudstack pull request: CLOUDSTACK-8993: DHCP fails with "no addr...

[serbaut]

Github user serbaut commented on the pull request:

https://github.com/apache/cloudstack/pull/981#issuecomment-152481954
  
Afaict, the DHCP code is rewritten in 4.6 so this exact issue shouldn't 
affect 4.5.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1915: CLOUDSTACK-9746 system-vm: logrotate config c...

[serbaut]

GitHub user serbaut opened a pull request:

https://github.com/apache/cloudstack/pull/1915

CLOUDSTACK-9746 system-vm: logrotate config causes critical failures

* rotate both daily and by size by using maxsize in stead of size
* decrease the max size to 10M and reduce the number of kept files to a 
maximum of 5

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/serbaut/cloudstack CLOUDSTACK-9746

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1915.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1915


commit de9c0ab0a4cf31f097c9f7e7039add4240a62ac9
Author: Joakim Sernbrant 
Date:   2017-01-23T11:13:15Z

CLOUDSTACK-9746 system-vm: logrotate config causes critical failures

* rotate both daily and by size by using maxsize in stead of size
* decrease the max size to 10M and reduce the number of kept files to a 
maximum of 5




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1919: CLOUDSTACK-9763: set ssh public key when crea...

[serbaut]

GitHub user serbaut opened a pull request:

https://github.com/apache/cloudstack/pull/1919

CLOUDSTACK-9763: set ssh public key when creating vm data



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/serbaut/cloudstack CLOUDSTACK-9763

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1919.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1919






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1919: CLOUDSTACK-9763: set ssh public key when creating vm...

[serbaut]

Github user serbaut commented on the issue:

https://github.com/apache/cloudstack/pull/1919
  
The VPC VR maintains metadata 
(http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/virtual_machines/user-data.html)
 as static files in /var/www/html/metadata. When a VR is destroyed and 
recreated (by e.g. "restart with cleanup") this metadata is rebuilt by 
createVmDataCommandForVMs(). public-keys is missing from that function so it 
becomes empty after the rebuild and a request for latest/meta-data/public-keys 
no longer returns the correct key.

This PR adds public-key to the rebuild.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1915: CLOUDSTACK-9746 system-vm: logrotate config causes c...

[serbaut]

Github user serbaut commented on the issue:

https://github.com/apache/cloudstack/pull/1915
  
@dmabry yes we are using redundant VPC routers. 

Using `size` instead of `maxsize` would delay the rotation of logs until 
they reach `size`. 10 logfiles with size 10MB could consume 200MB due to 
deplaycompress. Using `maxsize` would rotate the slowly growing logfiles daily 
and avoid this issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1915: CLOUDSTACK-9746 system-vm: logrotate config causes c...

[serbaut]

Github user serbaut commented on the issue:

https://github.com/apache/cloudstack/pull/1915
  
Is it safe to remove delaycompress across the board, I assume it is there 
for a reason?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1915: CLOUDSTACK-9746 system-vm: logrotate config causes c...

[serbaut]

Github user serbaut commented on the issue:

https://github.com/apache/cloudstack/pull/1915
  
Ok. I removed it from rsyslog since it should be safe there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---