Re: [Openstack] Suggestions for shared-storage cluster file system
I haven't played that much with Glusterfs and pacemaker but I'm pretty sure that there is already a resource agent for this. Maybe the Filesystem RA can do it but you probably might have a look at the upstream version on the Cluster Lab Github. Cheers On 15 févr. 2013, at 20:54, Samuel Winchenbach wrote: > Thanks, > > I think I will go with GlusterFS. MooseFS looks interesting, but > maintaining a package outside the repo/cloud archive is not something I want > to deal with. > > Along the same lines... is it possible to mount a GlusterFS volume in > pacemaker? I have tried both ocf:heartbeat:Filesystem and > ocf:redhat:netfs.sh without much luck. I have managed to get the service > started with upstart though. > > Thanks, > Sam > > > On Fri, Feb 15, 2013 at 2:29 PM, Sébastien Han > wrote: >> Hi, >> >> >>> Important: Mount the CephFS filesystem on the client machine, not the >>> cluster machine. >> >> It's just like NFS, if you mount an NFS export on the NFS server, you get >> kernel locks. >> >> Unfortunately even if love Ceph far more than the other, I won't go with >> CephFS, at least not know. But if are in a hurry and looking for a DFS then >> GlusterFS seems to be a good candidate. NFS works pretty well too. >> >> Cheers. >> >> -- >> Regards, >> Sébastien Han. >> >> >> On Fri, Feb 15, 2013 at 4:49 PM, JuanFra Rodriguez Cardoso >> wrote: >>> Another one: >>> >>> - MooseFS >>> (http://docs.openstack.org/trunk/openstack-compute/admin/content/installing-moosefs-as-backend.html) >>> - GlusterFS >>> - Ceph >>> - Lustre >>> >>> Regards, >>> JuanFra >>> >>> >>> 2013/2/15 Samuel Winchenbach Hi All, Can anyone give me a recommendation for a good shared-storage cluster filesystem? I am running kvm-libvirt and would like to enable live migration. I have a number of hosts (up to 16) each with 2xTB drives. These hosts are also my compute/network/controller nodes. The three I am considering are: GlusterFS - I have the most experience with this, and it seems the easiest. CephFS/RADOS - Interesting because glance supports the rbd backend. Slightly worried because of this though "Important: Mount the CephFS filesystem on the client machine, not the cluster machine." (I wish it said why...) and "CephFS is not quite as stable as the block device and the object storage gateway." Lustre - A little hesitant now that Oracle is involved with it. If anyone has any advice, or can point out another that I should consider it would be greatly appreciated. Thanks! Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp >>> >>> >>> ___ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] HA Openstack with Pacemaker
Hi, Good to hear that you finally managed to get it working. Usually the postrouting rule is more for clients that needs to be routed. Cheers! On 16 févr. 2013, at 03:06, Samuel Winchenbach wrote: > Well I got it to work. I was being stupid, and forgot to change over the > endpoints in keystone. > > One thing I find interesting is that if I call "keystone user-list" from > test1 it _always_ sends the request to test2 and vice versa. > > Also I did not need to add the POSTROUTING rule... I am not sure why. > > > On Fri, Feb 15, 2013 at 3:44 PM, Samuel Winchenbach > wrote: >> Hrmmm it isn't going so well: >> >> root@test1# ip a s dev eth0 >> 2: eth0: mtu 1500 qdisc mq state UP qlen >> 1000 >> link/ether 00:25:90:10:00:78 brd ff:ff:ff:ff:ff:ff >> inet 10.21.0.1/16 brd 10.21.255.255 scope global eth0 >> inet 10.21.1.1/16 brd 10.21.255.255 scope global secondary eth0 >> inet 10.21.21.1/16 scope global secondary eth0 >> inet6 fe80::225:90ff:fe10:78/64 scope link >>valid_lft forever preferred_lft forever >> >> >> root@test1# ipvsadm -L -n >> IP Virtual Server version 1.2.1 (size=4096) >> Prot LocalAddress:Port Scheduler Flags >> -> RemoteAddress:Port Forward Weight ActiveConn InActConn >> TCP 10.21.21.1:5000 wlc persistent 600 >> -> 10.21.0.1:5000 Masq1000 1 >> -> 10.21.0.2:5000 Masq1000 0 >> TCP 10.21.21.1:35357 wlc persistent 600 >> -> 10.21.0.1:35357 Masq1000 0 >> -> 10.21.0.2:35357 Masq1000 0 >> >> root@test1# iptables -L -v -tnat >> Chain PREROUTING (policy ACCEPT 283 packets, 24902 bytes) >> pkts bytes target prot opt in out source >> destination >> >> Chain INPUT (policy ACCEPT 253 packets, 15256 bytes) >> pkts bytes target prot opt in out source >> destination >> >> Chain OUTPUT (policy ACCEPT 509 packets, 37182 bytes) >> pkts bytes target prot opt in out source >> destination >> >> Chain POSTROUTING (policy ACCEPT 196 packets, 12010 bytes) >> pkts bytes target prot opt in out source >> destination >> 277 16700 MASQUERADE all -- anyeth0anywhere anywhere >> >> root@test1:~# export OS_AUTH_URL="http://10.21.21.1:5000/v2.0/"; >> root@test1:~# keystone user-list >> No handlers could be found for logger "keystoneclient.client" >> Unable to communicate with identity service: [Errno 113] No route to host. >> (HTTP 400) >> >> >> I still have some debugging to do with tcpdump, but I thought I would post >> my initial results. >> >> >> On Fri, Feb 15, 2013 at 2:56 PM, Sébastien Han >> wrote: >>> Well if you follow my article, you will get LVS-NAT running. It's fairly >>> easy, no funky stuff. Yes you will probably need the postrouting rule, as >>> usual :). Let me know how it goes ;) >>> >>> -- >>> Regards, >>> Sébastien Han. >>> >>> >>> On Fri, Feb 15, 2013 at 8:51 PM, Samuel Winchenbach >>> wrote: I didn't give NAT a shot because it didn't seem as well documented. I will give NAT a shot. Will I need to enable to iptables and add a rule to the nat table? None of the documentation mentioned that but every time I have ever done NAT I had to setup a rule like... iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE Thanks for helping me with this. On Fri, Feb 15, 2013 at 2:07 PM, Sébastien Han wrote: > Ok but why direct routing instead of NAT? If the public IPs are _only_ > on LVS there is no point to use LVS-DR. > > LVS has the public IPs and redirects to the private IPs, this _must_ work. > > Did you try NAT? Or at least can you give it a shot? > -- > Regards, > Sébastien Han. > > > On Fri, Feb 15, 2013 at 3:55 PM, Samuel Winchenbach > wrote: > > Sure... I have undone these settings but I saved a copy: > > > > two hosts: > > test1 eth0: 10.21.0.1/16 eth1: 130.x.x.x/24 > > test2 eth0: 10.21.0.2/16 eth1: 130.x.x.x/24 > > > > VIP: 10.21.21.1 (just for testing, later I would add a 130.x.x.x/24 > > VIP for > > public APIs > > > > k > > eystone is bound to 10.21.0.1 on test1 and 10.21.0.2 on test2 > > > > > > > > in /etc/sysctl.conf: > >net.ipv4.conf.all.arp_ignore = 1 > >net.ipv4.conf.eth0.arp_ignore = 1 > >net.ipv4.conf.all.arp_announce = 2 > >net.ipv4.conf.eth0.arp_announce = 2 > > > > root# sysctl -p > > > > in /etc/sysctl.conf: > > > > checktimeout= > > 3 > > > > > > checkinterval= > > 5 > > > > > > autoreload= > > yes > > > > > > logfile="/var/log/ldirectord.log" > > > > quiescent=no > > > > virtua
Re: [Openstack] Can't attach a RBD volume to an instance
Hi,
Couple of days ago, I updated my guide about this issue.
Cheers!
On 14 juil. 2012, at 04:03, Travis Rhoden wrote:
> Josh,
>
> Thanks so much for the pointers. I did not figure out that libvirt was
> trying to read /etc/ceph/ceph.conf. It was in fact being denied by apparmor.
> I fixed my problem by editing /etc/apparmor.d/libvirt/TEMPLATE and adding
> "/etc/ceph/** r," to the template. Now it works for newly spawned VMs.
>
> Again, thanks a lot.
>
> - Travis
>
> On Fri, Jul 13, 2012 at 7:40 PM, Josh Durgin wrote:
> On 07/13/2012 03:09 PM, Travis Rhoden wrote:
> Was this problem ever resolved? I'm having the identical issue right now
> (perhaps because I am following Sebastien's guide [0]). I found this
> question raised on LP [1], but no helpful responses, and no linked bug. I
> can confirm that I can attach an RBD instance manually through virsh, no
> problem. So it is not a libvirt thing. It sure appears to be a faulty disk
> attachment definition from OpenStack.
>
> I didn't notice this launchpad question before, but I'll copy my answer
> below for the mailing list archives:
>
> Right now the RBD driver in OpenStack requires you to have your monitor
> addresses in /etc/ceph/ceph.conf. When you have them there, putting them in
> the libvirt xml is unnecessary. You do have to make sure that the unix user
> who ends up running qemu/kvm has read access to /etc/ceph/ceph.conf, and e.g.
> apparmor isn't preventing qemu/kvm from reading it.
>
> This dependency on /etc/ceph/ceph.conf should be removed in the future.
>
>
> I'm running the latest packages on Ubuntu 12.04 for OpensStack (essex) and
> Ceph (argonaut).
>
> I can confirm when using nova volume-attach it does not matter what device
> you pick, dev/vdb, dev/vdf, whatever. Always fails. /dev/vdb manually
> works great.
>
> If the answer above doesn't help, what error message do you get?
> Is there any info in the libvirt logs for the instance?
>
> Josh
>
> [0]
> http://www.sebastien-han.fr/blog/2012/06/10/introducing-ceph-to-openstack/
> [1] https://answers.launchpad.net/nova/+question/201366
>
> - Travis
>
> On Fri, May 25, 2012 at 8:32 PM, Josh Durgin wrote:
>
> On 05/25/2012 01:31 AM, Sébastien Han wrote:
>
> Hi everyone,
>
> I setup a ceph cluster and I use the RBD driver for nova-volume.
> I can create volumes and snapshots but currently I can't attach them to
> an instance.
> Apparently the volume is detected as busy but it's not, no matter which
> name I choose.
> I tried from horizon and the command line, same issue.
>
> $ nova --debug volume-attach de3743be-69be-4ec3-8de3-**fc7df6903a97 12
>
> /dev/vdc
> connect: (localhost, 5000)
> send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
> localhost:5000\r\nContent-**Length: 100\r\ncontent-type:
> application/json\r\naccept-**encoding: gzip, deflate\r\naccept:
> application/json\r\nuser-**agent: python-novaclient\r\n\r\n{"**auth":
>
> {"tenantName": "admin", "passwordCredentials": {"username": "admin",
> "password": "admin"}}}'
> reply: 'HTTP/1.1 200 OK\r\n'
> header: Content-Type: application/json
> header: Vary: X-Auth-Token
> header: Date: Fri, 25 May 2012 08:29:46 GMT
> header: Transfer-Encoding: chunked
> connect: (172.17.1.6, 8774)
> send: u'GET
> /v2/**d1f5d27ccf594cdbb034c8a4123494**e9/servers/de3743be-69be-4ec3-**
>
> 8de3-fc7df6903a97
> HTTP/1.1\r\nHost: 172.17.1.6:8774
> \r\nx-**auth-project-id: admin\r\nx-auth-token:
>
> 37ff6354b9114178889128175494c6**66\r\naccept-encoding: gzip,
> deflate\r\naccept: application/json\r\nuser-**agent:
>
> python-novaclient\r\n\r\n'
> reply: 'HTTP/1.1 200 OK\r\n'
> header: X-Compute-Request-Id: req-fc7686dd-7dff-4ea2-b2dd-**be584dadb7c1
>
> header: Content-Type: application/json
> header: Content-Length: 1371
> header: Date: Fri, 25 May 2012 08:29:46 GMT
> send: u'POST
> /v2/**d1f5d27ccf594cdbb034c8a4123494**e9/servers/de3743be-69be-4ec3-**
> 8de3-fc7df6903a97/os-volume_**attachments
>
> HTTP/1.1\r\nHost: 172.17.1.6:8774
> \r\**nContent-Length: 60\r\nx-auth-project-id:
>
>
> admin\r\naccept-encoding: gzip, deflate\r\naccept:
> application/json\r\nx-auth-**token:
> 37ff6354b9114178889128175494c6**66\r\nuser-agent:
> python-novaclient\r\ncontent-**type:
> application/json\r\n\r\n{"**volumeAttachment": {"device": "/dev/vdc",
>
> "volumeId": 12}}'
> reply: 'HTTP/1.1 200 OK\r\n'
> header: X-Compute-Request-Id: req-4a22f603-4aad-4ae9-99a2-**6a7bcacf974f
>
> header: Content-Type: application/json
> header: Content-Length: 48
> header: Date: Fri, 25 May 2012 08:29:46 GMT
> $ echo $?
> 0
>
> See the logs here:
> http://paste.openstack.org/**show/18163/
Re: [Openstack] Volume creation stuck at status "Creating"
Hi,
Does the libvirt logs show something?
On 31 mai 2012, at 16:27, John Griffith wrote:
>
>
> On Thu, May 31, 2012 at 5:58 AM, Alisson Soares Limeira Pontes
> wrote:
> Something changed in the nova-volume ubuntu packages instalation.
> I'd never had problems with nova-volume, but in my last two setups i cannot
> attach volume and vm. Logs do not show any problem, so no clue why this
> problem.
>
>
> 2012/5/31 Milind
> Thanks,
>
> Any one know what is the problem or it is another bug?
>
> REgards,
> Milind
>
> On Thu, May 31, 2012 at 12:22 PM, William Herry
> wrote:
> I can't tell what's wrong about your config, let others help you
>
> On Thu, May 31, 2012 at 2:50 PM, Milind wrote:
> Hi Just ran command... here is the output...
>
> root@ucmaster:/var/log/upstart# nova --debug volume-create
> --display_name=novavol5 --display_description=volumenova5 1
> connect: (ucmaster, 5000)
> send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: ucmaster:5000\r\nContent-Length:
> 117\r\ncontent-type: application/json\r\naccept-encoding: gzip,
> deflate\r\naccept: application/json\r\nuser-agent:
> python-novaclient\r\n\r\n{"auth": {"tenantName": "openstackDemo",
> "passwordCredentials": {"username": "adminUser", "password": "secretword"}}}'
> reply: 'HTTP/1.1 200 OK\r\n'
> header: Content-Type: application/json
> header: Vary: X-Auth-Token
> header: Date: Thu, 31 May 2012 06:42:50 GMT
> header: Transfer-Encoding: chunked
> connect: (ucmaster, 8776)
> send: u'POST /v1/68124ec3a778447093eba446aa896105/volumes HTTP/1.1\r\nHost:
> ucmaster:8776\r\nContent-Length: 131\r\nx-auth-project-id:
> openstackDemo\r\naccept-encoding: gzip, deflate\r\naccept:
> application/json\r\nx-auth-token:
> b0b433d2506145d0b4f5ab8e1fcb6478\r\nuser-agent:
> python-novaclient\r\ncontent-type: application/json\r\n\r\n{"volume":
> {"snapshot_id": null, "display_name": "novavol5", "volume_type": null,
> "display_description": "volumenova5", "size": 1}}'
> reply: 'HTTP/1.1 200 OK\r\n'
> header: X-Compute-Request-Id: req-58d480c3-f729-4670-8c86-74ef9991993a
> header: Content-Type: application/json
> header: Content-Length: 274
> header: Date: Thu, 31 May 2012 06:42:50 GMT
> root@ucmaster:/var/log/upstart# clear
>
>
>
>
>
>
> On Thu, May 31, 2012 at 12:12 PM, Milind wrote:
>
>
>
> On Thu, May 31, 2012 at 12:08 PM, William Herry
> wrote:
> I can't see any unusual log
>
> try:
>
> nova --debug volume-create --display_name=novavol4
> --display_description=volumenova4 1
>
>
> On Thu, May 31, 2012 at 2:33 PM, Milind wrote:
> I did that and issued following command.
>
> nova volume-create --display_name=novavol4 --display_description=volumenova4 1
>
> I have attached relevant part of log.
>
> Regards,
> Milind
>
>
> On Thu, May 31, 2012 at 11:48 AM, William Herry
> wrote:
> add debug=true to /etc/nova/nova.conf and restart service(nova-volume I think)
>
> On Thu, May 31, 2012 at 2:16 PM, Milind wrote:
> In which config file?
>
>
> On Thu, May 31, 2012 at 11:43 AM, William Herry
> wrote:
> have you add debug=true to config file
>
> On Thu, May 31, 2012 at 2:11 PM, Milind wrote:
> There is nothing in the logs :(
>
>
>
> On Thu, May 31, 2012 at 11:37 AM, William Herry
> wrote:
> you should check logs to see what's going on
>
> On Thu, May 31, 2012 at 1:50 PM, Milind wrote:
> Hi,
>
> I am not able to resolve this error quite frustrated :(, it gets stuck at
> volume creation with status as "Creating"
>
>
>
>
> Regards,
> Milind
> --
> Alisson Pontes
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>
> Hi Milind,
>
> Sorry if you've already answered this, but what sort of setup is this? Are
> you using Essex, or latest from Trunk? Is this a devstack install or other?
> This is something I've seen before as well and it had to do with issues in my
> configuration for the scheduler. Let me know a bit more about your setup if
> you have a chance. Perhaps I can try to recreate and have a closer look.
>
> Thanks,
> John
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
