Re: [Openstack] Command "nova-manage service list" output
Hi
Or you can just redirect the stderr:
nova-manage service list 2> /dev/null
HTH
--
Andrea Rosa
From: Openstack
[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On Behalf Of
Jay Lau
Sent: 20 July 2013 15:29
To: amogh patel
Cc: openstack
Subject: Re: [Openstack] Command "nova-manage service list" output
If you configured "debug=True" in nova.conf, just remove it and have a try.
Thanks,
Jay
2013/7/20 amogh patel mailto:amoghpate...@gmail.com>>
Hi everyone,
My Openstack environment is up and running, but when I'm trying to fire
"nova-manage service list" command it gives me below DEBUG message, though I'm
getting the smiley as well. Could anyone please advise on how to fix this DEBUG
issue?
2013-07-19 22:10:02.869 DEBUG nova.servicegroup.api
[req-a57f42be-0e31-48c3-92db-9d60b7dfda33 None None] Check if the given member
[{'binary': u'nova-compute', 'availability_zone': 'nova', 'deleted': 0L,
'created_at': datetime.datetime(2013, 7, 19, 0, 39, 6), 'updated_at':
datetime.datetime(2013, 7, 20, 5, 10), 'report_count': 9782L, 'topic':
u'compute', 'host': u'sjcubuntu13', 'disabled': False, 'deleted_at': None,
'id': 33L}] is part of the ServiceGroup, is up service_is_up
/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:92
2013-07-19 22:10:02.870 DEBUG nova.servicegroup.drivers.db
[req-a57f42be-0e31-48c3-92db-9d60b7dfda33 None None] DB_Driver.is_up
last_heartbeat = 2013-07-20 05:10:00 elapsed = 2.870004 is_up
/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:67
nova-compute sjcubuntu13 nova enabled
:-) 2013-07-20 05:10:00
2013-07-19 22:10:02.870 DEBUG nova.servicegroup.api
[req-a57f42be-0e31-48c3-92db-9d60b7dfda33 None None] Check if the given member
[{'binary': u'nova-network', 'availability_zone': 'internal', 'deleted': 0L,
'created_at': datetime.datetime(2013, 7, 19, 0, 39, 11), 'updated_at':
datetime.datetime(2013, 7, 20, 5, 10, 1), 'report_count': 9996L, 'topic':
u'network', 'host': u'sjcubuntu13', 'disabled': False, 'deleted_at': None,
'id': 34L}] is part of the ServiceGroup, is up service_is_up
/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:92
2013-07-19 22:10:02.870 DEBUG nova.servicegroup.drivers.db
[req-a57f42be-0e31-48c3-92db-9d60b7dfda33 None None] DB_Driver.is_up
last_heartbeat = 2013-07-20 05:10:01 elapsed = 1.870295 is_up
/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:67
nova-network sjcubuntu13 internal enabled
:-) 2013-07-20 05:10:01
thanks,
Amogh
___
Mailing list: https://launchpad.net/~openstack
Post to :
openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack + Nova list, nova show location
Hi
As reported below "nova" command are processed by the nova client, you have to
look for it.
Cheers
--
Andrea Rosa
From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net
[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On Behalf Of
Desta Haileselassie Hagos
Sent: 26 October 2012 09:52
To: openstack@lists.launchpad.net
Subject: [Openstack] OpenStack + Nova list, nova show location
Dear All,
Would you kindly help me where i can find the scripts for "nova list" and "nova
show"
I just can't find it in /user/bin/
I only have the rapper class
#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT:
'python-novaclient==2012.1','console_scripts','nova'
__requires__ = 'python-novaclient==2012.1'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('python-novaclient==2012.1', 'console_scripts',
'nova')()
)
With best regards,
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] High Available queues in rabbitmq
Hi >As for consumer cancellation notifications - I need to remember when >exactly they happen in an HA setup. Maybe you're right. With the HA active/active configuration clients always consume from the Master node so we can have a scenario where the queue is declared on a Slave node but the clients are consuming from the Master node. In this situation if the master node goes down the client doesn't get any error/exception from the connection and the consumer is abruptly disconnected, so the client needs to re-consume the queue (register a new consumer) to be able to get messages from the queue. The consumer cancellation notifications is a way for a client to get a notification (via a basic.cancel message) when the consumer has been disconnected. With the HA configuration we should care about of two things: 1 connection dying : this is already addressed by the existing code 2 basic.cancel notification received by the clients: in my opinion this is not yet addressed >As for duplicated messages - the situation here is no different >whether you have 1 or many rabbits; reconnection logic was already in >place. Perhaps something should be done here, but it seems that the >lack of this logic didn't hurt anyone so far. Maybe it is because IIRC >messages are ack'd immediately and the failure window is very small. Yes you are absolutely right, but with the HA configuration is more likely to have retransmissions. If I have a single rabbit and NOT durable queue, all messages (also messages not yet ack'ed) in the queues are lost if the node goes down, that means that when the server restarts there are no messages left and so no retransmissions. With multiple rabbits, messages not yet ack'ed are mirrored in the other queues and in the event of failure of the master those messages will be retransmitted. >> Is there some plan to have a blueprint for this change? >I don't have such a plan. Should I? I need to deeply investigate and tests the HA Active/Active configuration, but If I can recreate some "complex" configuration to prove that we need to deal with the basic.cancel messages maybe it's worth to have one. Regards -- Andrea Rosa ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] High Available queues in rabbitmq
Hi Eugene, Thanks for the patch. I have a question: it seems to me that this patch is a (good) starting point of a broader change to be able to use HA in active/active configuration with RMQ. As far as I know, with that configuration we need to add some extra logic for consumers to deal with "consumer cancellation notification" and duplicated messages due to a (potentially) re-send after a failover. Is that correct? Is there some plan to have a blueprint for this change? Regards -- Andrea Rosa >-Original Message- >From: Eugene Kirpichov [mailto:ekirpic...@gmail.com] >Sent: 26 July 2012 00:46 >To: Alessandro Tagliapietra; rbry...@redhat.com >Cc: Rosa, Andrea (HP Cloud Services); OpenStack Development Mailing >List; openstack@lists.launchpad.net >Subject: Re: [openstack-dev] [Openstack] High Available queues in >rabbitmq > >Gentlemen, > >Here is my patch: https://review.openstack.org/#/c/10305/ >It also depends on another small patch >https://review.openstack.org/#/c/10197 > >I'd like to ask someone to review it. >Also, how to get these changes into nova? It seems that nova has a >copy-paste of openstack-common inside it, should I just mirror the >changes to nova once they're accepted in openstack-common? > >I'm cc'ing Russell Bryant because he originally created the >openstack-common module. > >On Wed, Jul 25, 2012 at 3:03 AM, Alessandro Tagliapietra > wrote: >> Yup, using as resource is a "old" way as >http://www.rabbitmq.com/ha.html >> Active/active makes sure that you have no downtime and it's simple as >you >> don't need to use DRBD. >> >> 2012/7/25 Rosa, Andrea (HP Cloud Services) >> >>> Sorry for my question, I have just seen from the original thread that >>> we are talking about HA with Active/Active solution. >>> -- >>> Andrea Rosa >>> >>> >-Original Message- >>> >From: Rosa, Andrea (HP Cloud Services) >>> >Sent: 25 July 2012 10:45 >>> >To: Eugene Kirpichov >>> >Cc: openstack-...@lists.openstack.org; Alessandro Tagliapietra; >>> >openstack@lists.launchpad.net >>> >Subject: Re: [openstack-dev] [Openstack] High Available queues in >>> >rabbitmq >>> > >>> >Hi >>> > >>> >Your patch doesn't use a Resource manager, so are you working on an >>> >Active/Active >>> >configuration using mirrored queues? Or are you working on a cluster >>> >configuration? >>> > >>> >I am really interested in that change, thanks for your help. >>> >Regards >>> >-- >>> >Andrea Rosa >>> > >>> >>-Original Message- >>> >>From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net >>> >>[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] >On >>> >>Behalf Of Alessandro Tagliapietra >>> >>Sent: 24 July 2012 17:58 >>> >>To: Eugene Kirpichov >>> >>Cc: openstack-...@lists.openstack.org; >openstack@lists.launchpad.net >>> >>Subject: Re: [Openstack] High Available queues in rabbitmq >>> >> >>> >>Oh, so without the need to put an IP floating between hosts. >>> >>Good job, thanks for helping >>> >> >>> >>Best >>> >> >>> >>Alessandro >>> >> >>> >>Il giorno 24/lug/2012, alle ore 17:49, Eugene Kirpichov ha scritto: >>> >> >>> >>> Hi Alessandro, >>> >>> >>> >>> My patch is about removing the need for pacemaker (and it's >pacemaker >>> >>> that I denoted with the term "TCP load balancer"). >>> >>> >>> >>> I didn't submit the patch yesterday because I underestimated the >>> >>> effort to write unit tests for it and found a few issues on the >way. >>> >I >>> >>> hope I'll finish today. >>> >>> >>> >>> On Tue, Jul 24, 2012 at 12:00 AM, Alessandro Tagliapietra >>> >>> wrote: >>> >>>> Sorry for the delay, i was out from work. >>> >>>> Awesome work Eugene, I don't need the patch instantly as i'm >still >>> >>building the infrastructure. >>> >>>> Will it will take alot of time to go in Ubuntu repositories? >>> >>>> >>> >>>> Why you said you need load balancing? You can use only the >ma
Re: [Openstack] [openstack-dev] High Available queues in rabbitmq
Sorry for my question, I have just seen from the original thread that we are talking about HA with Active/Active solution. -- Andrea Rosa >-Original Message- >From: Rosa, Andrea (HP Cloud Services) >Sent: 25 July 2012 10:45 >To: Eugene Kirpichov >Cc: openstack-...@lists.openstack.org; Alessandro Tagliapietra; >openstack@lists.launchpad.net >Subject: Re: [openstack-dev] [Openstack] High Available queues in >rabbitmq > >Hi > >Your patch doesn't use a Resource manager, so are you working on an >Active/Active >configuration using mirrored queues? Or are you working on a cluster >configuration? > >I am really interested in that change, thanks for your help. >Regards >-- >Andrea Rosa > >>-Original Message- >>From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net >>[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On >>Behalf Of Alessandro Tagliapietra >>Sent: 24 July 2012 17:58 >>To: Eugene Kirpichov >>Cc: openstack-...@lists.openstack.org; openstack@lists.launchpad.net >>Subject: Re: [Openstack] High Available queues in rabbitmq >> >>Oh, so without the need to put an IP floating between hosts. >>Good job, thanks for helping >> >>Best >> >>Alessandro >> >>Il giorno 24/lug/2012, alle ore 17:49, Eugene Kirpichov ha scritto: >> >>> Hi Alessandro, >>> >>> My patch is about removing the need for pacemaker (and it's pacemaker >>> that I denoted with the term "TCP load balancer"). >>> >>> I didn't submit the patch yesterday because I underestimated the >>> effort to write unit tests for it and found a few issues on the way. >I >>> hope I'll finish today. >>> >>> On Tue, Jul 24, 2012 at 12:00 AM, Alessandro Tagliapietra >>> wrote: >>>> Sorry for the delay, i was out from work. >>>> Awesome work Eugene, I don't need the patch instantly as i'm still >>building the infrastructure. >>>> Will it will take alot of time to go in Ubuntu repositories? >>>> >>>> Why you said you need load balancing? You can use only the master >>node and in case the rabbitmq-server dies, switch the ip to the new >>master with pacemaker, that's how I would do. >>>> >>>> Best Regards >>>> >>>> Alessadro >>>> >>>> >>>> Il giorno 23/lug/2012, alle ore 21:49, Eugene Kirpichov ha scritto: >>>> >>>>> +openstack-dev@ >>>>> >>>>> To openstack-dev: this is a discussion of an upcoming patch about >>>>> native RabbitMQ H/A support in nova. I'll post the patch for >>>>> codereview today. >>>>> >>>>> On Mon, Jul 23, 2012 at 12:46 PM, Eugene Kirpichov >> wrote: >>>>>> Yup, that's basically the same thing that Jay suggested :) Obvious >>in >>>>>> retrospect... >>>>>> >>>>>> On Mon, Jul 23, 2012 at 12:42 PM, Oleg Gelbukh >> wrote: >>>>>>> Eugene, >>>>>>> >>>>>>> I suggest just add option 'rabbit_servers' that will override >>>>>>> 'rabbit_host'/'rabbit_port' pair, if present. This won't break >>anything, in >>>>>>> my understanding. >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> Oleg Gelbukh >>>>>>> Mirantis, Inc. >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 23, 2012 at 10:58 PM, Eugene Kirpichov >> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm working on a RabbitMQ H/A patch right now. >>>>>>>> >>>>>>>> It actually involves more than just using H/A queues (unless >>you're >>>>>>>> willing to add a TCP load balancer on top of your RMQ cluster). >>>>>>>> You also need to add support for multiple RabbitMQ's directly to >>nova. >>>>>>>> This is not hard at all, and I have the patch ready and tested >in >>>>>>>> production. >>>>>>>> >>>>>>>> Alessandro, if you need this urgently, I can send you the patch >>right >>>>>>>> now before the discussion co
Re: [Openstack] High Available queues in rabbitmq
Hi Your patch doesn't use a Resource manager, so are you working on an Active/Active configuration using mirrored queues? Or are you working on a cluster configuration? I am really interested in that change, thanks for your help. Regards -- Andrea Rosa >-Original Message- >From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net >[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On >Behalf Of Alessandro Tagliapietra >Sent: 24 July 2012 17:58 >To: Eugene Kirpichov >Cc: openstack-...@lists.openstack.org; openstack@lists.launchpad.net >Subject: Re: [Openstack] High Available queues in rabbitmq > >Oh, so without the need to put an IP floating between hosts. >Good job, thanks for helping > >Best > >Alessandro > >Il giorno 24/lug/2012, alle ore 17:49, Eugene Kirpichov ha scritto: > >> Hi Alessandro, >> >> My patch is about removing the need for pacemaker (and it's pacemaker >> that I denoted with the term "TCP load balancer"). >> >> I didn't submit the patch yesterday because I underestimated the >> effort to write unit tests for it and found a few issues on the way. I >> hope I'll finish today. >> >> On Tue, Jul 24, 2012 at 12:00 AM, Alessandro Tagliapietra >> wrote: >>> Sorry for the delay, i was out from work. >>> Awesome work Eugene, I don't need the patch instantly as i'm still >building the infrastructure. >>> Will it will take alot of time to go in Ubuntu repositories? >>> >>> Why you said you need load balancing? You can use only the master >node and in case the rabbitmq-server dies, switch the ip to the new >master with pacemaker, that's how I would do. >>> >>> Best Regards >>> >>> Alessadro >>> >>> >>> Il giorno 23/lug/2012, alle ore 21:49, Eugene Kirpichov ha scritto: >>> +openstack-dev@ To openstack-dev: this is a discussion of an upcoming patch about native RabbitMQ H/A support in nova. I'll post the patch for codereview today. On Mon, Jul 23, 2012 at 12:46 PM, Eugene Kirpichov > wrote: > Yup, that's basically the same thing that Jay suggested :) Obvious >in > retrospect... > > On Mon, Jul 23, 2012 at 12:42 PM, Oleg Gelbukh > wrote: >> Eugene, >> >> I suggest just add option 'rabbit_servers' that will override >> 'rabbit_host'/'rabbit_port' pair, if present. This won't break >anything, in >> my understanding. >> >> -- >> Best regards, >> Oleg Gelbukh >> Mirantis, Inc. >> >> >> On Mon, Jul 23, 2012 at 10:58 PM, Eugene Kirpichov > >> wrote: >>> >>> Hi, >>> >>> I'm working on a RabbitMQ H/A patch right now. >>> >>> It actually involves more than just using H/A queues (unless >you're >>> willing to add a TCP load balancer on top of your RMQ cluster). >>> You also need to add support for multiple RabbitMQ's directly to >nova. >>> This is not hard at all, and I have the patch ready and tested in >>> production. >>> >>> Alessandro, if you need this urgently, I can send you the patch >right >>> now before the discussion codereview for inclusion in core nova. >>> >>> The only problem is, it breaks backward compatibility a bit: my >patch >>> assumes you have a flag "rabbit_addresses" which should look like >>> "rmq-host1:5672,rmq-host2:5672" instead of the prior rabbit_host >and >>> rabbit_port flags. >>> >>> Guys, can you advise on a way to do this without being ugly and >>> without breaking compatibility? >>> Maybe have "rabbit_host", "rabbit_port" be ListOpt's? But that >sounds >>> weird, as their names are in singular. >>> Maybe have "rabbit_host", "rabbit_port" and also "rabbit_host2", >>> "rabbit_port2" (assuming we only have clusters of 2 nodes)? >>> Something else? >>> >>> On Mon, Jul 23, 2012 at 11:27 AM, Jay Pipes >wrote: On 07/23/2012 09:02 AM, Alessandro Tagliapietra wrote: > Hi guys, > > just an idea, i'm deploying Openstack trying to make it HA. > The missing thing is rabbitmq, which can be easily started in > active/active mode, but it needs to declare the queues adding >an > x-ha-policy entry. > http://www.rabbitmq.com/ha.html > It would be nice to add a config entry to be able to declare >the queues > in that way. > If someone know where to edit the openstack code, else i'll try >to do > that in the next weeks maybe. https://github.com/openstack/openstack- >common/blob/master/openstack/common/rpc/impl_kombu.py You'll need to add the config options there and the queue is >declared here with the options supplied to the ConsumerBase constructor: https://github.com/openstack/openstack- >common/blob/master/openstack/common/rpc/impl_kombu.py#L114 Best, -jay ___ Mailing list:
Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request
Hi
In my opinion there is another point to consider: at this moment it's possible,
in rabbitmq by auth-mechanism-ssl plugin, to use client and server
authentication through certificates.
I don't know 0MQ, so maybe the answer to my question is addressed by 0MQ
documentation (I'll have a look), but is there some feature to easily implement
this security feature?
Best Regards
--
Andrea Rosa
>-Original Message-
>From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net
>[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On
>Behalf Of Yun Mao
>Sent: 27 January 2012 22:17
>To: Alexis Richardson
>Cc: openstack@lists.launchpad.net
>Subject: Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request
>
>Sorry to bring back a rather quiet thread from 3 days ago.
>
>How fast do we need to queueing component to be? My observation from
>Amazon EC2 us-east-1 is about 2 VMs provisioned per second on average.
>Let's say there are 100 messages exchanged for the workload per second
>per VM (which I believe is over estimated), and the peak time workload
>is 100x higher. Then we need a queue that can do 20,000 messages per
>second at the peak rate. Either Rabbit or 0MQ should handle this very
>easily. So I'm assuming performance is not a concern.
>
>Now if we go brokerless completely for all messages, that's an obvious
>gain as we get rid of one type source of failure. Can that be
>achieved? My impression after quickly skimming through the 0MQ
>document is that those direct connection can be brokerless but things
>more like broadcast can't be. I might very much be wrong at this, and
>I would appreciate a lot if someone could help to explain.
>
>Thanks,
>
>Yun
>
>On Wed, Jan 25, 2012 at 11:56 AM, Alexis Richardson
> wrote:
>> Eric
>>
>> Understood ;-)
>>
>> I am all in favour of community experimentation.
>>
>> 1:1 messaging is a core use case for RabbitMQ. Unlike regular
>> queueing systems which use queues for shared topics, Rabbit is
>> designed to support very large numbers of short lived queues as well
>> as long lived queues. These can be private or shared. In other
>> words: queues are buffers. ZeroMQ goes one step further and
>> co-locates the consumer with the buffer, and the routing logic with
>> the producer. The cases for which this is useful are discussed on the
>> web site.
>>
>> alexis
>>
>>
>>
>> On Wed, Jan 25, 2012 at 4:49 PM, Eric Windisch
>wrote:
>>> Alexis,
>>>
>>> It is also obvious that the link I provided is a particularly biased
>source,
>>> so it should be taken with a grain of salt. I only mentioned Qpid
>because
>>> Nova just received a driver for it, I didn't know the differences in
>such
>>> detail.
>>>
>>> One of the problems Nova has is that it registers N queues for N
>hosts, with
>>> one host pulling from each queue (1:1). This is why ZeroMQ is a good
>fit,
>>> whereby messages can be sent directly to those hosts. There are also
>a small
>>> (but active) number of N to N queues which remain centralized and for
>which
>>> running Rabbit or Qpid is a good fit.
>>>
>>> It would be interesting exercise to allow the ZeroMQ driver to defer
>back to
>>> the Kombu or Qpid driver for those messages which must remain
>centralized.
>>>
>>> --
>>> Eric Windisch
>>>
>>> On Wednesday, January 25, 2012 at 1:18 AM, Alexis Richardson wrote:
>>>
>>> On Wed, Jan 25, 2012 at 4:46 AM, Eric Windisch
>
>>> wrote:
>>>
>>> Sorry, I had originally sent only to Yun Mao. Sending to list.
>>>
>>> ---
>>>
>>> Rather than attempt to answer this, I defer to the ZeroMQ guide. It
>should
>>> be noted that the designers of AMPQ, iMatix, designed and build
>ZeroMQ.
>>> (RabbitMQ and QUID implement AMQP)
>>>
>>>
>>> Hold on a second there...
>>>
>>> There has been a LOT of muddle and fud ("fuddle"?) around AMQP. Let
>>> me try to clear that up.
>>>
>>> Qpid's default option is AMQP 0-10. While feature-rich, 0-10 is not
>>> widely used and was described by the AMQP chairman as too long and
>>> complicated not long after it was published. See also commentary on
>>> the web, on the subject of its length. Rabbit does not and will not
>>> support this version, and other folks have not implemented it either.
>>>
>>> WHEREAS --
>>>
>>> RabbitMQ implements AMQP 0-91, a 40 page spec. It's the one most
>people use.
>>>
>>> 0-9-1 is the version of AMQP that is used across a very large number
>>> of use cases, that is quite easy to implement. It was created by all
>>> the implementers of AMQP that existed at time of writing including
>>> Rabbit, Redhat, JPMorgan, and of course iMatix. Pieter @iMatix was
>>> the spec editor, and did a fantastic job. 0-9-1 provides
>>> interoperable messaging as witnessed by the large number (100s) of
>>> clients and add-ons that have been created by the community. There
>>> have also been several servers implemented, that all just work with
>>> those clients. For example Kaazing, StormMQ, and OpenAMQ. I believe
>>> that Qpid also supports it, which might be important f
