Re: [ClusterLabs] How to implement a fencing agent

[Jan Pokorný]

On 09/08/18 14:10 -0500, Ryan Thomas wrote:
> I did some more investigation and was able to answer two of my questions:
> 
> First, why did "pcs stonith list" not show my fence_foo agent? pcs runs the
> meta-data action on the agent to get the description.  Since my fence_foo
> agent wasn't implemented, this would fail, and pcs would not display it.
> If I ran "pcs stonith list --nodesc" then the agent would be shown.

Good work figuring this out.

> Second, how should I import the fencing.py lib?  I noticed that all the
> exiting fence_agents did explicitly add the path the the fencing lib, by
> sys.path.append(PATH).
> 
> My only remaining question:  Is "yum install fence-agents-common.x86_64"
> the appropriate way to install the fencing.py library?

That really depends on which system/distribution you are targeting.
If for instance it's for Enterprise Linux incl. CentOS family or
Fedora + derivatives, then the proper answer is:

  "fence-agents-common" is relatively stable implementation detail
  but watch for any sort of rename (and/or file move since you can
  also "yum install /usr/share/fence/fencing.py")

And you've actually revealed a pitiful packaging deficiencies
-- please omit the architecture specifier (.x86_64) when referring
to that package as it will eventually become "noarch" and your
fully qualified identification will have stopped working then:
https://src.fedoraproject.org/rpms/fence-agents/pull-request/4

> I suspect there may not be any best practice in this area.

I think you are first to explore this path (at least so openly)
since the common practice used to be that even 3rd party contributed,
shared agents ended up as a direct part of fence-agents project in
the past.  Which of course stipulates you would be OK with opensourcing
your final work and licensing that under the same terms (variations
likely negotiable).

Hope this helps.

-- 
Nazdar,
Jan (Poki)


pgpLB1_dYGIgg.pgp
Description: PGP signature
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] How to implement a fencing agent

[Ryan Thomas]

I did some more investigation and was able to answer two of my questions:

First, why did "pcs stonith list" not show my fence_foo agent? pcs runs the
meta-data action on the agent to get the description.  Since my fence_foo
agent wasn't implemented, this would fail, and pcs would not display it.
If I ran "pcs stonith list --nodesc" then the agent would be shown.

Second, how should I import the fencing.py lib?  I noticed that all the
exiting fence_agents did explicitly add the path the the fencing lib, by
sys.path.append(PATH).

My only remaining question:  Is "yum install fence-agents-common.x86_64"
the appropriate way to install the fencing.py library?  I suspect there may
not be any best practice in this area.

Thanks,
Ryan

On Thu, Aug 9, 2018 at 12:24 PM, Ryan Thomas 
wrote:

> Thanks for the advice and information.
>
> >> 1. The documentation encourages the use of the python fencing library.
>
> >>How does one install this?> Fencing library is basically this file
> (sans the doubled extension):
> > https://github.com/ClusterLabs/fence-agents/blob/v4.2.1/lib/
> fencing.py.py
> 
> > that shall be assurably in predefined/run-time-mangled
> > search path for Python modules (sys.path), so that you can then
> > "from fencing import ...".  You can follow the native agents for
> > how it is to be used.
>
> My system did not have a fencing.py installed. However, I found that after
> running "yum install fence-agents-common.x86_64", fencing.py could be found
> in /usr/share/fence as you said.  Is "yum install
> fence-agents-common.x86_64" the appropriate way to install the fencing.py
> library? It seems like fencing.py being installed was a side-effect, rather
> than explictly being installed. Also, although the fencing.py did appear,
> it is not in the python path.  So "import fencing" fails with "ImportError:
> No module named fencing".
>   I can of course add /usr/share/fence to the python path.
>
> >> 2. How to a “register” the agent so it shows up in the “pcs stonith
> >>list” output, and the “stonith_admin –list-installed” output?
> >>So, I can use my new agent?
> > Living convention for this is that whatever in /usr/sbin
> >  (symbolically in pacemaker context: RH_STONITH_DIR) named in way
> >  that conforms "fence_*" glob pattern (symbolically RH_STONITH_PREFIX)
> >  will qualify automatically.  Obviously, it shall be marked executable.
>
> Following your advice, I ran "echo 'import fencing' > /usr/sbin/fence_foo"
> and "chmod +x /usr/sbin/fence_foo".  After doing this the "stonith_admin
> --list-installed" did list fence_foo.  However, the "pcs stonith list" >
> did not, so there must be something more that needs to be done for the
> fence agent to be found by pcs.
>
> Thanks,
> Ryan
>
> On Thu, Aug 9, 2018 at 8:09 AM, Jan Pokorný  wrote:
>
>> On 09/08/18 07:59 +0200, Ulrich Windl wrote:
>>  Ryan Thomas  schrieb am 08.08.2018 um
>> 23:26 in
>> > Nachricht
>> > :
>> >> I’m attempting to implement a fencing agent.
>> >>
>> >> The ClusterLabs/fence-agent github repo has some helpful information
>> >> including fence-agents/doc/FenceAgentAPI.md, but I haven’t found the
>> >> answers to a few basic questions.
>> >
>> > Personally I think a specification should not recommend any library to
>> use,
>> > but state in simple and precise words what the thing (=fencing agent)
>> > is expected to do.
>>
>> +1 for a complete decoupling of API specification and referential
>> library, it's pretty possible there are still some undocumented bits
>> relied upon while only hardwired into the fencing library.  Note that
>> pacemaker adds a thin configuration surface and decision logic on top
>> of the base line set forth in the mentioned document.
>>
>> Also, mentioning Python below, there are certainly reasons why it can
>> be more blessed than say shell scripting, though I stay vocal that
>> transition from compiled fence agents was rather a regress than
>> progress (not to speak about original Linux HA project going as far as
>> to have ready-to-use agents preloaded in memory to achieve
>> unprecedented robustness -- please correct me if I get these gory
>> details about the past wrong).
>>
>> The problem with Python is that even for a trivial proxying of the
>> operations towards 3rd party tooling (e.g. ipmitool), the costs of
>> bringing the run-time up alone is unexpectedly I/O intensive (trying
>> to open plethora files, preinitializing some objects that won't be
>> used, anyway, etc.), with excessive complexity of the task/memory
>> footprint proportion, and currently also carrying a risk of being
>> affected/delayed with user customization of the Python deployment
>> that gets loaded automatically unless suppressed [1].  That's not
>> very aligned with the goal of preventing dangerous access to shared
>> resources in timely and reliable fashion.
>>
>> Perhaps a reasonable tradeoff at this point would be to consider
>> 

Re: [ClusterLabs] How to implement a fencing agent

[Ryan Thomas]

Thanks for the advice and information.

>> 1. The documentation encourages the use of the python fencing library.
>>How does one install this?> Fencing library is basically this file
(sans the doubled extension):
> https://github.com/ClusterLabs/fence-agents/blob/v4.2.1/lib/fencing.py.py

> that shall be assurably in predefined/run-time-mangled
> search path for Python modules (sys.path), so that you can then
> "from fencing import ...".  You can follow the native agents for
> how it is to be used.

My system did not have a fencing.py installed. However, I found that after
running "yum install fence-agents-common.x86_64", fencing.py could be found
in /usr/share/fence as you said.  Is "yum install
fence-agents-common.x86_64" the appropriate way to install the fencing.py
library? It seems like fencing.py being installed was a side-effect, rather
than explictly being installed. Also, although the fencing.py did appear,
it is not in the python path.  So "import fencing" fails with "ImportError:
No module named fencing".
  I can of course add /usr/share/fence to the python path.

>> 2. How to a “register” the agent so it shows up in the “pcs stonith
>>list” output, and the “stonith_admin –list-installed” output?
>>So, I can use my new agent?
> Living convention for this is that whatever in /usr/sbin
>  (symbolically in pacemaker context: RH_STONITH_DIR) named in way
>  that conforms "fence_*" glob pattern (symbolically RH_STONITH_PREFIX)
>  will qualify automatically.  Obviously, it shall be marked executable.

Following your advice, I ran "echo 'import fencing' > /usr/sbin/fence_foo"
and "chmod +x /usr/sbin/fence_foo".  After doing this the "stonith_admin
--list-installed" did list fence_foo.  However, the "pcs stonith list" >
did not, so there must be something more that needs to be done for the
fence agent to be found by pcs.

Thanks,
Ryan

On Thu, Aug 9, 2018 at 8:09 AM, Jan Pokorný  wrote:

> On 09/08/18 07:59 +0200, Ulrich Windl wrote:
>  Ryan Thomas  schrieb am 08.08.2018 um
> 23:26 in
> > Nachricht
> > :
> >> I’m attempting to implement a fencing agent.
> >>
> >> The ClusterLabs/fence-agent github repo has some helpful information
> >> including fence-agents/doc/FenceAgentAPI.md, but I haven’t found the
> >> answers to a few basic questions.
> >
> > Personally I think a specification should not recommend any library to
> use,
> > but state in simple and precise words what the thing (=fencing agent)
> > is expected to do.
>
> +1 for a complete decoupling of API specification and referential
> library, it's pretty possible there are still some undocumented bits
> relied upon while only hardwired into the fencing library.  Note that
> pacemaker adds a thin configuration surface and decision logic on top
> of the base line set forth in the mentioned document.
>
> Also, mentioning Python below, there are certainly reasons why it can
> be more blessed than say shell scripting, though I stay vocal that
> transition from compiled fence agents was rather a regress than
> progress (not to speak about original Linux HA project going as far as
> to have ready-to-use agents preloaded in memory to achieve
> unprecedented robustness -- please correct me if I get these gory
> details about the past wrong).
>
> The problem with Python is that even for a trivial proxying of the
> operations towards 3rd party tooling (e.g. ipmitool), the costs of
> bringing the run-time up alone is unexpectedly I/O intensive (trying
> to open plethora files, preinitializing some objects that won't be
> used, anyway, etc.), with excessive complexity of the task/memory
> footprint proportion, and currently also carrying a risk of being
> affected/delayed with user customization of the Python deployment
> that gets loaded automatically unless suppressed [1].  That's not
> very aligned with the goal of preventing dangerous access to shared
> resources in timely and reliable fashion.
>
> Perhaps a reasonable tradeoff at this point would be to consider
> MicroPython, alternative lightweight implementation with a trimmed
> standard library: https://micropython.org/.  As a first step, fencing
> library would likely require some touches for compatibility, though.
>
> > Inspecting the source of SBD I found out that there is too much
> > undocumented magic implemented.
>
> >> 1. The documentation encourages the use of the python fencing library.
> >>How does one install this?
>
> Fencing library is basically this file (sans the doubled extension):
> https://github.com/ClusterLabs/fence-agents/blob/v4.2.1/lib/fencing.py.py
> that shall be assurably in predefined/run-time-mangled
> search path for Python modules (sys.path), so that you can then
> "from fencing import ...".  You can follow the native agents for
> how it is to be used.
>
> For instance, EL family of distros will provide that file as
> /usr/share/fence/fencing.py by the means of 

Re: [ClusterLabs] How to implement a fencing agent

[Jan Pokorný]

On 09/08/18 07:59 +0200, Ulrich Windl wrote:
 Ryan Thomas  schrieb am 08.08.2018 um 23:26 in
> Nachricht
> :
>> I’m attempting to implement a fencing agent.
>> 
>> The ClusterLabs/fence-agent github repo has some helpful information
>> including fence-agents/doc/FenceAgentAPI.md, but I haven’t found the
>> answers to a few basic questions.
> 
> Personally I think a specification should not recommend any library to use,
> but state in simple and precise words what the thing (=fencing agent)
> is expected to do.

+1 for a complete decoupling of API specification and referential
library, it's pretty possible there are still some undocumented bits
relied upon while only hardwired into the fencing library.  Note that
pacemaker adds a thin configuration surface and decision logic on top
of the base line set forth in the mentioned document.

Also, mentioning Python below, there are certainly reasons why it can
be more blessed than say shell scripting, though I stay vocal that
transition from compiled fence agents was rather a regress than
progress (not to speak about original Linux HA project going as far as
to have ready-to-use agents preloaded in memory to achieve
unprecedented robustness -- please correct me if I get these gory
details about the past wrong).

The problem with Python is that even for a trivial proxying of the
operations towards 3rd party tooling (e.g. ipmitool), the costs of
bringing the run-time up alone is unexpectedly I/O intensive (trying
to open plethora files, preinitializing some objects that won't be
used, anyway, etc.), with excessive complexity of the task/memory
footprint proportion, and currently also carrying a risk of being
affected/delayed with user customization of the Python deployment
that gets loaded automatically unless suppressed [1].  That's not
very aligned with the goal of preventing dangerous access to shared
resources in timely and reliable fashion.

Perhaps a reasonable tradeoff at this point would be to consider
MicroPython, alternative lightweight implementation with a trimmed
standard library: https://micropython.org/.  As a first step, fencing
library would likely require some touches for compatibility, though.

> Inspecting the source of SBD I found out that there is too much
> undocumented magic implemented.

>> 1. The documentation encourages the use of the python fencing library.  
>>How does one install this?

Fencing library is basically this file (sans the doubled extension):
https://github.com/ClusterLabs/fence-agents/blob/v4.2.1/lib/fencing.py.py
that shall be assurably in predefined/run-time-mangled
search path for Python modules (sys.path), so that you can then
"from fencing import ...".  You can follow the native agents for
how it is to be used.

For instance, EL family of distros will provide that file as
/usr/share/fence/fencing.py by the means of fence-agents-common
package.

>> 2. How to a “register” the agent so it shows up in the “pcs stonith
>>list” output, and the “stonith_admin –list-installed” output? 
>>So, I can use my new agent?

Living convention for this is that whatever in /usr/sbin
(symbolically in pacemaker context: RH_STONITH_DIR) named in way
that conforms "fence_*" glob pattern (symbolically RH_STONITH_PREFIX)
will qualify automatically.  Obviously, it shall be marked executable.


Hope this helps.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1600893

-- 
Nazdar,
Jan (Poki)


pgpy5YICLU0_K.pgp
Description: PGP signature
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org