Re: [Puppet Users] Circular RPM dependencies...

2011-08-25 Thread Jacob Helwig 
On Tue, 23 Aug 2011 12:05:30 +0200, Jakov Sosic wrote:
 
 On 08/22/2011 06:29 PM, Mike Lococo wrote:
 
  This is a long-standing bug, and one that I consider fairly major but
  has been hard to get puppetlabs focused on.
  
  - The circular deps bug was reported 2 years ago.
http://projects.puppetlabs.com/issues/1935
  - A few months later a patch is submitted to batch rpm transactions.
This solves the circular-deps issue and also has significant
performance benefits during puppet runs with many package installs.
http://projects.puppetlabs.com/issues/2198
  
  This comes up periodically and inevitably someone proposes that the
  architecturally correct solution is for RedHat not to employ circular
  deps, which shuts down discussion of a potential solution for another
  3-6 months:
  http://groups.google.com/group/puppet-users/browse_thread/thread/8a083899386909d5/
  
  http://groups.google.com/group/puppet-users/browse_thread/thread/9cbeadad62741b0a/
 
 
 I could incorporate that patch into RPM's... ensure = purged is
 obviously not a good idea after all (although it works).
 
 
 I have another question about packages...
 
 What if I define something like this:
 
 package {'httpd': ensure = absent, }
 package {'mod_ssl': ensure = latest, }
 
 It is obvious that httpd is a dependency of mod_ssl. What will happen in
 this case?!
 
 
 I'm asking because I have a template for all my machines with minimal
 package requirements, and I do that with lots of ensure=absent.
 
 But if some package requires some of the absent pacakges, what happens
 then?
 
 

I would expect Puppet to end up installing and uninstalling packages
every run, since it has no knowledge of the package dependencies.

-- 
Jacob Helwig
,
| Join us for PuppetConf, September 22nd and 23rd in Portland, OR
| http://bit.ly/puppetconfsig
`


signature.asc
Description: Digital signature

Re: [Puppet Users] Circular RPM dependencies...

2011-08-25 Thread Mike Lococo 

On 08/25/2011 01:35 PM, Jacob Helwig wrote:

I would expect Puppet to end up installing and uninstalling packages
every run, since it has no knowledge of the package dependencies.


This kind of thing won't happen unless one manually specifies an 
impossible state, and even then I'm not sure that would be the outcome. 
 Really, the feature being proposed as the solution here is not to 
extend puppet to understand RPM dependencies, but simply to allow 
batched transactions so that two packages can be installed/removed in a 
single command-line run.  There is a patch to implement this feature 
awaiting review (it's been in that state for over 2 years).


As an aside, it's pretty hard to fully understand this issue without 
reading through all the bugs and threads listed in my last email.  Key 
background is:


1) You must understand how the puppet implements the rpm package
   provider.  Specifically, you must know when the rpm command is
   called by puppet, and when the yum command is called by puppet.
   (rpm is used to check install status, yum is used to perform
   installs, rpm is used to perform removals).
2) You must understand how the yum and rpm os commands work, and
   specifically how they each handle dependencies and circular deps
   (yum works to satisfy deps, rpm fails unless all deps are specified
   on a single command-line).
3) You don't actually have to understand much about puppet dependencies,
   and you also don't have to understand much about rpm dependencies.
   The behaviors involved are pretty straightforward, but you can't
   make big assumptions about what dependency cycles mean based on your
   experience in other systems.

Cheers,
Mike Lococo

--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Circular RPM dependencies...

2011-08-25 Thread R P Herrold 

On Thu, 25 Aug 2011, Mike Lococo wrote:


2) You must understand how the yum and rpm os commands work, and
  specifically how they each handle dependencies and circular deps
  (yum works to satisfy deps, rpm fails unless all deps are specified
  on a single command-line).


The use of verb forms projecting overtones of virtues or 
deficiency is overstated here.  You might have well said:


yum will 'do what I mean, not what I said' even if it
would result in a useless installation and
recklessly add and remove whole swaths of packages to
satisfy deps; rpm will carefully limit itself to not
going beyond what is asked to protect your system until
the person seeking the transaction enumerates all deps
in a single command trasaction

Each program is just a mindless tool, with no capacity for for 
beneficence or malice.


Both tools use the RPM database, accessible through 'librpm', 
and of course Puppet might be extended to consult this 
database natively rather than resorting to the present rather 
ugly 'exec' command which ignores this source of information 
[thus forcing a need for knowledge of th4 package state and 
dependency trees of all potentially packages to manage, 
exterior to a specific machine into the equasion]


The program design for Yum is not enabled to proceed to 
perform a transaction without the general explicit 
confirmation with the '-y' command line option [applicable to 
both additions and removals, assuning it has been provided a 
set of 'repositories' with 'closure']


RPM (as shipped by Red Hat) is designed to not add or remove 
packages which have leaf node dependencies beneath it without 
being explicitly instructed to ignore such potential error 
conditions: the --nodeps and --force options.  The issue of 
automated package retrieval and dependency is partially out of 
scope for RPM in the for under which Red hat presently ships 
it, but can be done inder alternative vendor's approaches, 
with a so called 'rpmdb' database


-- Russ herrold

--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-25 Thread Mike Lococo 
Perhaps I'm being dense, but I don't follow how your response relates to 
managing packages that have circular deps in puppet.


The original question was how to delete two packages with circular deps 
using puppet. The correct answer is to ensure purged.  The problem 
with that answer is that it invokes yum multiple times per puppet-run, 
which incurs many catalog-downloads from redhat servers, which results 
in blacklisting by redhat.  There's a patch that would provide a second 
option, batched transactions implemented via rpm.  If accepted, that 
patch would enable a common use-case that is an epic pain to handle in 
puppet right now.  I'm sure there are other ways to handle the problem, 
but this one has a patch pending review which means there's a good 
chance that it could be easily implemented for the next release.  It 
also has performance benefits and no technical downside that has been 
identified.


I have no opinions about whether yum or rpm is superior in the general 
case, or whether it would be better for puppet to interface directly 
with librpm, or whether it's better to allow automatic dependency 
resolution or to require admins to specify package-deps manually.


Cheers,
Mike Lococo


On 08/25/2011 06:29 PM, R P Herrold wrote:

On Thu, 25 Aug 2011, Mike Lococo wrote:


2) You must understand how the yum and rpm os commands work, and
specifically how they each handle dependencies and circular deps
(yum works to satisfy deps, rpm fails unless all deps are specified
on a single command-line).


The use of verb forms projecting overtones of virtues or deficiency is
overstated here. You might have well said:

yum will 'do what I mean, not what I said' even if it
would result in a useless installation and
recklessly add and remove whole swaths of packages to
satisfy deps; rpm will carefully limit itself to not
going beyond what is asked to protect your system until
the person seeking the transaction enumerates all deps
in a single command trasaction

Each program is just a mindless tool, with no capacity for for
beneficence or malice.

Both tools use the RPM database, accessible through 'librpm', and of
course Puppet might be extended to consult this database natively rather
than resorting to the present rather ugly 'exec' command which ignores
this source of information [thus forcing a need for knowledge of th4
package state and dependency trees of all potentially packages to
manage, exterior to a specific machine into the equasion]

The program design for Yum is not enabled to proceed to perform a
transaction without the general explicit confirmation with the '-y'
command line option [applicable to both additions and removals, assuning
it has been provided a set of 'repositories' with 'closure']

RPM (as shipped by Red Hat) is designed to not add or remove packages
which have leaf node dependencies beneath it without being explicitly
instructed to ignore such potential error conditions: the --nodeps and
--force options. The issue of automated package retrieval and dependency
is partially out of scope for RPM in the for under which Red hat
presently ships it, but can be done inder alternative vendor's
approaches, with a so called 'rpmdb' database

-- Russ herrold



--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-23 Thread Jakov Sosic 
On 08/22/2011 06:29 PM, Mike Lococo wrote:

 This is a long-standing bug, and one that I consider fairly major but
 has been hard to get puppetlabs focused on.
 
 - The circular deps bug was reported 2 years ago.
   http://projects.puppetlabs.com/issues/1935
 - A few months later a patch is submitted to batch rpm transactions.
   This solves the circular-deps issue and also has significant
   performance benefits during puppet runs with many package installs.
   http://projects.puppetlabs.com/issues/2198
 
 This comes up periodically and inevitably someone proposes that the
 architecturally correct solution is for RedHat not to employ circular
 deps, which shuts down discussion of a potential solution for another
 3-6 months:
 http://groups.google.com/group/puppet-users/browse_thread/thread/8a083899386909d5/
 
 http://groups.google.com/group/puppet-users/browse_thread/thread/9cbeadad62741b0a/


I could incorporate that patch into RPM's... ensure = purged is
obviously not a good idea after all (although it works).


I have another question about packages...

What if I define something like this:

package {'httpd': ensure = absent, }
package {'mod_ssl': ensure = latest, }

It is obvious that httpd is a dependency of mod_ssl. What will happen in
this case?!


I'm asking because I have a template for all my machines with minimal
package requirements, and I do that with lots of ensure=absent.

But if some package requires some of the absent pacakges, what happens
then?


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-22 Thread Mike Lococo 

On 08/18/2011 06:56 PM, Jakov Sosic wrote:

I encountered a following problem, if I state for example:

package{aspell-en: ensure =  absent, }
package{aspell:ensure =  absent, }

I get error in logs because of dependencies. It seems that puppet uses
'yum' for installation on CentOS and 'rpm -e' for removal of packages,
so it cannot handle this kind of dependencies:

# rpm -e --test aspell-en
error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64
# rpm -e --test aspell
error: Failed dependencies:
aspell= 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64


This is a long-standing bug, and one that I consider fairly major but 
has been hard to get puppetlabs focused on.


- The circular deps bug was reported 2 years ago.
  http://projects.puppetlabs.com/issues/1935
- A few months later a patch is submitted to batch rpm transactions.
  This solves the circular-deps issue and also has significant
  performance benefits during puppet runs with many package installs.
  http://projects.puppetlabs.com/issues/2198

This comes up periodically and inevitably someone proposes that the 
architecturally correct solution is for RedHat not to employ circular 
deps, which shuts down discussion of a potential solution for another 
3-6 months:

http://groups.google.com/group/puppet-users/browse_thread/thread/8a083899386909d5/
http://groups.google.com/group/puppet-users/browse_thread/thread/9cbeadad62741b0a/

Denmat wrote:

and see if you can order it. You might also get luck from 'before = '


Ordering doesn't matter, the dependencies involved are rpm-deps, not 
puppet-deps. In order to succeed, the action has to be atomically 
performed in a single rpm call (which requires #2198 to land and enable 
batch transactions), or it has to be performed with a 
rpm-dependency-aware tool like yum (but that has it's own problems 
discussed below).



Jonathan Gazeley wrote:

Is it possible to force use of the yum backend for package
removals  like this?

package { ['aspell', 'aspell-en']:
  ensure = absent,
  provider = 'yum',
}


Jakov Sosic wrote:

No, it uses rpm and that's it. And it cycles through the list item
by item. If it would only run rpm -e package1 package2, but not
even that

devzero2000 wrote:

The right answer is

ensure = purged as this tell puppet to use yum -y (and not rpm) in
the yum provider.


As devzero notes, ensuring purged will force the yum-provider to use 
the yum-command instead of the rpm-command for package removals, but 
that's a good way to get a box blacklisted by Redhat for pinging their 
update-servers too often.  Yum will be run for every single package that 
you manage this way, and will update its package catalog every time, 
resulting in what looks like misconfiguration or dos to RedHat.


In my opinion, this bug makes the yum provider very close to useless. 
I'm watching bug #2198 closely to see if it ever lands, but it seems to 
be very low on the priority list given that there's been a patch sitting 
unmerged for almost two years.


Folks who work around this issue do so with execs, but I think more 
often people just give up and let unneeded packages build up like cruft.


Cheers,
Mike Lococo

--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-20 Thread devzero2000 
On Fri, Aug 19, 2011 at 12:56 AM, Jakov Sosic jso...@srce.hr wrote:

 Hi.

 I encountered a following problem, if I state for example:

 package{aspell-en: ensure = absent, }
 package{aspell:ensure = absent, }

 I get error in logs because of dependencies. It seems that puppet uses
 'yum' for installation on CentOS and 'rpm -e' for removal of packages,
 so it cannot handle this kind of dependencies:

 # rpm -e --test aspell-en
 error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64
 # rpm -e --test aspell
 error: Failed dependencies:
aspell = 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64


 Do you have any idea or advice how to solve this kind of problems?

 I've searched the list and found:

 http://www.mail-archive.com/puppet-users@googlegroups.com/msg03702.html

 and consensus is that RPM circular dependencies have nothing to do with
 puppet. OK, but any advice on how to circumvent this issue as elegant
 as possible?

 The right answer is

*ensure = purged* as this tell puppet to  use yum -y (and not rpm) in the
yum provider.

But yes the docu could be better perhaps.

hth

 --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to
 puppet-users+unsubscr...@googlegroups.com.
 /For more options, visit this group at
 http://groups.google.com/group/puppet-users?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Circular RPM dependencies...

2011-08-19 Thread Jakov Sosic 
Hi.

I encountered a following problem, if I state for example:

package{aspell-en: ensure = absent, }
package{aspell:ensure = absent, }

I get error in logs because of dependencies. It seems that puppet uses
'yum' for installation on CentOS and 'rpm -e' for removal of packages,
so it cannot handle this kind of dependencies:

# rpm -e --test aspell-en
error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64
# rpm -e --test aspell
error: Failed dependencies:
aspell = 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64


Do you have any idea or advice how to solve this kind of problems?

I've searched the list and found:

http://www.mail-archive.com/puppet-users@googlegroups.com/msg03702.html

and consensus is that RPM circular dependencies have nothing to do with
puppet. OK, but any advice on how to circumvent this issue as elegant
as possible?

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-19 Thread Denmat 
Hi,

Haven't tested but can you try:
 package{aspell-en: ensure = absent, } -
 package{aspell:ensure = absent, }


and see if you can order it. You might also get luck from 'before = '

Cheers,
Den
On 19/08/2011, at 8:56, Jakov Sosic jso...@srce.hr wrote:

 Hi.
 
 I encountered a following problem, if I state for example:
 
 package{aspell-en: ensure = absent, }
 package{aspell:ensure = absent, }
 
 I get error in logs because of dependencies. It seems that puppet uses
 'yum' for installation on CentOS and 'rpm -e' for removal of packages,
 so it cannot handle this kind of dependencies:
 
 # rpm -e --test aspell-en
 error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64
 # rpm -e --test aspell
 error: Failed dependencies:
aspell = 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64
 
 
 Do you have any idea or advice how to solve this kind of problems?
 
 I've searched the list and found:
 
 http://www.mail-archive.com/puppet-users@googlegroups.com/msg03702.html
 
 and consensus is that RPM circular dependencies have nothing to do with
 puppet. OK, but any advice on how to circumvent this issue as elegant
 as possible?
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to 
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/puppet-users?hl=en.
 

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-19 Thread Jonathan Gazeley 
Is it possible to force use of the yum backend for package removals like 
this?


package { ['aspell', 'aspell-en']:
  ensure = absent,
  provider = 'yum',
}

I'm not near a puppet machine to test this, but it's probably worth a shot.

Cheers,
Jonathan



On 18/08/11 23:56, Jakov Sosic wrote:

Hi.

I encountered a following problem, if I state for example:

package{aspell-en: ensure =  absent, }
package{aspell:ensure =  absent, }

I get error in logs because of dependencies. It seems that puppet uses
'yum' for installation on CentOS and 'rpm -e' for removal of packages,
so it cannot handle this kind of dependencies:

# rpm -e --test aspell-en
error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64
# rpm -e --test aspell
error: Failed dependencies:
aspell= 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64


Do you have any idea or advice how to solve this kind of problems?

I've searched the list and found:

http://www.mail-archive.com/puppet-users@googlegroups.com/msg03702.html

and consensus is that RPM circular dependencies have nothing to do with
puppet. OK, but any advice on how to circumvent this issue as elegant
as possible?



--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Circular RPM dependencies...

2011-08-19 Thread Jakov Sosic 
On 08/19/2011 12:56 PM, Jonathan Gazeley wrote:
 Is it possible to force use of the yum backend for package removals like
 this?
 
 package { ['aspell', 'aspell-en']:
   ensure = absent,
   provider = 'yum',
 }
 
 I'm not near a puppet machine to test this, but it's probably worth a shot.


Fri Aug 19 18:55:21 +0200 2011
/Stage[main]/Packages/Package[aspell-en]/ensure (err): change from
6.0-2.1 to absent failed: Execution of '/bin/rpm -e
aspell-en-6.0-2.1.x86_64' returned 1: error: Failed dependencies:
aspell-en is needed by (installed) aspell-0.60.3-7.1.x86_64


Fri Aug 19 18:56:00 +0200 2011
/Stage[main]/Packages/Package[aspell]/ensure (err): change from
0.60.3-7.1 to absent failed: Execution of '/bin/rpm -e
aspell-0.60.3-7.1.x86_64' returned 1: error: Failed dependencies:
aspell = 12:0.60 is needed by (installed) aspell-en-6.0-2.1.x86_64


No, it uses rpm and that's it. And it cycles through the list item by
item. If it would only run rpm -e package1 package2, but not even that :(

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.