[Puppet Users] Re: Adobe Reader Recipe
On 11/11/08 8:38, Simon J Mudd wrote: > [EMAIL PROTECTED] (RijilV) writes: > >> On Fri, Nov 7, 2008 at 2:47 PM, Simon J Mudd <[EMAIL PROTECTED]> wrote: > [snipped discussion] > cfengine does EXACTLY the same thing but you can also copy over files > which are large. So why can't puppet? I you want the technical reason here it is: Up to (and includingd) version 0.24, puppet uses XMLRPC over HTTP/SSL for its communications, including file serving. XMLRPC is a remote procedure call protocol that marshalls/unmarshalls native types to XMLRPC types that are transferred in textual form. That means a bunch of bytes (ie a byte string representing a file) is transferred as a text string with non-ASCII characters being escaped ala: AB%00CD%20... That means that each non 32-127 byte takes 3 bytes instead of one. Your 40MB file was certainly transferred as a big 120MB string. That's why it's inefficient, and clearly not O(n). The other inefficiency is that Puppet has to read the whole file in RAM to perform the escaping and XMLRPC transformation, because XMLRPC is not a "streaming" protocol. Now, starting with the awaited and upcoming 0.25, Luke has moved puppet from XMLRPC to REST where binary blobs are transferred as binary blobs, so you won't have the marshalling penalty. I don't know for the streaming thing. As others have already stated, Puppet wasn't designed to transfer files, it was designed as a configuration engine. The ability to transfer files comes "free" with the system, but is inefficient with large file (OK inefficient with files atm). The good point is that puppet is still a young product, and is maturing days after days. Luke and the other developpers are pretty aware of the current issues and are worked on them :-) Hope that helps, -- Brice Figureau Days of Wonder http://www.daysofwonder.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
Hi > Puppet _can_ copy over large files. However, one of the artefacts of > the current implementation is that copying large files across is painfully > slow. and the main reason therefore is XMLRPC which puppet is currently using. and with xmlrpc everything have to be escaped which makes the whole thing painfully slow. see list archive for more discussions about that. > The original idea was that most people would leave the installation of > binary packages to the package manager and copy over small text files. > > Performance tuning for copying large files across is work in progress, and > is part of a major refactoring of the file transfer protocol in use. > > I believe this is due in the current stable release (or the next). switchover to REST is the big thing for 0.25.0 > We are working around this by converting binary packages into RPMs > and deploying those directly. and with that you can even use the package type to manage these packages. and you want to manage packages don't you? cheers pete --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
On Tue, Nov 11, 2008 at 08:38:25AM +0100, Simon J Mudd wrote: > > Therefore I expect it to work. > > cfengine does EXACTLY the same thing but you can also copy over files > which are large. So why can't puppet? > Puppet _can_ copy over large files. However, one of the artefacts of the current implementation is that copying large files across is painfully slow. The original idea was that most people would leave the installation of binary packages to the package manager and copy over small text files. Performance tuning for copying large files across is work in progress, and is part of a major refactoring of the file transfer protocol in use. I believe this is due in the current stable release (or the next). We are working around this by converting binary packages into RPMs and deploying those directly. Devdas Bhagat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
[EMAIL PROTECTED] (RijilV) writes:
> On Fri, Nov 7, 2008 at 2:47 PM, Simon J Mudd <[EMAIL PROTECTED]> wrote:
>
>
>[EMAIL PROTECTED] ("Evan Hisey") writes:
>
>
> > Depending on your puppet version, yes. It is a known
> issue/annoyance
> > that puppet is lousy at transferring large directories of files.
>
>
>
>
> Lousy? Really awful. Makes it unworkable especially if you have a lot
> of servers to copy to. We've had to adjust the install script and pull
> from an http server first and then run the install script but that's
> a pain. cfengine is _MUCH_ better at this.
>
>
>> I under stand that a change to a ReST desgin for the file
> transfer
> > is coming that should resolve this.
>
>
>
>
> Well I wait for that change as the current design is just unworkable
> for large files. And 40MB is not really that large.
>
>
>
>
> Eh, is puppet a replacement for ftp/http/rsync or is it configuration
> management? The fact you can easily ship around small
> files is key, but large packages are IMHO better suited for more traditional
> transfer protocols. The authors of apt/yum/portage
> didn't try to build a new way of pushing files around and I think its a
> mistake for puppet to really aim at replacing already
> existing file transfer protocols. It would be cool if puppet supported more
> URIs though.
I think you miss the point. Puppet DOES provide facilities for
securely copying files from a server to a number of clients using part
of the current client-server communication that it uses for other
tasks. cfengine does the same thing. That's great.
When you start working with a large number of servers over a large
number of networks perhaps geographically located in different
locations the NETWORK may not be as open as you have on a simple
single vlan. That's because there are extra layers of security in
place which can be a nuisance.
The network department thus LIKE the fact that puppet provides a
single protocal for communication between client and server. If puppet
provides that facility then it should work. I have not seen anywhere
in the puppet documentation:
"DO NOT USE THE file: type with files over MB"
or
"THIS IS VERY SLOW WITH FILES OVER MB"
Therefore I expect it to work.
cfengine does EXACTLY the same thing but you can also copy over files
which are large. So why can't puppet?
When you talk about configuration management then I think of this as
"server configuration management" and if part of that is to copy a
file over, and custom run an installer to get the software installed,
followed by other tasks to get it running then that for me is
configuration management. The simpler tasks of installing rpms and
adjusting their configuration files solves most problems but not all
of them, and the added flexibility to be able to easily build your own
extras as site specific requirements demand is a must.
In the end I've had to adjust my install script to copy the binary by
http, and then get the site-specific custom installer to download the
binary by http and run it appropriately. That splits the 2 files into
separate locations which have to be "managed" separately, and is
therefore more work, so I've made a workaround, but that's only because
current behaviour is not usable as advertised.
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
On Fri, Nov 7, 2008 at 2:47 PM, Simon J Mudd <[EMAIL PROTECTED]> wrote:
>
> [EMAIL PROTECTED] ("Evan Hisey") writes:
> > Depending on your puppet version, yes. It is a known issue/annoyance
> > that puppet is lousy at transferring large directories of files.
>
> Lousy? Really awful. Makes it unworkable especially if you have a lot
> of servers to copy to. We've had to adjust the install script and pull
> from an http server first and then run the install script but that's
> a pain. cfengine is _MUCH_ better at this.
>
> > I under stand that a change to a ReST desgin for the file transfer
> > is coming that should resolve this.
>
> Well I wait for that change as the current design is just unworkable
> for large files. And 40MB is not really that large.
Eh, is puppet a replacement for ftp/http/rsync or is it configuration
management? The fact you can easily ship around small files is key, but
large packages are IMHO better suited for more traditional transfer
protocols. The authors of apt/yum/portage didn't try to build a new way of
pushing files around and I think its a mistake for puppet to really aim at
replacing already existing file transfer protocols. It would be cool if
puppet supported more URIs though.
.r'
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
[EMAIL PROTECTED] ("Evan Hisey") writes:
...
>>>
>>> I get the feeling that I don't have to create ar-8.1.3-1.install.sh
>>> script and that the whole thing can be done in puppet or facteur.
>>
>> We've started doing this with another non-rpm package on
>> Linux. however I was startled to see at least on our setup that it
>> took several minutes to copy the directory including the binary and
>> install script over. Total space about 40MB. Very slow.
>>
>> Is this the expected speed?
>>
>> Our network is 100Mb and puppet is (custom built):
>>
>> # rpm -q puppet
>> puppet-0.24.6-1ce4.1
>>
>> running x86_64 CentOS 4.
>>
> Depending on your puppet version, yes. It is a known issue/annoyance
> that puppet is lousy at transferring large directories of files.
Lousy? Really awful. Makes it unworkable especially if you have a lot
of servers to copy to. We've had to adjust the install script and pull
from an http server first and then run the install script but that's
a pain. cfengine is _MUCH_ better at this.
> I under stand that a change to a ReST desgin for the file transfer
> is coming that should resolve this.
Well I wait for that change as the current design is just unworkable
for large files. And 40MB is not really that large.
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
>> >> We've started doing this with another non-rpm package on >> Linux. however I was startled to see at least on our setup that it >> took several minutes to copy the directory including the binary and >> install script over. Total space about 40MB. Very slow. >> >> Is this the expected speed? >> >> Our network is 100Mb and puppet is (custom built): >> >> # rpm -q puppet >> puppet-0.24.6-1ce4.1 >> >> running x86_64 CentOS 4. >> > Depending on your puppet version, yes. It is a known issue/annoyance > that puppet is lousy at transferring large directories of files. I > under stand that a change to a ReST desgin for the file transfer is > coming that should resolve this. > Just saw Luke's roadmap it looks like the ReST redesign is the target for the 0.25 release of puppet. Evan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
On Thu, Nov 6, 2008 at 1:21 PM, Simon J Mudd <[EMAIL PROTECTED]> wrote:
>
> [EMAIL PROTECTED] (Loobster) writes:
>
>> I was thinking of creating a little script (ar-8.1.3-1.install.sh) to
>> do that and also figure out which version of reader is installed.
>>
>> -bash-3.00$ /opt/Adobe/Reader8/bin/acroread -v
>> 8.1.2
>>
>> So far, this is what I came up with on the puppet side of things:
>>
>> # Omited class and case statements.
>>
>> # create staging directory
>> file { "/sysprov/apps/adobe":
>> owner => puppet,
>> group => puppet,
>> ensure => directory
>> }
>>
>> # copy files the staging area
>> file { "/sysprov/apps/adobe/AdobeReader_enu-8.1.3-1.sparc.tar.gz":
>> owner => puppet,
>> group => puppet,
>> source => "puppet://puppet/apps/adobe/
>> AdobeReader_enu-8.1.3-1.sparc.tar.gz",
>> }
>>
>> file { "/sysprov/apps/adobe/bar-ar-8.1.3-1.install.sh":
>> owner => puppet,
>> group => puppet,
>> source => "puppet://puppet/apps/adobe/adobe/ar-8.1.3-1.install.sh",
>> }
>>
>> # Run the installer
>> exec { "ar-8.1.3-1.install.sh":
>> command => "/sysprov/apps/adobe/ar-8.1.3-1.install.sh",
>> require => File[ "/sysprov/apps/adobe/
>> AdobeReader_enu-8.1.3-1.sparc.tar.gz" ]
>> require => File[ "/sysprov/apps/adobe/ar-8.1.3-1.install.sh" ]
>> refreshonly => true,
>> }
>>
>> I get the feeling that I don't have to create ar-8.1.3-1.install.sh
>> script and that the whole thing can be done in puppet or facteur.
>
> We've started doing this with another non-rpm package on
> Linux. however I was startled to see at least on our setup that it
> took several minutes to copy the directory including the binary and
> install script over. Total space about 40MB. Very slow.
>
> Is this the expected speed?
>
> Our network is 100Mb and puppet is (custom built):
>
> # rpm -q puppet
> puppet-0.24.6-1ce4.1
>
> running x86_64 CentOS 4.
>
Depending on your puppet version, yes. It is a known issue/annoyance
that puppet is lousy at transferring large directories of files. I
under stand that a change to a ReST desgin for the file transfer is
coming that should resolve this.
Evan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
[EMAIL PROTECTED] (Loobster) writes:
> I was thinking of creating a little script (ar-8.1.3-1.install.sh) to
> do that and also figure out which version of reader is installed.
>
> -bash-3.00$ /opt/Adobe/Reader8/bin/acroread -v
> 8.1.2
>
> So far, this is what I came up with on the puppet side of things:
>
> # Omited class and case statements.
>
> # create staging directory
> file { "/sysprov/apps/adobe":
> owner => puppet,
> group => puppet,
> ensure => directory
> }
>
> # copy files the staging area
> file { "/sysprov/apps/adobe/AdobeReader_enu-8.1.3-1.sparc.tar.gz":
> owner => puppet,
> group => puppet,
> source => "puppet://puppet/apps/adobe/
> AdobeReader_enu-8.1.3-1.sparc.tar.gz",
> }
>
> file { "/sysprov/apps/adobe/bar-ar-8.1.3-1.install.sh":
> owner => puppet,
> group => puppet,
> source => "puppet://puppet/apps/adobe/adobe/ar-8.1.3-1.install.sh",
> }
>
> # Run the installer
> exec { "ar-8.1.3-1.install.sh":
> command => "/sysprov/apps/adobe/ar-8.1.3-1.install.sh",
> require => File[ "/sysprov/apps/adobe/
> AdobeReader_enu-8.1.3-1.sparc.tar.gz" ]
> require => File[ "/sysprov/apps/adobe/ar-8.1.3-1.install.sh" ]
> refreshonly => true,
> }
>
> I get the feeling that I don't have to create ar-8.1.3-1.install.sh
> script and that the whole thing can be done in puppet or facteur.
We've started doing this with another non-rpm package on
Linux. however I was startled to see at least on our setup that it
took several minutes to copy the directory including the binary and
install script over. Total space about 40MB. Very slow.
Is this the expected speed?
Our network is 100Mb and puppet is (custom built):
# rpm -q puppet
puppet-0.24.6-1ce4.1
running x86_64 CentOS 4.
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
On 11/6/2008 10:53 AM, Loobster wrote: > Thanks for the reply! We are a Solaris shop. > > As far as I know, there is no Solaris package for Adobe Reader. One possible route: install Reader by hand somewhere. Make your own Solaris package out of what it installs [1] (I've documented how to do it Blastwave-style, but the package creation process should be identical). Put your new Reader package in a location accessible by either NFS (or http, for newer Solaris versions). Install it with Puppet's native package type [2]. [1] http://blogs.cae.tntech.edu/mwr/2008/05/21/making-solaris-packages-from-commercial-software/ [2] http://reductivelabs.com/trac/puppet/wiki/TypeReference#package -- Mike Renfro / R&D Engineer, Center for Manufacturing Research, 931 372-3601 / Tennessee Technological University --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
Thanks for the reply! We are a Solaris shop.
As far as I know, there is no Solaris package for Adobe Reader.
You have to run an installer.
In order to run the installer, you need to unzip/untar the archive.
I was thinking of creating a little script (ar-8.1.3-1.install.sh) to
do that and also figure out which version of reader is installed.
-bash-3.00$ /opt/Adobe/Reader8/bin/acroread -v
8.1.2
So far, this is what I came up with on the puppet side of things:
# Omited class and case statements.
# create staging directory
file { "/sysprov/apps/adobe":
owner => puppet,
group => puppet,
ensure => directory
}
# copy files the staging area
file { "/sysprov/apps/adobe/AdobeReader_enu-8.1.3-1.sparc.tar.gz":
owner => puppet,
group => puppet,
source => "puppet://puppet/apps/adobe/
AdobeReader_enu-8.1.3-1.sparc.tar.gz",
}
file { "/sysprov/apps/adobe/bar-ar-8.1.3-1.install.sh":
owner => puppet,
group => puppet,
source => "puppet://puppet/apps/adobe/adobe/ar-8.1.3-1.install.sh",
}
# Run the installer
exec { "ar-8.1.3-1.install.sh":
command => "/sysprov/apps/adobe/ar-8.1.3-1.install.sh",
require => File[ "/sysprov/apps/adobe/
AdobeReader_enu-8.1.3-1.sparc.tar.gz" ]
require => File[ "/sysprov/apps/adobe/ar-8.1.3-1.install.sh" ]
refreshonly => true,
}
I get the feeling that I don't have to create ar-8.1.3-1.install.sh
script and that the whole thing can be done in puppet or facteur.
Karl
On Nov 6, 10:37 am, "Evan Hisey" <[EMAIL PROTECTED]> wrote:
> On Thu, Nov 6, 2008 at 10:04 AM, Loobster <[EMAIL PROTECTED]> wrote:
>
> > I'm wondering if someone would like to share their Adobe Reader
> > recipe.
> >http://www.adobe.com/products/acrobat/readstep2.html
>
> > OR something similar that does the following:
>
> > Installs the software if the program does not exist or install the
> > software if the version is lower than x.y.z.
>
> > Thanks
> > Karl
>
> I just use the the adobe linux repo and use the package type to install it.
>
> The repo RPM is
> :http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1...
>
> I also do my flash plugin this way but I think the flash rpm is
> broken, as I had to add the plugin link for the browser with puppet. I
> have not checked the lo see what version is out now.
>
> Evan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: Adobe Reader Recipe
On Thu, Nov 6, 2008 at 10:04 AM, Loobster <[EMAIL PROTECTED]> wrote: > > I'm wondering if someone would like to share their Adobe Reader > recipe. > http://www.adobe.com/products/acrobat/readstep2.html > > OR something similar that does the following: > > Installs the software if the program does not exist or install the > software if the version is lower than x.y.z. > > Thanks > Karl > I just use the the adobe linux repo and use the package type to install it. The repo RPM is : http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpm I also do my flash plugin this way but I think the flash rpm is broken, as I had to add the plugin link for the browser with puppet. I have not checked the lo see what version is out now. Evan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
