Re: [Puppet Users] Re: Puppet and installing packages from source
On 01/04/2013 02:35 PM, Hans van der Made wrote: Many of us used to compile from source earlier in our careers, so don't asume we don't know about anything but ready made packages ;-) Many of us still do that today, but compile to build packages and not to shuffle files around production root filesytem ;) -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Puppet and installing packages from source
Thanks Hans, great answer.
I'll definitely get into customized packaging, demystify their magics.
Regards,
Magic.
On Friday, January 4, 2013 3:35:20 PM UTC+2, Hans van der Made wrote:
On Friday, January 4, 2013 10:39:53 AM UTC+1, magi...@gmail.com wrote:
I understand exactly what source packages do, if it makes it easier for
you I'll call them tar.gz files.
Also, I don't find your comparison valid, as the result of package
installation be it yum or ./configure make make install is the same:
software being installed.
My meaning is that compiling from source will always be with us, and it's
the prominent way of releasing new software.
When all the required features of a source build install environment are
met, like uninstalling, upgrading, listing, you end up with a new type of
package provider. As you don't want a split between OS-provided software
(libraries your source file depends on), it's easier to use the provider
your OS comes with.
The requirements for building source are often many. You'll need a
compiler and a toolchain, at least. Many don't want all of these tools
installed on their (virtual) production servers, because they consume
space, require updates, and because they facilitate exploits. Less software
means less work and less risk.
Building software on a separate system and deploying the resulting
binaries and documentation to production systems is one step up from
compiling on your production systems.
Many of us used to compile from source earlier in our careers, so don't
asume we don't know about anything but ready made packages ;-)
Now for a quick hack to address your problem:
Why don't you try to do a ./configure --prefix=/usr/local --specialopts
make make install on a clean system and create a tar file of /usr/local/
afterwards? Missing dependencies can be met by installing OS packages on
your test system. The installation on your target system can be a hack like
this:
# dependencies
package { [ dep1, dep2, dep3, ]: ensure = installed, }
file { /var/tmp/curl-custom-version.tar.gz:
ensure = file,
source = puppet:///module/curl-custom-version.tar.gz,
notify = Exec[unpackcurl],
}
exec { unpackcurl:
command = tar -C /usr/local -xzf /var/tmp/curl-custom-version.tar.gz,
creates = /usr/local/bin/curl,
path = /usr/sbin:/usr/bin:/sbin:/bin,
logoutput = true,
refreshonly = false,
}
If possible, use packages, but if you can't meet your deadline this way,
try the above (and discover the wonderful world of packaging anyway at a
later time).
Best,
Hans
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/ZD0p1hwTWdEJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Puppet and installing packages from source
I understand exactly what source packages do, if it makes it easier for you
I'll call them tar.gz files.
Also, I don't find your comparison valid, as the result of package
installation be it yum or ./configure make make install is the same:
software being installed.
My meaning is that compiling from source will always be with us, and it's
the prominent way of releasing new software.
Thanks,
Magic.
On Wednesday, January 2, 2013 11:32:55 AM UTC+2, magi...@gmail.com wrote:
Hello Everyone,
I'm looking for a way to install packages from source via puppet, I was
able to locate maybe 5 posts on this subject which is very strange, as I
would expect people to still use ./configure with their own customized
options.
For example, I need to compile Curl with c-ares support, for that I need
to set an option for ./configure but can't find a way to do it.
Here's the manifest, it works up to the point of configure which fails:
file { /usr/local/src: ensure = directory }
file { /usr/local/src/c-ares-1.9.1.tar.gz:
source = puppet:///modules/web_dev/c-ares-1.9.1.tar.gz,
alias = ares-source-tgz,
before = Exec[untar-ares-source]
}
exec { tar xzf /usr/local/src/c-ares-1.9.1.tar.gz:
cwd = /usr/local/src,
path = /bin,
creates = /usr/local/src/c-ares-1.9.1,
alias = untar-ares-source,
subscribe = File[ares-source-tgz]
}
exec { configure:
command = ./configure,
cwd = /usr/local/src/c-ares-1.9.1,
path = /usr/local/src/c-ares-1.9.1,
require = Exec[untar-ares-source],
creates = /usr/local/src/c-ares-1.9.1/config.h,
}
The ERROR:
Debug: Exec[configure](provider=posix): Executing './configure'
Debug: Executing './configure'
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: 1:
./configure: expr: not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: 1:
./configure: expr: not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: 95:
./configure: as_fn_error: not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
83: expr: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
90: expr: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
95: as_fn_error: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
465: sed: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
464: expr: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
465: sed: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: ./configure: line
479: sed: command not found
Notice: /Stage[main]/Web_dev/Exec[configure]/returns: : error: cannot
create .lineno; rerun with a POSIX shell
Error: ./configure returned 1 instead of one of [0]
Error: /Stage[main]/Web_dev/Exec[configure]/returns: change from notrun to
0 failed: ./configure returned 1 instead of one of [0]
Anyone did this or knows whats wrong ?
Thanks,
Magic.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/zLcAvPzXtPYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Puppet and installing packages from source
- Original Message -
From: magic rh magic...@gmail.com
To: puppet-users@googlegroups.com
Sent: Friday, January 4, 2013 9:39:53 AM
Subject: [Puppet Users] Re: Puppet and installing packages from source
I understand exactly what source packages do, if it makes it easier for you
I'll call them tar.gz files.
Also, I don't find your comparison valid, as the result of package
installation be it yum or ./configure make make install is the same:
software being installed.
it's not really though is it? the make install route drops a bunch of files
down, just tossing it out there with no regard for what comes before or after
the yum package apart from putting files on disks also does to name just a few:
- manages the serving, locating and transferring of the package onto your OS
for you, this is important because built into this process is things like
check summing, mirroring including fastest mirror determination, local
caches and so forth. You get some of this of course with the puppet file
server like the checksums but Puppet isnt really great at it.
- manage dependencies: makes sure you have all the pre-determined libraries
you need for the software to function. the ./configure method will adjust
the software based on the requirements met - you might find that on one
machine your ./configure results in a tool minus a key feature while on
the others it has it due to a simple ordering problem with when you deliver
devel libraries
- manage conflicts: if a OS update in the future comes with a file that might
conflict with your ./configure build you'll be none the wiser, it'll just be
overridden. Packages won't do this, it'll register a conflict and refuse to
install thus surfacing this behavior change and potential problem in a mature
way
- manage file lists: every file the package installs are registered and tracked
and you can later compare them for changes. or when you upgrade to the next
version orphaned files are detected and removed, config files are not blindly
overwritten but backed up and possibly not even replaced. you can later
safely
uninstall it based on this database
- handles things like integration with the OS init, rc, pam, logging and other
subsystems
- registers the new capability for other packages to be able to rely on it
- provide many CLI tools to query, validate and generally manage the software
you
have on your system in a single consistent way that means you never need to
track - as a user or as an automation system - things like different
configure
flags depending on the type of software. It's all standardised and unified
into
a single coherent system.
All in a way that combines with how your OS vendor chose to design their OS so
you
know that by working with their approach you'll end up with a better functioning
whole in the end
Puppet is designed to use the facilities provided by your package manager, for
example
the package resource has the ability to install, uninstall and upgrade a package
and it needs the ability to query if a package is installed. By using these
facilities it provides a idempotent means of managing the full life cycle of
your
software in a mature way. Because your tgz files do not do any of the above
things
puppet is simply not capable of providing a solid management experience of
software
delivered in such a manner - the key information sources and standardised
management
just does not exist for tgz files as they do for true packages.
The configure method does the absolute minimal, it just puts files down in some
non deterministic location. Worse it requires you to know and manage a wide
range
of different configure flags and have appropriate error handling for every
possible
scenario wedged awkwardly into some exec statements. This is not package
management,
they are not packages. This is just wholesale creation of technical debt.
My meaning is that compiling from source will always be with us, and it's
the prominent way of releasing new software.
We'll always be compiling software as you say, it's how you manage the
compilation
and delivery of the software that changes. fpm is no harder to use than the tar
command
and while it does not provide all of these benefits it goes a long way to
improving
what happens to your software once its compiled.
Thanks,
Magic.
On Wednesday, January 2, 2013 11:32:55 AM UTC+2, magi...@gmail.com wrote:
Hello Everyone,
I'm looking for a way to install packages from source via puppet, I was
able to locate maybe 5 posts on this subject which is very strange, as I
would expect people to still use ./configure with their own customized
options.
For example, I need to compile Curl with c-ares support, for that I need
to set an option for ./configure but can't find a way to do it.
Here's the manifest, it works up to the point of configure which fails:
file { /usr/local/src
[Puppet Users] Re: Puppet and installing packages from source
On Friday, January 4, 2013 10:39:53 AM UTC+1, magi...@gmail.com wrote:
I understand exactly what source packages do, if it makes it easier for
you I'll call them tar.gz files.
Also, I don't find your comparison valid, as the result of package
installation be it yum or ./configure make make install is the same:
software being installed.
My meaning is that compiling from source will always be with us, and it's
the prominent way of releasing new software.
When all the required features of a source build install environment are
met, like uninstalling, upgrading, listing, you end up with a new type of
package provider. As you don't want a split between OS-provided software
(libraries your source file depends on), it's easier to use the provider
your OS comes with.
The requirements for building source are often many. You'll need a compiler
and a toolchain, at least. Many don't want all of these tools installed on
their (virtual) production servers, because they consume space, require
updates, and because they facilitate exploits. Less software means less
work and less risk.
Building software on a separate system and deploying the resulting binaries
and documentation to production systems is one step up from compiling on
your production systems.
Many of us used to compile from source earlier in our careers, so don't
asume we don't know about anything but ready made packages ;-)
Now for a quick hack to address your problem:
Why don't you try to do a ./configure --prefix=/usr/local --specialopts
make make install on a clean system and create a tar file of /usr/local/
afterwards? Missing dependencies can be met by installing OS packages on
your test system. The installation on your target system can be a hack like
this:
# dependencies
package { [ dep1, dep2, dep3, ]: ensure = installed, }
file { /var/tmp/curl-custom-version.tar.gz:
ensure = file,
source = puppet:///module/curl-custom-version.tar.gz,
notify = Exec[unpackcurl],
}
exec { unpackcurl:
command = tar -C /usr/local -xzf /var/tmp/curl-custom-version.tar.gz,
creates = /usr/local/bin/curl,
path = /usr/sbin:/usr/bin:/sbin:/bin,
logoutput = true,
refreshonly = false,
}
If possible, use packages, but if you can't meet your deadline this way,
try the above (and discover the wonderful world of packaging anyway at a
later time).
Best,
Hans
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/dprmfK80yh8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Puppet and installing packages from source
I understand the points you mentioned, but as long as software is still being distributed as source packages I think it should be supported at some level by Puppet. That being said, I'm off to learn some packaging techniques. Thanks for all the help. Magic. On Wednesday, January 2, 2013 7:30:01 PM UTC+2, jcbollinger wrote: On Wednesday, January 2, 2013 3:32:55 AM UTC-6, magi...@gmail.com wrote: Hello Everyone, I'm looking for a way to install packages from source via puppet, I was able to locate maybe 5 posts on this subject which is very strange, as I would expect people to still use ./configure with their own customized options. Why would you expect that? Responsible sysadmins use packages wherever possible. The alternative is madness. Moreover, if you continue with the puppet-controlled build approach, you will find that you have to go to a fair amount of effort to ensure that the program is built and installed only at need. Then, eventually you may find that it's even trickier to fit updates into the picture. And you may discover that you need to explicitly manage dependencies -- hopefully before something breaks unexpectedly. Relying on your package manager makes all this a lot easier to deal with. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/8TpS7e3N-aIJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Puppet and installing packages from source
- Original Message - From: magic rh magic...@gmail.com To: puppet-users@googlegroups.com Sent: Thursday, January 3, 2013 10:52:55 AM Subject: [Puppet Users] Re: Puppet and installing packages from source I understand the points you mentioned, but as long as software is still being distributed as source packages I think it should be supported at some level by Puppet. I think the problem here is that you think they are source packages they are not packages at all, they are source. Sources used as input to creating packaged means of deploying code tailored to suit the needs of your distribution Just as you cannot consider bread and flour to be the same thing or say flour has the same properties as bread or offer the same advantages and usage models you cannot consider source a form of package. That being said, I'm off to learn some packaging techniques. Thanks for all the help. Magic. On Wednesday, January 2, 2013 7:30:01 PM UTC+2, jcbollinger wrote: On Wednesday, January 2, 2013 3:32:55 AM UTC-6, magi...@gmail.com wrote: Hello Everyone, I'm looking for a way to install packages from source via puppet, I was able to locate maybe 5 posts on this subject which is very strange, as I would expect people to still use ./configure with their own customized options. Why would you expect that? Responsible sysadmins use packages wherever possible. The alternative is madness. Moreover, if you continue with the puppet-controlled build approach, you will find that you have to go to a fair amount of effort to ensure that the program is built and installed only at need. Then, eventually you may find that it's even trickier to fit updates into the picture. And you may discover that you need to explicitly manage dependencies -- hopefully before something breaks unexpectedly. Relying on your package manager makes all this a lot easier to deal with. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/8TpS7e3N-aIJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Puppet and installing packages from source
On Wednesday, January 2, 2013 3:32:55 AM UTC-6, magi...@gmail.com wrote: Hello Everyone, I'm looking for a way to install packages from source via puppet, I was able to locate maybe 5 posts on this subject which is very strange, as I would expect people to still use ./configure with their own customized options. Why would you expect that? Responsible sysadmins use packages wherever possible. The alternative is madness. Moreover, if you continue with the puppet-controlled build approach, you will find that you have to go to a fair amount of effort to ensure that the program is built and installed only at need. Then, eventually you may find that it's even trickier to fit updates into the picture. And you may discover that you need to explicitly manage dependencies -- hopefully before something breaks unexpectedly. Relying on your package manager makes all this a lot easier to deal with. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/tx2YshpjgVcJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
