Transferring ports
Hi, I have an idea and a request for people familiar with ports & pkgdb infrastructure: a utility (preferably written in C, Python or as a shell script) that would transfer *installed* ports from one system tree to the other, including their dependencies. It would transfer only some ports, specified on the command line. The details: imagine there are two or more full FreeBSD installation trees in the file system (e.g. complete jails). The utility would transfer (installed) packages from one tree to the other. The easy, brute-force way would be to generate package files (tbz) from the installed tree and then install them to the other tree, but I can't do that because of performance and disk space reasons. Is there a utility that would do that, and if not, does anyone have the time to write one? signature.asc Description: OpenPGP digital signature
Re: Transferring ports
* Ivan Voras ([EMAIL PROTECTED]) wrote:
> I have an idea and a request for people familiar with ports & pkgdb
> infrastructure: a utility (preferably written in C, Python or as a shell
> script) that would transfer *installed* ports from one system tree to
> the other, including their dependencies. It would transfer only some
> ports, specified on the command line.
There's no way to do it clearly. Not only such utility will have
to deal with dependencies anyway, but also there are ports that do more
than just copy files on installation (such as registering uids/gids,
handling user-modified configs nicely etc.).
> The details: imagine there are two or more full FreeBSD installation
> trees in the file system (e.g. complete jails). The utility would
> transfer (installed) packages from one tree to the other. The easy,
> brute-force way would be to generate package files (tbz) from the
> installed tree and then install them to the other tree, but I can't do
> that because of performance and disk space reasons.
I think that the easiest and most correct way will be to use packages.
You can share packages/ directory between jails via nullfs (or
between hosts via nfs) - single set of packages surely will take
no more space than a single set of installed ports in a jail. And
the only performance overhead is bzipping a package one time and
bunzipping number_of_jails times.
And actually, what you want is done pretty easily, like this:
JAIL=/path/to/target/jail
pkg_info -q -L [...] | while read file; do
mkdir -p $JAIL/`dirname "$file"`
cp -pP "$file" $JAIL/"$file"
done
(this is just a scratch, use with care)
> Is there a utility that would do that, and if not, does anyone have the
> time to write one?
Actually, I've already had an idea of utility with pretty similar
functionality for a long time. The utility would copy directory
hierarchies recursively based on file include/exclude list, like this:
+/{etc,bin,sbin,lib}
+/usr
-/usr/local
+/usr/local/{bin,sbin,libexec,share,lib}
-/usr/share/locale
+/usr/share/locale/ru_RU*
so `my_cool_copy_utility / /path/to/jail` will copy /etc,/bin,/sbin,/lib
and /usr dirs to jail, but in /usr/share/locale will only copy
russian locales, but no others, and in usr/local it won't copy
man, include and other dirs not needed in a jail.
The purpose is similar - creating jails out of host system in fast
and easy way, possibility to strip everything unneeded (useful for
secure minimal jails or flash/livecd/embedded installations of
minimal size) and add something extra, like stuff from /usr/local
without installing full packages in a jail, or, say, copying over
additional tree of jail-specific changes (mostly stuff under /etc
and /usr/local/etc).
Such an utility is something I still might start working on.
--
Dmitry A. Marakasov| jabber: [EMAIL PROTECTED]
[EMAIL PROTECTED] | http://www.amdmi3.ru
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Transferring ports
On 13/03/2008, Dmitry Marakasov <[EMAIL PROTECTED]> wrote: > * Ivan Voras ([EMAIL PROTECTED]) wrote: > > > I have an idea and a request for people familiar with ports & pkgdb > > infrastructure: a utility (preferably written in C, Python or as a shell > > script) that would transfer *installed* ports from one system tree to > > the other, including their dependencies. It would transfer only some > > ports, specified on the command line. > There's no way to do it clearly. Not only such utility will have > to deal with dependencies anyway, but also there are ports that do more > than just copy files on installation (such as registering uids/gids, > handling user-modified configs nicely etc.). I only need the functionality that now exists by doing "pkg_create -b" to create a package, and then install it. However "pkg_create -b" does it, that's how I need it. > Actually, I've already had an idea of utility with pretty similar > functionality for a long time. The utility would copy directory > hierarchies recursively based on file include/exclude list, like this: > The purpose is similar - creating jails out of host system in fast > and easy way, possibility to strip everything unneeded (useful for > secure minimal jails or flash/livecd/embedded installations of > minimal size) and add something extra, like stuff from /usr/local > without installing full packages in a jail, or, say, copying over > additional tree of jail-specific changes (mostly stuff under /etc > and /usr/local/etc). This seems like something that would be also useful to me, if it would also read pkgdb :). I need to clarify so people don't flood me with nullfs suggestions: I don't actually need it for jails, but that was the easiest way for me to describe it - I need it to set up new installations. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Transferring ports
On Fri, Mar 14, 2008 at 12:02:42AM +0300, Dmitry Marakasov wrote:
> * Ivan Voras ([EMAIL PROTECTED]) wrote:
> > Is there a utility that would do that, and if not, does anyone have the
> > time to write one?
>
> Actually, I've already had an idea of utility with pretty similar
> functionality for a long time. The utility would copy directory
> hierarchies recursively based on file include/exclude list, like this:
>
> +/{etc,bin,sbin,lib}
> +/usr
> -/usr/local
> +/usr/local/{bin,sbin,libexec,share,lib}
> -/usr/share/locale
> +/usr/share/locale/ru_RU*
>
> so `my_cool_copy_utility / /path/to/jail` will copy /etc,/bin,/sbin,/lib
> and /usr dirs to jail, but in /usr/share/locale will only copy
> russian locales, but no others, and in usr/local it won't copy
> man, include and other dirs not needed in a jail.
>
> The purpose is similar - creating jails out of host system in fast
> and easy way, possibility to strip everything unneeded (useful for
> secure minimal jails or flash/livecd/embedded installations of
> minimal size) and add something extra, like stuff from /usr/local
> without installing full packages in a jail, or, say, copying over
> additional tree of jail-specific changes (mostly stuff under /etc
> and /usr/local/etc).
>
> Such an utility is something I still might start working on.
Err... why not use rsync for that? It works locally, too -
I use it to copy directories all over the place all the time.
Come to think of it... oh, just go install net/rsync, take a look at
its manual page and the "FILTER RULES" section in particular - it even
supports rules with prefixes, "-" for exclude, "+" for include, just
like you want them :) Well, okay, you might need to list separate
directories on separate lines (it doesn't seem to support the {bin,sbin}
syntax), but other than that, it seems to fit your requirements pretty
well :)
G'luck,
Peter
--
Peter Pentchev [EMAIL PROTECTED][EMAIL PROTECTED][EMAIL PROTECTED]
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553
This sentence claims to be an Epimenides paradox, but it is lying.
pgpvZaprrNwIx.pgp
Description: PGP signature
Re: Transferring ports
Peter Pentchev wrote: On Fri, Mar 14, 2008 at 12:02:42AM +0300, Dmitry Marakasov wrote: * Ivan Voras ([EMAIL PROTECTED]) wrote: Is there a utility that would do that, and if not, does anyone have the time to write one? >> Would this not be an appropriate use for packages? If one creates a package for every installed port on the "host" system, then one simply installs the package on the target system. I have used this technique with some success when "transferring" ports from one system to another. In my situation, I'm using the same architecture (i386) and OS version (6.3 --> 6.3 or 7.0 --> 7.0). -- Regards, Doug ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Transferring ports
On 20/03/2008, Doug Poland <[EMAIL PROTECTED]> wrote: > Peter Pentchev wrote: > > On Fri, Mar 14, 2008 at 12:02:42AM +0300, Dmitry Marakasov wrote: > >> * Ivan Voras ([EMAIL PROTECTED]) wrote: > >>> Is there a utility that would do that, and if not, does anyone have the > >>> time to write one? > >> > > Would this not be an appropriate use for packages? If one creates a > package for every installed port on the "host" system, then one simply > installs the package on the target system. Yes, that's exactly what I need (the same functionality as "pkg_create -b" + install on the other system), only without the actual package file being created. Pipes would also be acceptable (piping the output of pkg_create from one machine to the other, etc). ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Transferring ports
Peter Pentchev wrote: On Fri, Mar 14, 2008 at 12:02:42AM +0300, Dmitry Marakasov wrote: The purpose is similar - creating jails out of host system in fast and easy way, possibility to strip everything unneeded (useful for secure minimal jails or flash/livecd/embedded installations of minimal size) and add something extra, like stuff from /usr/local without installing full packages in a jail, or, say, copying over additional tree of jail-specific changes (mostly stuff under /etc and /usr/local/etc). Such an utility is something I still might start working on. I don't use the host system.. I keep a special pristine jail just for that purpose (to act as a source for other jails). sometimes I also use null=mounts, and sometimes if the jails are on one big partition, I hardlink some stuff.. e.g binaries in /bin etc betweem teh jails.. saves memory and disk.. Of course that is only when I basically trust the jail user (me). ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Transferring ports
Ivan Voras wrote: On 20/03/2008, Doug Poland <[EMAIL PROTECTED]> wrote: Peter Pentchev wrote: > On Fri, Mar 14, 2008 at 12:02:42AM +0300, Dmitry Marakasov wrote: >> * Ivan Voras ([EMAIL PROTECTED]) wrote: >>> Is there a utility that would do that, and if not, does anyone have the >>> time to write one? >> Would this not be an appropriate use for packages? If one creates a package for every installed port on the "host" system, then one simply installs the package on the target system. Yes, that's exactly what I need (the same functionality as "pkg_create -b" + install on the other system), only without the actual package file being created. Pipes would also be acceptable (piping the output of pkg_create from one machine to the other, etc). > Too bad you cannot accept the package file. If pkg_create would accept a - instead of specifying the output tarball, then one could do some foo with nc, i.e., target# nc -l 1234 | tar -xf - source# pkg_create -b mypackage - | nc target 1234 -- Regards, Doug ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "[EMAIL PROTECTED]"
