Re: [arch-general] pkg-proxy ?
> like:
> - does the code in "how to use" box at the forum post go in the
> pacproxy machine configs ?
do you mean the client machines? yes you just point the clients to
the proxy as if it were another mirror. you setup the actual mirror
by editing this line:
mirrors = {'mirrors.gigenet.com': '/archlinux/$repo/os/$arch'}
in the script.
> - what/where do i set/use this in the client(s) ?
/etc/pacman.conf
> - whats /archlinux dir ? should i create it, how ?
this is on the repo, not your local machine. don't worry about it,
it's virtual.
> - can't i use /var/cache/pacman/pkg/ ?
no. not as is anyway. the script looks to it's own cache
(~/.pacproxy) or to the internet; it does NOT check the local pacman
cache of the proxy machine.
> - does this work with the arch installer ?
sure. point the installer at the proxy on a net install. worked fine
for me (although the script needs some love.. i had some quirks here
and there like timeouts for some reason)
> sounds like a nice workaround. but does this cache the packages twice
> ? (apache cache and pacman cache)
yes. if localhost is configured to use the proxy, packages will be
cached twice.
i would be happy to discuss this further in the forums
[http://bbs.archlinux.org/viewtopic.php?id=87115], as the list members
are probably not particularly interested in this script.
C Anthony
Re: [arch-general] pkg-proxy ?
On Mon, May 17, 2010 at 8:46 PM, Wolfgang wrote: > > Hi, > >>> >> - not usable from arch installer. i assume i cant use it as proxy >>> >> setting what is asked in the installer. but maybe i can somehow else >>> >> use it from the installer? > > I'm doing something like this for a couple of centos machines... using > apache as a caching reverse proxy (also serving kickstart files). > > > # the usual servername, logfile, etc... > LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so > LoadModule disk_cache_module /usr/lib/apache2/modules/mod_disk_cache.so > CacheRoot /home/sites/ks/cache > CacheEnable disk / > CacheDirLevels 2 > CacheDirLength 8 > CacheMaxFileSize 2097152000 > > ProxyRequests off > ProxyPass /centos/ http://ftp.halifax.rwth-aachen.de/centos/ > > > it should just work with something like: > ProxyPass /archlinux/ http://some.mirror/path_to/archlinux/ > point your installers there and you should be fine > > I know you said squid is to much hassle... maybe your more comfortable > with apache ;) > > Regards, > Wolfgang sounds like a nice workaround. but does this cache the packages twice ? (apache cache and pacman cache)
Re: [arch-general] pkg-proxy ?
On Mon, May 17, 2010 at 7:35 PM, C Anthony Risinger wrote: > On May 17, 2010, at 12:22 PM, Gregory Eric Sanderson > wrote: > >> On Mon, May 17, 2010 at 10:45 AM, Rogutės Sparnuotos > m >>> wrote: >> >>> Andre Osku Schmidt (2010-05-16 13:33): 2010/5/16 Ng Oon-Ee : > On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: >> 2010/5/15 Ng Oon-Ee : >>> Check out 'pkgd' >> >> thanks, works nicely! >> >> only couple issues that would be nice if it could do too (or >> already >> does, and i just didn't find out how) >> >> - not usable from arch installer. i assume i cant use it as proxy >> setting what is asked in the installer. but maybe i can somehow >> else >> use it from the installer? > > You would need (AFAIK) to install some packages first to start > using >>> it. >> >> - if the package is not found on the pkgd server, it will be >> loaded >> from internet to the client machine. is there a way to tell the >> pkgd >> server to download it and serve it to the client ? > > Not that I'm aware off. Talk to Xyne. > > Perhaps you'd just want to set up your own mirror. There's > projects for > that as well in the AUR, just search. do you mean a general mirroring tool ? as i didn't found anything pkg specific mirror tool... and wouldn't a mirror tool require my server to have ALL core/extra/community packages ? how big are those repos ? >>> <...> >>> >>> I don't really remember your initial question, but I use one package >>> directory for 3 computers by simply having a central /var/cache/ >>> pacman, >>> which I then mount read-write with samba. To use it from the >>> installer, >>> you would have to install 'smbclient' after booting. >>> >> >> That solution is also mentioned on the wiki, but I see 2 >> disadvatages to >> mounting /var/cache/pacman through the network. >> 1. If you're on a laptop and not at home, you don't have access to >> your >> packages, and If you want to install packages anyway you have to >> manually >> unmount, install, and remove the packages from the directory to to >> cause >> problems for when it will be remounted. >> 2. Although rare, if or have frequent network connectivity problems >> (for >> example, your connection goes dead in the middle of copying a file) >> then it >> becomes a hassle > > This is the same conclusion I came to, and why I started the > "pacproxy" app... My sshfs mount would be down without me knowing; I > just don't think it's a very good/elegant solution. > > I intended pacproxy to be an apt-proxy clone, with ABS "auto > repository" support, and foriegn ABS support (so I could have separate > build machines, and be able to broadcast their ABS tree as an > independent repo) right now though, it works well for simple proxying > and caching, and is a viable solution to the OP's problem. > > Perhaps we could make it more feature complete, and include it in the > official repos as a more comprehensive solution to fickle network > mounting. > > Else I will finish it eventually :-) > > C Anthony > hi, this sounds perfect! http://bbs.archlinux.org/viewtopic.php?id=87115 could you put it in a VCS (like gitorious.org) ? and a more verbose howto ? like: - does the code in "how to use" box at the forum post go in the pacproxy machine configs ? - what/where do i set/use this in the client(s) ? - whats /archlinux dir ? should i create it, how ? - can't i use /var/cache/pacman/pkg/ ? - does this work with the arch installer ? these may be "stupid" questions, but i'm still pretty new to Arch way of things. cheers .andre
Re: [arch-general] pkg-proxy ?
Hi, >> >> - not usable from arch installer. i assume i cant use it as proxy >> >> setting what is asked in the installer. but maybe i can somehow else >> >> use it from the installer? I'm doing something like this for a couple of centos machines... using apache as a caching reverse proxy (also serving kickstart files). # the usual servername, logfile, etc... LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so LoadModule disk_cache_module /usr/lib/apache2/modules/mod_disk_cache.so CacheRoot /home/sites/ks/cache CacheEnable disk / CacheDirLevels 2 CacheDirLength 8 CacheMaxFileSize 2097152000 ProxyRequests off ProxyPass /centos/ http://ftp.halifax.rwth-aachen.de/centos/ it should just work with something like: ProxyPass /archlinux/ http://some.mirror/path_to/archlinux/ point your installers there and you should be fine I know you said squid is to much hassle... maybe your more comfortable with apache ;) Regards, Wolfgang
Re: [arch-general] pkg-proxy ?
On May 17, 2010, at 12:22 PM, Gregory Eric Sanderson wrote: > On Mon, May 17, 2010 at 10:45 AM, Rogutės Sparnuotos m >> wrote: > >> Andre Osku Schmidt (2010-05-16 13:33): >>> 2010/5/16 Ng Oon-Ee : On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: > 2010/5/15 Ng Oon-Ee : >> Check out 'pkgd' > > thanks, works nicely! > > only couple issues that would be nice if it could do too (or > already > does, and i just didn't find out how) > > - not usable from arch installer. i assume i cant use it as proxy > setting what is asked in the installer. but maybe i can somehow > else > use it from the installer? You would need (AFAIK) to install some packages first to start using >> it. > > - if the package is not found on the pkgd server, it will be > loaded > from internet to the client machine. is there a way to tell the > pkgd > server to download it and serve it to the client ? Not that I'm aware off. Talk to Xyne. Perhaps you'd just want to set up your own mirror. There's projects for that as well in the AUR, just search. >>> >>> do you mean a general mirroring tool ? >>> as i didn't found anything pkg specific mirror tool... >>> >>> and wouldn't a mirror tool require my server to have ALL >>> core/extra/community packages ? how big are those repos ? >> <...> >> >> I don't really remember your initial question, but I use one package >> directory for 3 computers by simply having a central /var/cache/ >> pacman, >> which I then mount read-write with samba. To use it from the >> installer, >> you would have to install 'smbclient' after booting. >> > > That solution is also mentioned on the wiki, but I see 2 > disadvatages to > mounting /var/cache/pacman through the network. > 1. If you're on a laptop and not at home, you don't have access to > your > packages, and If you want to install packages anyway you have to > manually > unmount, install, and remove the packages from the directory to to > cause > problems for when it will be remounted. > 2. Although rare, if or have frequent network connectivity problems > (for > example, your connection goes dead in the middle of copying a file) > then it > becomes a hassle This is the same conclusion I came to, and why I started the "pacproxy" app... My sshfs mount would be down without me knowing; I just don't think it's a very good/elegant solution. I intended pacproxy to be an apt-proxy clone, with ABS "auto repository" support, and foriegn ABS support (so I could have separate build machines, and be able to broadcast their ABS tree as an independent repo) right now though, it works well for simple proxying and caching, and is a viable solution to the OP's problem. Perhaps we could make it more feature complete, and include it in the official repos as a more comprehensive solution to fickle network mounting. Else I will finish it eventually :-) C Anthony
Re: [arch-general] pkg-proxy ?
On Mon, May 17, 2010 at 10:45 AM, Rogutės Sparnuotos wrote: > Andre Osku Schmidt (2010-05-16 13:33): > > 2010/5/16 Ng Oon-Ee : > > > On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: > > >> 2010/5/15 Ng Oon-Ee : > > >> > Check out 'pkgd' > > >> > > >> thanks, works nicely! > > >> > > >> only couple issues that would be nice if it could do too (or already > > >> does, and i just didn't find out how) > > >> > > >> - not usable from arch installer. i assume i cant use it as proxy > > >> setting what is asked in the installer. but maybe i can somehow else > > >> use it from the installer? > > > > > > You would need (AFAIK) to install some packages first to start using > it. > > >> > > >> - if the package is not found on the pkgd server, it will be loaded > > >> from internet to the client machine. is there a way to tell the pkgd > > >> server to download it and serve it to the client ? > > > > > > Not that I'm aware off. Talk to Xyne. > > > > > > Perhaps you'd just want to set up your own mirror. There's projects for > > > that as well in the AUR, just search. > > > > do you mean a general mirroring tool ? > > as i didn't found anything pkg specific mirror tool... > > > > and wouldn't a mirror tool require my server to have ALL > > core/extra/community packages ? how big are those repos ? > <...> > > I don't really remember your initial question, but I use one package > directory for 3 computers by simply having a central /var/cache/pacman, > which I then mount read-write with samba. To use it from the installer, > you would have to install 'smbclient' after booting. > That solution is also mentioned on the wiki, but I see 2 disadvatages to mounting /var/cache/pacman through the network. 1. If you're on a laptop and not at home, you don't have access to your packages, and If you want to install packages anyway you have to manually unmount, install, and remove the packages from the directory to to cause problems for when it will be remounted. 2. Although rare, if or have frequent network connectivity problems (for example, your connection goes dead in the middle of copying a file) then it becomes a hassle > -- > -- Rogutės Sparnuotos > -- "All musicians are drug addicts, no question about it. The ecstasy we get during a concert is proof enough. yet there is a slight difference between us, the musicians, and the typical 'street-junkie'... Instead of consuming powder, we consume vibrations" Will et/ou Gregory Eric Sanderson Turcot Temlett MacDonnell Forbes et/ou Touffa! :)
Re: [arch-general] pkg-proxy ?
Andre Osku Schmidt (2010-05-16 13:33): > 2010/5/16 Ng Oon-Ee : > > On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: > >> 2010/5/15 Ng Oon-Ee : > >> > Check out 'pkgd' > >> > >> thanks, works nicely! > >> > >> only couple issues that would be nice if it could do too (or already > >> does, and i just didn't find out how) > >> > >> - not usable from arch installer. i assume i cant use it as proxy > >> setting what is asked in the installer. but maybe i can somehow else > >> use it from the installer? > > > > You would need (AFAIK) to install some packages first to start using it. > >> > >> - if the package is not found on the pkgd server, it will be loaded > >> from internet to the client machine. is there a way to tell the pkgd > >> server to download it and serve it to the client ? > > > > Not that I'm aware off. Talk to Xyne. > > > > Perhaps you'd just want to set up your own mirror. There's projects for > > that as well in the AUR, just search. > > do you mean a general mirroring tool ? > as i didn't found anything pkg specific mirror tool... > > and wouldn't a mirror tool require my server to have ALL > core/extra/community packages ? how big are those repos ? <...> I don't really remember your initial question, but I use one package directory for 3 computers by simply having a central /var/cache/pacman, which I then mount read-write with samba. To use it from the installer, you would have to install 'smbclient' after booting. -- -- Rogutės Sparnuotos
Re: [arch-general] pkg-proxy ?
On May 16, 2010, at 4:32 AM, Ng Oon-Ee wrote: > On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: >> 2010/5/15 Ng Oon-Ee : >>> Check out 'pkgd' >> >> thanks, works nicely! >> >> only couple issues that would be nice if it could do too (or already >> does, and i just didn't find out how) >> >> - not usable from arch installer. i assume i cant use it as proxy >> setting what is asked in the installer. but maybe i can somehow else >> use it from the installer? > > You would need (AFAIK) to install some packages first to start using > it. >> >> - if the package is not found on the pkgd server, it will be loaded >> from internet to the client machine. is there a way to tell the pkgd >> server to download it and serve it to the client ? > > Not that I'm aware off. Talk to Xyne. > > Perhaps you'd just want to set up your own mirror. There's projects > for > that as well in the AUR, just search. I wrote a Piece of Junk app to do exactly this... Serve packages from a cache or retrieve from the net and serve to client. I used it to install custom stuff during install (btrfs-progs and friends in my case). Works well enough for one time type stuff. Search forums for "pacproxy". C Anthony
Re: [arch-general] pkg-proxy ?
2010/5/16 Ng Oon-Ee : > On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: >> 2010/5/15 Ng Oon-Ee : >> > Check out 'pkgd' >> >> thanks, works nicely! >> >> only couple issues that would be nice if it could do too (or already >> does, and i just didn't find out how) >> >> - not usable from arch installer. i assume i cant use it as proxy >> setting what is asked in the installer. but maybe i can somehow else >> use it from the installer? > > You would need (AFAIK) to install some packages first to start using it. >> >> - if the package is not found on the pkgd server, it will be loaded >> from internet to the client machine. is there a way to tell the pkgd >> server to download it and serve it to the client ? > > Not that I'm aware off. Talk to Xyne. > > Perhaps you'd just want to set up your own mirror. There's projects for > that as well in the AUR, just search. do you mean a general mirroring tool ? as i didn't found anything pkg specific mirror tool... and wouldn't a mirror tool require my server to have ALL core/extra/community packages ? how big are those repos ? it seems there is (yet) no pkg-proxy tool (ala apt-proxy[0]) that could be used from installer and on. also the proxy should get new files only when a client needs it... i assume Xyne is the author of pkgd, ill contact him/her later and ask if these features are possible to implement in pkgd. but for now, ill just get packages (net)install wants from internet, and use pkgd after that (and add pkgs that clients want manually to the pkgd server) cheers .andre [0] http://apt-proxy.sourceforge.net/
Re: [arch-general] pkg-proxy ?
On Sat, 2010-05-15 at 23:47 +0200, Andre "Osku" Schmidt wrote: > 2010/5/15 Ng Oon-Ee : > > Check out 'pkgd' > > thanks, works nicely! > > only couple issues that would be nice if it could do too (or already > does, and i just didn't find out how) > > - not usable from arch installer. i assume i cant use it as proxy > setting what is asked in the installer. but maybe i can somehow else > use it from the installer? You would need (AFAIK) to install some packages first to start using it. > > - if the package is not found on the pkgd server, it will be loaded > from internet to the client machine. is there a way to tell the pkgd > server to download it and serve it to the client ? Not that I'm aware off. Talk to Xyne. Perhaps you'd just want to set up your own mirror. There's projects for that as well in the AUR, just search.
Re: [arch-general] pkg-proxy ?
2010/5/15 Ng Oon-Ee : > Check out 'pkgd' thanks, works nicely! only couple issues that would be nice if it could do too (or already does, and i just didn't find out how) - not usable from arch installer. i assume i cant use it as proxy setting what is asked in the installer. but maybe i can somehow else use it from the installer? - if the package is not found on the pkgd server, it will be loaded from internet to the client machine. is there a way to tell the pkgd server to download it and serve it to the client ?
Re: [arch-general] pkg-proxy ?
On Sat, 2010-05-15 at 11:47 +0200, Andre "Osku" Schmidt wrote: > Hi, > > i would like to experiment with (semi) automatic installation and > configuration of my arch system(s). and to make it more faster i would > like to setup a "pkg-proxy". > > on debian there are many tools for this > http://www.debian-administration.org/polls/139 but i would rather not > yet use squid (too much hassle) ;) > > at the moment i would use it on my host system, serving packages for vm > clients. > (when i get more money/hardware, i would try squid as transparent proxy) > > so, is there a tool for arch linux that caches arch packages, and > which i can use as proxy in the arch installer and beyond. > > thanks > .andre Check out 'pkgd'
[arch-general] pkg-proxy ?
Hi, i would like to experiment with (semi) automatic installation and configuration of my arch system(s). and to make it more faster i would like to setup a "pkg-proxy". on debian there are many tools for this http://www.debian-administration.org/polls/139 but i would rather not yet use squid (too much hassle) ;) at the moment i would use it on my host system, serving packages for vm clients. (when i get more money/hardware, i would try squid as transparent proxy) so, is there a tool for arch linux that caches arch packages, and which i can use as proxy in the arch installer and beyond. thanks .andre
