Re: Questions on portmaster
On 7/2/09, Manish Jain wrote: > b. f. wrote: >> Manish Jain wrote: > Regarding the download speeds I was getting with portmaster fetches (0.7 > kBps to 4.0 kBps), I immediately booted into Windows/Cygwin and did a > wget from the same site portmaster was using. The speed I got from wget > was ~ 35 kBps. This happened not just once but multiple times. Each time > I did this, I had to interrupt portmaster. Finally I managed to get to > install the wget port on FreeBSD itself. When I ran wget from FreeBSD, > it reported comparable transfer rates (~ 35 kBps) from the same sites as > portmaster was using. > This has nothing to do with portmaster. Remember, portmaster is just a (sometimes) convenient shell script wrapper for the normal ports infrastructure. What you're probably seeing is either the usual fluctuations in network performance, or the difference between wget and FreeBSD fetch(1), which by default is used for getting distfiles and packages. This is set in bsd.port.mk, and you can override it if you prefer to use something else. See the comments concerning FETCH_BINARY, FETCH_ARGS, FETCH_CMD, FETCH_{BEFORE,AFTER}_ARGS, FETCH_ENV, and FETCH_REGET at the top of bsd.port.mk, or the relevant portion of that script. You can also try tuning your network settings, which can speed things up significantly in some cases. >>> It respects IGNORE (it checks for it in the port Makefile, and also >>> hands off to bsd.port.mk, which respects it). > > I got a curl port from portsnap marked IGNORE. portmaster did not ignore > it anywhere near gracefully enough and finally killed off all child > processes and itself. > Uh, yeah -- but that's another matter. It did IGNORE it, but it just stopped entirely. This is another one of the things that need improvement. portupgrade, by contrast, will keep going, only ignoring the port in question and any ports that depend upon it, but updating the rest. You should be able to patch portmaster to mimic this behavior. > Anyway, my system became so unpredictable that I had to reinstall > FreeBSD. For the moment, I am avoiding portmaster till I can try it out > on a dummy PC first. > Well, something is awry if your system was damaged that badly, and it probably wasn't because of portmaster. In general, few ports ought to damage the base system. You're probably better off trying to remove and then reinstall all of your ports first (the instructions in the portmaster manpage are good in this regard), before reinstalling everything. > BTW, I also tried portmanager and it segfaulted at the stage of > generating a report. > Oh, well. Sounds like a job for rnol...@freebsd.org. But wait -- who wants to bother him about some tiny piece of port management software when he is heroically staging a single-handed campaign to maintain X11 on FreeBSD? b. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Questions on portmaster
b. f. wrote: Manish Jain wrote: ...Does each child start 'make fetch' in the background ? make checksum, yes. Further, how can portmaster be tuned to automatically ignore ports which are actually marked as IGNORE in the port directory ? This is not covered in the manpage. It respects IGNORE (it checks for it in the port Makefile, and also hands off to bsd.port.mk, which respects it). If you mean +IGNOREME, the others have answered your question. While doing portmaster -a, I have only managed to stop portmaster from building exactly one port specified with the -x option. Can I get to stop multiple ports from being built ? Is there regular expression support for the -x option ? Not in the sense that you mean, at least that I'm aware of. This one of the things that needs improvement. It is a bit awkward, because it uses the shell's built-in POSIX getopts to parse options, and then calls itself recursively. One way you could fix it would be to apply a patch like: --- portmaster.orig 2009-07-01 12:36:14.0 -0400 +++ portmaster 2009-07-01 18:55:59.0 -0400 @@ -9,7 +9,7 @@ if [ -z "$PARENT_PID" ]; then PARENT_PID=$$ : ${TMPDIR:=/tmp} - UPGRADE_TOOL=portmaster + UPGRADE_TOOL=$0 # /usr/local is needed in the path for make PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin @@ -788,7 +788,11 @@ u) UNATTENDED=uopt; ARGS="-u $ARGS" ;; v) PM_VERBOSE=vopt; ARGS="-v $ARGS" ;; w) SAVE_SHARED=wopt; ARGS="-w $ARGS" ;; - x) EXCL=$OPTARG ;; + x) if [ -z "${OPTARG%%-*}" ]; then + fail 'The -x option requires an argument' + else + EXCL="-x $OPTARG $EXCL" + fi ;; *) echo '' ; echo "===>>> Try ${0##*/} --help"; exit 1 ;; esac done @@ -810,10 +814,7 @@ [ -n "$FETCH_ONLY" -a -n "$NO_RECURSIVE_CONFIG" ] && fail "The -F and -G options are mutually exclusive" if [ -n "$EXCL" ]; then - case "$EXCL" in - -*) fail 'The -x option requires an argument' ;; - *) ARGS="-x $EXCL $ARGS" ;; - esac + ARGS="$EXCL $ARGS" fi #=== Begin functions for getopts features and main === @@ -1461,14 +1462,17 @@ check_exclude () { [ -n "$EXCL" ] || return 0 - case "$1" in - *${EXCL}*) - if [ -n "$PM_VERBOSE" ]; then - echo "===>>> Skipping $1" - echo " because it matches the pattern: *${EXCL}*" - fi - return 1 ;; - esac + for pkgglob in `echo "$EXCL" | sed -e 's#-x##g'` + do + case "$1" in + *${pkgglob}*) + if [ -n "$PM_VERBOSE" ]; then + echo "===>>> Skipping $1" + echo " because it matches the pattern: *${pkgglob}*" + fi + return 1 ;; + esac ; + done return 0 } (Mind the whitespace because of my MUA.) Then you could just use repeated -x flags, each with one and only one package glob that you wanted to exclude. I changed the definition of UPGRADE_TOOL so that you could put this script in your path under another name, say "jainpmaster", and then call it independently of the original portmaster. As usual, I make no claim that this is the best, only, or most elegant way to do this. Regards, b. Hello BF/Roland, Thanks for the clarifications. Regarding the download speeds I was getting with portmaster fetches (0.7 kBps to 4.0 kBps), I immediately booted into Windows/Cygwin and did a wget from the same site portmaster was using. The speed I got from wget was ~ 35 kBps. This happened not just once but multiple times. Each time I did this, I had to interrupt portmaster. Finally I managed to get to install the wget port on FreeBSD itself. When I ran wget from FreeBSD, it reported comparable transfer rates (~ 35 kBps) from the same sites as portmaster was using. It respects IGNORE (it checks for it in the port Makefile, and also hands off to bsd.port.mk, which respects it). I got a curl port from portsnap marked IGNORE. portmaster did not ignore it anywhere near gracefully enough and finally killed off all child processes and itself. The IGNOREME way is fine for me for avoiding multiple ports I know beforehand I do not want to build. But if a portsnap update creates an IGNORE port, portmaster should be skipping it entirely rather than having to kill all child processes and itself. In my experience, this did not happen. Or maybe my system was badly broken already by that time. Anyway, my system became so unpredictable that I had to reinstall FreeBSD. For the moment, I am avoiding portmaster till I can try it out on a dummy PC first. BTW, I also tried portmanager and it segfaulted at the stage of generating a report. Thanks for all the help. Manish Jai
Re: Questions on portmaster
Manish Jain wrote: >...Does each child start 'make fetch' in the background ? make checksum, yes. >Further, how can portmaster be tuned to automatically ignore ports which >are actually marked as IGNORE in the port directory ? This is not >covered in the manpage. It respects IGNORE (it checks for it in the port Makefile, and also hands off to bsd.port.mk, which respects it). If you mean +IGNOREME, the others have answered your question. >While doing portmaster -a, I have only managed to stop portmaster from >building exactly one port specified with the -x option. Can I get to >stop multiple ports from being built ? Is there regular expression >support for the -x option ? Not in the sense that you mean, at least that I'm aware of. This one of the things that needs improvement. It is a bit awkward, because it uses the shell's built-in POSIX getopts to parse options, and then calls itself recursively. One way you could fix it would be to apply a patch like: --- portmaster.orig 2009-07-01 12:36:14.0 -0400 +++ portmaster 2009-07-01 18:55:59.0 -0400 @@ -9,7 +9,7 @@ if [ -z "$PARENT_PID" ]; then PARENT_PID=$$ : ${TMPDIR:=/tmp} - UPGRADE_TOOL=portmaster + UPGRADE_TOOL=$0 # /usr/local is needed in the path for make PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin @@ -788,7 +788,11 @@ u) UNATTENDED=uopt; ARGS="-u $ARGS" ;; v) PM_VERBOSE=vopt; ARGS="-v $ARGS" ;; w) SAVE_SHARED=wopt; ARGS="-w $ARGS" ;; - x) EXCL=$OPTARG ;; + x) if [ -z "${OPTARG%%-*}" ]; then + fail 'The -x option requires an argument' + else + EXCL="-x $OPTARG $EXCL" + fi ;; *) echo '' ; echo "===>>> Try ${0##*/} --help"; exit 1 ;; esac done @@ -810,10 +814,7 @@ [ -n "$FETCH_ONLY" -a -n "$NO_RECURSIVE_CONFIG" ] && fail "The -F and -G options are mutually exclusive" if [ -n "$EXCL" ]; then - case "$EXCL" in - -*) fail 'The -x option requires an argument' ;; - *) ARGS="-x $EXCL $ARGS" ;; - esac + ARGS="$EXCL $ARGS" fi #=== Begin functions for getopts features and main === @@ -1461,14 +1462,17 @@ check_exclude () { [ -n "$EXCL" ] || return 0 - case "$1" in - *${EXCL}*) - if [ -n "$PM_VERBOSE" ]; then - echo "===>>> Skipping $1" - echo " because it matches the pattern: *${EXCL}*" - fi - return 1 ;; - esac + for pkgglob in `echo "$EXCL" | sed -e 's#-x##g'` + do + case "$1" in + *${pkgglob}*) + if [ -n "$PM_VERBOSE" ]; then + echo "===>>> Skipping $1" + echo " because it matches the pattern: *${pkgglob}*" + fi + return 1 ;; + esac ; + done return 0 } (Mind the whitespace because of my MUA.) Then you could just use repeated -x flags, each with one and only one package glob that you wanted to exclude. I changed the definition of UPGRADE_TOOL so that you could put this script in your path under another name, say "jainpmaster", and then call it independently of the original portmaster. As usual, I make no claim that this is the best, only, or most elegant way to do this. Regards, b. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Questions on portmaster
On Wed, Jul 01, 2009 at 11:32:47AM +0530, Manish Jain wrote: > > Hi, > > I am using portmaster on FreeBSD-7.2 to keep my ports up to date. When > there are multiple ports to be updated, portmaster will fork off child > processes. Does each child start 'make fetch' in the background ? The fetches are not done in parallel. > I ask > this because the parent process attached to the console reports very low > data transfer rates (in the range of 0.7 to 4.0 kBps, while normally I > get 30 to 35 kBps). This is not enough data to ascertain there is a problem with fetch. If you are fetching from a busy site, or via a busy upstream channel it can be slow. > Further, how can portmaster be tuned to automatically ignore ports which > are actually marked as IGNORE in the port directory ? touch /var/db/pkg//+IGNOREME > This is not covered in the manpage. It is. Look in the FILES section of portmaster(8). Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgp4qK8UEAKmn.pgp Description: PGP signature
Re: Questions on portmaster
On Wed, Jul 01, 2009 at 11:14:10AM -0400, mfv wrote: > On Wednesday, 1 July 2009 02:02:47 Manish Jain wrote: > excluded. Nevertheless, I use portmaster daily and am very pleased with it. > > With thanks to Doug Barton. portmaster rules, excellent tool, Doug is a star! -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 928 8233 Fax: +44 (0)117 929 4423 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Questions on portmaster
On Wednesday, 1 July 2009 02:02:47 Manish Jain wrote: > Hi, > > I am using portmaster on FreeBSD-7.2 to keep my ports up to date. When > there are multiple ports to be updated, portmaster will fork off child > processes. Does each child start 'make fetch' in the background ? I ask > this because the parent process attached to the console reports very low > data transfer rates (in the range of 0.7 to 4.0 kBps, while normally I > get 30 to 35 kBps). > > Further, how can portmaster be tuned to automatically ignore ports which > are actually marked as IGNORE in the port directory ? This is not > covered in the manpage. > > While doing portmaster -a, I have only managed to stop portmaster from > building exactly one port specified with the -x option. Can I get to > stop multiple ports from being built ? Is there regular expression > support for the -x option ? > > Thanks for any help. Hello Manish, Regarding your last point about excluding particular ports I always thought that it would be convenient to have a section in /usr/local/etc/portmaster.rc and/or $HOME/.portmasterrc to identify these particular ports. I prefer to have all this information in one place rather than scattered over many subdirectories. I had the same problem with trying to have multiple instances of "-x" but gave up and used +IGNOREME instead. Some time ago I succeeded with a regular expression but it became overly complicated if more than two ports were to be excluded. Nevertheless, I use portmaster daily and am very pleased with it. With thanks to Doug Barton. Cheers... Marek ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Questions on portmaster
Hi, I am using portmaster on FreeBSD-7.2 to keep my ports up to date. When there are multiple ports to be updated, portmaster will fork off child processes. Does each child start 'make fetch' in the background ? I ask this because the parent process attached to the console reports very low data transfer rates (in the range of 0.7 to 4.0 kBps, while normally I get 30 to 35 kBps). Further, how can portmaster be tuned to automatically ignore ports which are actually marked as IGNORE in the port directory ? This is not covered in the manpage. While doing portmaster -a, I have only managed to stop portmaster from building exactly one port specified with the -x option. Can I get to stop multiple ports from being built ? Is there regular expression support for the -x option ? Thanks for any help. -- Regards Manish Jain invalid.poin...@gmail.com +91-96500-10329 Laast year I kudn't spell Software Engineer. Now I are won. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"