-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 28.05.2013 22:24, schrieb Lars Engels:
> Someone in this thread proposed to change the port to use phttpget, so I > gave it a try using a German mirror nearby with 6 Mbit/s downlink: > > $ time /usr/libexec/phttpget ftp.vim.ossmirror.de $(eval echo > /pub/vim/patches/7.3/{$(make -C /usr/ports/editors/vim -VPATCHFILES | sed > 's/\ /,/g')}) > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.001: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.002: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.003: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.004: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.005: 200 OK > [...] > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.974: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.984: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.985: 200 OK > http://ftp.vim.ossmirror.de//pub/vim/patches/7.3/7.3.1000: 200 OK > > real 0m12.509s > user 0m0.154s > sys 0m0.089s > > > That's really nice! > > Compare this to the current version using fetch(1): This incurs massive overhead from the generation of the fetch URLs, on top of "fetch" being slower. You would have to come up with something similar to your phttpget line, only that it runs fetch(1) instead. Try: $ make -C /usr/ports/editors/vim fetch FETCH_CMD=true to see how slow URL generation is. On one of my reasonably fast amd64 machines (dual-core 2.1 GHz AMD BE-2350), residing on the T-Home domestic backbone near the Ruhr area in Germany with a 3000/384 kBit/s ADSL that has roughly 50 ms ping round trips, it generates approximately 1.7 fetch commands in 1 s, and see below for fetch speed: > time make PATCH_SITES=http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/ fetch > ===> Found saved configuration for vim-7.3.669_1 > ===> vim-7.3.1014 depends on file: /usr/local/sbin/pkg - found > => 7.3.002 doesn't seem to exist in /usr/ports/distfiles/vim. > => Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.002 > 7.3.002 100% of 1610 B 16 MBps 00m00s > => 7.3.003 doesn't seem to exist in /usr/ports/distfiles/vim. > => Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.003 > 7.3.003 100% of 1299 B 1281 kBps 00m00s > => 7.3.004 doesn't seem to exist in /usr/ports/distfiles/vim. > [...] > => 7.3.984 doesn't seem to exist in /usr/ports/distfiles/vim. > => Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.984 > 7.3.984 100% of 1706 B 2852 kBps 00m00s > => 7.3.985 doesn't seem to exist in /usr/ports/distfiles/vim. > => Attempting to fetch http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.985 > 7.3.985 100% of 1691 B 14 MBps 00m00s > => 7.3.1000 doesn't seem to exist in /usr/ports/distfiles/vim. > => Attempting to fetch > http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/7.3.1000 > 7.3.1000 100% of 1637 B 1715 kBps 00m00s > ===> Fetching all distfiles required by vim-7.3.1014 for building > > Total time : 3:48.55s > CPU utilisation (percentage) : 54.5% Now, using a similar approach as yours, I get, with fetch: $ time fetch $(eval echo -O http://ftp.vim.ossmirror.de/pub/vim/patches/7.3/{$(make -C /usr/ports/editors/vim -VPATCHFILES | tr ' ' ',')}) ... real 2m2.841s user 0m1.083s sys 0m5.238s Which takes some 2 min of wasted CPU time out of your calculation. Not that it helps overly in terms of wallclock time. Using curl with -O cuts this down to: real 1m5.195s user 0m0.668s sys 0m1.479s And wget -nv to: real 1m15.426s user 0m0.845s sys 0m1.536s Finally, so you can compare gains whilst taking line speed out of the picture, phttpget: real 0m22.673s user 0m0.530s sys 0m0.659s The computer on its link should be able to fetch a tarball in c. 11 s wallclock time, we have 3.7 MB of payload in patches and the link manages roughly 355 kB/s. Now, any takers for "make make fetch more efficient" jobs? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGlHjAACgkQvmGDOQUufZWaNgCgjERowRAqRo2Rgv8rQOZQiA7f ahcAoMkjJxeFHGBe7hcZIjZb8rrLh+0/ =XoQd -----END PGP SIGNATURE----- _______________________________________________ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"