On 05/17/12 05:43 AM, Shawn Walker wrote:
On 05/15/12 18:47, Tim Foster wrote:
https://cr.opensolaris.org/action/browse/pkg/timf/sysrepo-p5p/sysrepo-p5p-pkg-webrev-v4
...
Despite being slower than the equivalent file-based repository installs,
we never hit any of the default low-speed timeouts that pkg(1) enforces,
the downloads completed for all clients, and we didn't get errors in the
server-side logs.
What were the PKG_DUMP_STATS=1 output of each run for comparison? (That
is, the best of three runs with a file-repo, and the best of three runs
with a p5p archive?)
Here we go: I also included the total 'real' ptime for each of the
packaging commands as the PKG_DUMP_STATS figures seemed a bit odd.
I've included the results in the table attached - I'm slightly confused
by the numbers to be honest[1], but they seem to show that (at least
relative to the other publisher configurations) p5p performance through
the system repository is acceptable.
cheers,
tim
[1] I'd have thought the file:// repository being accessed directly
would have been much faster than it appeared to be.
Investigating the performance of the p5p system-repository support
==================================================================
For several freshly created user-images, we ran the following command, with
varying publisher configurations on .
ptime pkg -R <image> install --accept solaris-large-server
All of these were with clients using the system repository. In all cases, the
system-repository proxy cache was turned off. The repository was hosted on
the same machine, on the same zpool, but in a different dataset to the client
images. Between each run, the client-image dataset was destroyed and recreated.
The client publisher configurations were as follows:
Run ID Description
------ -----------
1.p5p a p5p archive accessed through the sysrepo
2.file a file repository accessed through the sysrepo
3.http a http repository on localhost accessed through the sysrepo
4.http-remote a http repository on ipkg, accessed through the sysrepo
5.file-direct a file repository, accessed directly by the client image
The table below shows a summary of the results including the ptime.
Run ID Transfer in kB/s ptime of pkg command
------ ---------------- --------------------
1.p5p 284 11:29
2.file 749 8:52
3.http 27 (?) 35:05 (18:27 on a subsequent run, and
95kB/s)
4.http-remote 199 12:33
5.file-direct ? 12:28 (PKG_DUMP_STATS didn't give
statistics for the file transfer,
but I find it odd we're slower
than 2.file above)
Detailed results:
=================
These are the ptime outputs, along with PKG_DUMP_STATS outputs for the various
runs.
1.p5p
-----
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 2 0 1 40 kB/s 694.00 B True
0.012307 3277
http://localhost:1008/solaris/6 484 0 21 284 kB/s 23.80 MB True
0.002813 25
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60672/60672 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 0 0 1 0 B/s 0.00 B True
0.012307 4001
real 11:29.582880667
user 4:45.931340923
sys 3:18.075621072
2.file
------
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008/solaris/c 484 0 22 5 MB/s 23.80 MB True
0.004184 31
http://localhost:1008 2 0 1 719 kB/s 694.00 B True
0.001131 4493
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60672/60672 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008/solaris/c 50285 0 528 549 kB/s 666.67 MB True
0.001895 37
http://localhost:1008 0 0 1 0 B/s 0.00 B True
0.001131 3988
real 8:52.145563307
user 2:44.397277647
sys 2:21.771420486
3.http
------
(the pkg server process was
/usr/lib/pkg.depotd -p 10000 -s 21 --readonly -d `pwd` --log-access none
again, proxied through the system repository)
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 2 0 1 59 kB/s 693.00 B True
0.001136 3265
http://localhost:10000 484 0 23 98 kB/s 23.80 MB True
0.008635 29
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60672/60672 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 0 0 1 0 B/s 0.00 B True
0.001136 4005
http://localhost:10000 50285 0 529 27 kB/s 666.67 MB True
0.350461 46
real 35:06.512632453
user 17:27.777784123
sys 7:31.085653694
Trying again, this time with the system-repository set to proxy to the
depot using the hostname of the machine, rather than localhost, our numbers
improve somewhat.
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 2 0 1 656 kB/s 699.00 B True
0.001041 4407
http://x4150-sfb-18:10000 481 0 21 293 kB/s 19.06 MB True
0.011987 59
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60672/60672 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
http://localhost:1008 0 0 1 0 B/s 0.00 B True
0.001041 3988
http://x4150-sfb-18:10000 50285 0 526 95 kB/s 666.67 MB True
0.001963 49
real 18:27.970061100
user 4:59.923389567
sys 3:31.861805841
4.http-remote
-------------
Those numbers seemed odd - I would expect overhead running the repository
on the same machine as the client, but not that low. Out of curiousity, I tried
with the solaris11/dev repository on our internal package server (again,
accessed through, the system repository) installing the same version of
solaris-large-server and got:
URL Good Err Conn Speed Size Used
CSpeed Qual
http://ipkg.us.oracle.com/solar 484 0 24 558 kB/s 12.04 MB True
0.003038 22
http://localhost:1008 2 0 1 438 kB/s 727.00 B True
0.001195 3985
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60703/60703 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
http://ipkg.us.oracle.com/solar 50304 0 530 199 kB/s 666.62 MB True
0.001308 14
http://localhost:1008 0 0 1 0 B/s 0.00 B True
0.001195 3997
real 12:33.656075943
user 5:26.032127636
sys 3:12.166610327
The numbers are better, I assume because of no overhead in serving the content.
5.file-direct
-------------
Just for reference, I thought I'd also do a test of a pkg install *without*
the overhead of the system repository, so installing directly from a local file
repository, we get the following.
Note that the statistics from PKG_DUMP_STATS seem odd:
URL Good Err Conn Speed Size Used
CSpeed Qual
file:///space/timf/repos/solari 484 0 0 7 MB/s 23.80 MB True
0.000000 155
DOWNLOAD PKGS FILES XFER (MB)
Completed 470/470 60672/60672 682.3/682.3
URL Good Err Conn Speed Size Used
CSpeed Qual
file:///space/timf/repos/solari 0 0 0 0 B/s 0.00 B True
0.000000 4020
PHASE ACTIONS
Install Phase 84068/85947
real 12:28.391991027
user 2:06.260998138
sys 1:24.240352222
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
