Angelo Rajadurai wrote: > Looks like pkg.depotd hangs in accept() after uploading 1500+ pkgs > using solaris.py. This seems to happen pretty consistently. The hang > happens at different pkgs but almost always after uploading a lot > of packages. A restart of pkg.depotd seems to clear this up but has > other unwanted effects like reseting the "npkg" value in attr file etc. > > I have a system in this state right now in SWAN if anyone > is interested in seeing the hang. Meanwhile here is some info. > > Stack trace on pkg.depotd > > fecb28d7 accept (4, 80474f0, 8047600, 1) > feb94430 sock_accept (8338bb8, 0) + 64 > fef2456a call_function (80476c4, 0) + 32a > fef2000b PyEval_EvalFrame (819914c) + bdf > fef2474c fast_function (8155ca4, 80477d4, 1, 1, 0) + a8 > fef2431a call_function (80477d4, 0) + da > fef2000b PyEval_EvalFrame (80b3684) + bdf > fef2474c fast_function (8103684, 80478e4, 1, 1, 0) + a8 > fef2431a call_function (80478e4, 0) + da > fef2000b PyEval_EvalFrame (80c04ec) + bdf > fef2474c fast_function (81033e4, 80479f4, 1, 1, 0) + a8 > fef2431a call_function (80479f4, 0) + da > fef2000b PyEval_EvalFrame (80d7174) + bdf > fef2474c fast_function (81033ac, 8047b04, 1, 1, 0) + a8 > fef2431a call_function (8047b04, 0) + da > fef2000b PyEval_EvalFrame (81190e4) + bdf > fef2334e PyEval_EvalCodeEx (8153520, 8079824, 8079824, 0, 0, 0) + 732 > fef1f38e PyEval_EvalCode (8153520, 8079824, 8079824) + 22 > fef3ee85 run_node (80613c8, 8047e78, 8079824, 8079824, 8047d5c) + 35 > fef3ee47 run_err_node (80613c8, 8047e78, 8079824, 8079824, 8047d5c) > + 1f > fef3ee1e PyRun_FileExFlags (fed48e30, 8047e78, 101, 8079824, > 8079824, 1) + 5e > fef3e1af PyRun_SimpleFileExFlags (fed48e30, 8047e78, 1, 8047d5c) + 12f > fef3dcba PyRun_AnyFileExFlags (fed48e30, 8047e78, 1, 8047d5c) + 56 > fef44212 Py_Main (6, 8047db0) + 75a > 08050a19 main (6, 8047db0, 8047dcc) + 11 > 0805097a _start (6, 8047e68, 8047e78, 8047e8c, 8047e8f, 8047e92) + > 7a > > > Stack trace of Solaris.py > fecb2947 recv (6, 250df314, 1, 0) > fe9d4d11 sock_recv (23ccb020, 24edf86c) + 81 > feef42ab PyCFunction_Call (8303f0c, 24edf86c, 0) + 15f > fef24646 call_function (8045b64, 1) + 406 > fef2000b PyEval_EvalFrame (8227e6c) + bdf > fef2334e PyEval_EvalCodeEx (815fb60, 81424f4, 0, 822fa78, 1, > 822fa7c) + 732 > fef247b6 fast_function (816902c, 8045ce4, 1, 1, 0) + 112 > fef2431a call_function (8045ce4, 0) + da > fef2000b PyEval_EvalFrame (822f914) + bdf > fef2474c fast_function (81b7b1c, 8045df4, 1, 1, 0) + a8 > fef2431a call_function (8045df4, 0) + da > fef2000b PyEval_EvalFrame (81350dc) + bdf > fef2474c fast_function (81b7b54, 8045f04, 1, 1, 0) + a8 > fef2431a call_function (8045f04, 0) + da > fef2000b PyEval_EvalFrame (813125c) + bdf > fef2474c fast_function (81ba064, 8046014, 1, 1, 0) + a8 > fef2431a call_function (8046014, 0) + da > fef2000b PyEval_EvalFrame (8134d84) + bdf > fef2474c fast_function (831387c, 8046124, 3, 3, 0) + a8 > fef2431a call_function (8046124, 2) + da > fef2000b PyEval_EvalFrame (8200d4c) + bdf > fef2334e PyEval_EvalCodeEx (81c2660, 81b5e84, 0, 23a33eb8, 2, 0) + 732 > feee5f22 function_call (83138b4, 23a33eac, 0) + 15e > feed5591 PyObject_Call (83138b4, 23a33eac, 0) + 1d > fef24ba3 ext_do_call (83138b4, 80462b8, 1, 1, 0) + fb > fef1ff7b PyEval_EvalFrame (80b06f4) + b4f > fef2334e PyEval_EvalCodeEx (81b0ea0, 81b5e84, 0, 82049d8, 5, > 82049ec) + 732 > fef247b6 fast_function (830ee2c, 8046434, 5, 5, 0) + 112 > fef2431a call_function (8046434, 4) + da > fef2000b PyEval_EvalFrame (8204874) + bdf > fef2334e PyEval_EvalCodeEx (81b0fa0, 81b5e84, 0, 80ae3c8, 3, > 80ae3d4) + 732 > fef247b6 fast_function (830ee9c, 80465b4, 3, 3, 0) + 112 > fef2431a call_function (80465b4, 2) + da > fef2000b PyEval_EvalFrame (80ae254) + bdf > fef2334e PyEval_EvalCodeEx (81b0f60, 81b5e84, 0, 23a8474c, 2, > 23a84754) + 732 > fef247b6 fast_function (830ee64, 8046734, 2, 2, 0) + 112 > fef2431a call_function (8046734, 1) + da > fef2000b PyEval_EvalFrame (23a845bc) + bdf > fef2334e PyEval_EvalCodeEx (81a3820, 830c68c, 0, 80c066c, 4, > 80c067c) + 732 > fef247b6 fast_function (81c11b4, 80468b4, 4, 4, 0) + 112 > fef2431a call_function (80468b4, 4) + da > fef2000b PyEval_EvalFrame (80c04ec) + bdf > fef2334e PyEval_EvalCodeEx (8185ce0, 819d604, 0, 80d7350, 4, > 80d7360) + 732 > fef247b6 fast_function (8313a3c, 8046a34, 4, 4, 0) + 112 > fef2431a call_function (8046a34, 3) + da > fef2000b PyEval_EvalFrame (80d7174) + bdf > fef2334e PyEval_EvalCodeEx (81dae20, 8079824, 0, 811923c, 1, > 8119240) + 732 > fef247b6 fast_function (831409c, 8046bb4, 1, 1, 0) + 112 > fef2431a call_function (8046bb4, 1) + da > fef2000b PyEval_EvalFrame (81190e4) + bdf > fef2334e PyEval_EvalCodeEx (81e3220, 8079824, 8079824, 0, 0, 0) + 732 > fef1f38e PyEval_EvalCode (81e3220, 8079824, 8079824) + 22 > fef3ee85 run_node (80613c8, 8047164, 8079824, 8079824, 8046e0c) + 35 > fef3ee47 run_err_node (80613c8, 8047164, 8079824, 8079824, 8046e0c) > + 1f > fef3ee1e PyRun_FileExFlags (fed48e30, 8047164, 101, 8079824, > 8079824, 1) + 5e > fef3e1af PyRun_SimpleFileExFlags (fed48e30, 8047164, 1, 8046e0c) + 12f > fef3dcba PyRun_AnyFileExFlags (fed48e30, 8047164, 1, 8046e0c) + 56 > fef44212 Py_Main (91, 8046e58) + 75a > 08050a19 main (91, 8046e58, 80470a0) + 11 > 0805097a _start (91, 8047154, 8047164, 8047171, 8047174, 8047179) > + 7a > > pfiles on the pkg.depotd for fd 4 is > 4: S_IFSOCK mode:0666 dev:306,0 ino:44657 uid:0 gid:0 size:0 > O_RDWR > SOCK_STREAM > SO_REUSEADDR,SO_SNDBUF(49152),SO_RCVBUF(49152) > sockname: AF_INET 0.0.0.0 port: 80 > > pfiles on the solaris.py for fd 6 is > 6: S_IFSOCK mode:0666 dev:306,0 ino:59022 uid:0 gid:0 size:0 > O_RDWR > SOCK_STREAM > SO_SNDBUF(49152),SO_RCVBUF(49152) > sockname: AF_INET 127.0.0.1 port: 52132 > peername: AF_INET 127.0.0.1 port: 80 > > > > I tried to dtrace it with something like > > dtrace -n python9850::: > > but this just hangs so looks like its really stuck in accept. > > Unfortunately python is not built with debug on so unable to get a > python stack trace from gdb. > > Let me know if you need more info > > -Angelo > > _______________________________________________ > pkg-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
Is this happening w/ a local repository? Eg is solaris.py running on same machine as depot? We've seen this hang on fast hardware, but only when both were on the same machine, all pkgs were local, etc. My initial guess is that you're running out of temporary port space, since solaris.py puts one file per http connection, and the error handling doesn't do anything useful. The fact that this doesn't happen when you slow things down is interesting as well.. - Bart -- Bart Smaalders Solaris Kernel Performance [EMAIL PROTECTED] http://blogs.sun.com/barts "You will contribute more with mercurial than with thunderbird". _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
