Greg,
I was mistaken. It wasn't a GET, it was a POST. Here's the POST
request:
POST /admin_toolshed/prepare_for_install?
includes_tools_for_display_in_tool_panel=True&encoded_repo_info_dicts=72
2fdbb60840f8caa6f7dd248a9f08b927df4325%
3A7b22746573745f6c6f6e675f757269223a205b227465737420746f20726570726f6475
6365205c2272657175657374206661696c65643a2055524920746f6f206c6f6e6720286c
6f6e676572207468616e2038313930295c22222c2022687474703a2f2f64657673686564
2e62782e7073752e6564752f7265706f732f7269636f2f746573745f6c6f6e675f757269
222c2022353836396263663736653838222c202230222c20227269636f222c207b226465
736372697074696f6e223a206e756c6c2c2022687474703a2f2f646576736865642e6278
2e7073752e6564755f5f455345505f5f7061636b6167655f70686173745f315f335f5f45
5345505f5f7269636f5f5f455345505f5f6339626465626330376566615f5f455345505f
5f54727565223a205b5b22687474703a2f2f646576736865642e62782e7073752e656475
2f222c20227061636b6167655f636c617061636b5f335f325f31222c20227269636f222c
2022323366366135663363366466222c202254727565225d5d2c2022687474703a2f2f64
6576736865642e62782e7073752e6564755f5f455345505f5f7061636b6167655f666973
6865725f305f315f345f5f455345505f5f7269636f5f5f455345505f5f66303165643539
36393339345f5f455345505f5f54727565223a205b5b22687474703a2f2f646576736865
642e62782e7073752e6564752f222c20227061636b6167655f6e756d70795f315f37222c
20227269636f222c2022353231356136303134303666222c202254727565225d5d2c2022
687474703a2f2f646576736865642e62782e7073752e6564755f5f455345505f5f706163
6b6167655f6d6174706c6f746c69625f315f325f5f455345505f5f7269636f5f5f455345
505f5f3039303562393835323961345f5f455345505f5f54727565223a205b5b22687474
703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167655f6e
756d70795f315f37222c20227269636f222c2022353231356136303134303666222c2022
54727565225d2c205b22687474703a2f2f646576736865642e62782e7073752e6564752f
222c20227061636b6167655f66726565747970655f325f34222c20227269636f222c2022
346238383437333030643731222c202254727565225d5d2c2022726f6f745f6b6579223a
2022687474703a2f2f646576736865642e62782e7073752e6564755f5f455345505f5f74
6573745f6c6f6e675f7572695f5f455345505f5f7269636f5f5f455345505f5f35383639
62636637366538385f5f455345505f5f46616c7365222c2022687474703a2f2f64657673
6865642e62782e7073752e6564755f5f455345505f5f746573745f6c6f6e675f7572695f
5f455345505f5f7269636f5f5f455345505f5f3538363962636637366538385f5f455345
505f5f46616c7365223a205b5b22687474703a2f2f646576736865642e62782e7073752e
6564752f222c20227061636b6167655f62656175746966756c736f75705f335f325f3122
2c20227269636f222c2022356231666636653736323961222c202254727565225d2c205b
22687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b61
67655f656967656e736f66745f355f305f31222c20227269636f222c2022356538306361
383230666439222c202254727565225d2c205b22687474703a2f2f646576736865642e62
782e7073752e6564752f222c20227061636b6167655f6669736865725f305f315f34222c
20227269636f222c2022663031656435393639333934222c202254727565225d2c205b22
687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167
655f67645f635f746f6f6c735f305f31222c20227269636f222c20226436656335383933
34323263222c202254727565225d2c205b22687474703a2f2f646576736865642e62782e
7073752e6564752f222c20227061636b6167655f6d6174706c6f746c69625f315f32222c
20227269636f222c2022303930356239383532396134222c202254727565225d2c205b22
687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167
655f6d656368616e697a655f305f325f35222c20227269636f222c202234346139303064
6638303964222c202254727565225d2c205b22687474703a2f2f646576736865642e6278
2e7073752e6564752f222c20227061636b6167655f6d756e6b7265735f315f305f355f34
222c20227269636f222c2022346663366163336137643736222c202254727565225d2c20
5b22687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b
6167655f6e6574776f726b785f315f385f31222c20227269636f222c2022633565623436
373265323163222c202254727565225d2c205b22687474703a2f2f646576736865642e62
782e7073752e6564752f222c20227061636b6167655f70686173745f315f33222c202272
69636f222c2022633962646562633037656661222c202254727565225d2c205b22687474
703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167655f71
7569636b747265655f315f31222c20227269636f222c2022373235353762356166656639
222c202254727565225d2c205b22687474703a2f2f646576736865642e62782e7073752e
6564752f222c20227061636b6167655f7261786d6c5f375f375f36222c20227269636f22
2c2022393565623066343665656263222c202254727565225d5d7d2c207b227068617374
2f312e33223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e
675f757269222c20226e616d65223a20227068617374222c202276657273696f6e223a20
22312e33222c20227265706f7369746f72795f6f776e6572223a20227269636f222c2022
6368616e67657365745f7265766973696f6e223a2022353836396263663736653838222c
202274797065223a20227061636b616765227d2c20227261786d6c2f372e372e36223a20
7b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269222c
20226e616d65223a20227261786d6c222c202276657273696f6e223a2022372e372e3622
2c20227265706f7369746f72795f6f776e6572223a20227269636f222c20226368616e67
657365745f7265766973696f6e223a2022353836396263663736653838222c2022747970
65223a20227061636b616765227d2c20226d756e6b7265732f312e302e352e34223a207b
227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269222c20
226e616d65223a20226d756e6b726573222c202276657273696f6e223a2022312e302e35
2e34222c20227265706f7369746f72795f6f776e6572223a20227269636f222c20226368
616e67657365745f7265766973696f6e223a2022353836396263663736653838222c2022
74797065223a20227061636b616765227d2c20226d656368616e697a652f302e322e3522
3a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269
222c20226e616d65223a20226d656368616e697a65222c202276657273696f6e223a2022
302e322e35222c20227265706f7369746f72795f6f776e6572223a20227269636f222c20
226368616e67657365745f7265766973696f6e223a202235383639626366373665383822
2c202274797065223a20227061636b616765227d2c20226e6574776f726b782f312e382e
31223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f75
7269222c20226e616d65223a20226e6574776f726b78222c202276657273696f6e223a20
22312e382e31222c20227265706f7369746f72795f6f776e6572223a20227269636f222c
20226368616e67657365745f7265766973696f6e223a2022353836396263663736653838
222c202274797065223a20227061636b616765227d2c20226d6174706c6f746c69622f31
2e322e31223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e
675f757269222c20226e616d65223a20226d6174706c6f746c6962222c20227665727369
6f6e223a2022312e322e31222c20227265706f7369746f72795f6f776e6572223a202272
69636f222c20226368616e67657365745f7265766973696f6e223a202235383639626366
3736653838222c202274797065223a20227061636b616765227d2c202262656175746966
756c736f75702f332e322e31223a207b227265706f7369746f72795f6e616d65223a2022
746573745f6c6f6e675f757269222c20226e616d65223a202262656175746966756c736f
7570222c202276657273696f6e223a2022332e322e31222c20227265706f7369746f7279
5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e
223a2022353836396263663736653838222c202274797065223a20227061636b61676522
7d2c202267645f635f746f6f6c732f302e31223a207b227265706f7369746f72795f6e61
6d65223a2022746573745f6c6f6e675f757269222c20226e616d65223a202267645f635f
746f6f6c73222c202276657273696f6e223a2022302e31222c20227265706f7369746f72
795f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f
6e223a2022353836396263663736653838222c202274797065223a20227061636b616765
227d2c2022717569636b747265652f312e31223a207b227265706f7369746f72795f6e61
6d65223a2022746573745f6c6f6e675f757269222c20226e616d65223a2022717569636b
74726565222c202276657273696f6e223a2022312e31222c20227265706f7369746f7279
5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e
223a2022353836396263663736653838222c202274797065223a20227061636b61676522
7d2c20226669736865722f302e312e34223a207b227265706f7369746f72795f6e616d65
223a2022746573745f6c6f6e675f757269222c20226e616d65223a202266697368657222
2c202276657273696f6e223a2022302e312e34222c20227265706f7369746f72795f6f77
6e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e223a20
22353836396263663736653838222c202274797065223a20227061636b616765227d2c20
22656967656e736f66742f352e302e31223a207b227265706f7369746f72795f6e616d65
223a2022746573745f6c6f6e675f757269222c20226e616d65223a2022656967656e736f
6674222c202276657273696f6e223a2022352e302e31222c20227265706f7369746f7279
5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e
223a2022353836396263663736653838222c202274797065223a20227061636b61676522
7d7d5d7d&includes_tools=True&tool_shed_url=http%3A%2F%
2Fdevshed.bx.psu.edu%2F&includes_tool_dependencies=True HTTP/1.1
Here is the POSTed data:
install_repository_dependencies=true&install_repository_dependencies=tru
e&install_tool_dependencies=true&install_tool_dependencies=true&shed_too
l_conf=.%
2Fshed_tool_conf.xml&new_tool_panel_section=Test&select_tool_panel_secti
on_button=Install
The POSTed URI is 8,320 bytes. The encoded_repo_info_dicts account
for 8,113 (97.5%) of these bytes.
In this particular case, it appears that the relevant code can be
found in install_repositories_by_revision() in lib/galaxy/webapps/
tool_shed/controllers/repository.py and in prepare_for_install() in
lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py.
It's possible that there's code elsewhere that also passes the
encoded_repo_info_dicts in the request, which would trigger the same
issue.
-rico
On Sep 26, 2013, at 9:21 AM, Greg Von Kuster wrote:
Hi Rico,
Can you provide the specific GET request URL with which you
encountered this problem? There are very few GET requests when
installing repositories from the tool shed - the only one I can
locate without a lot of digging is the following, which I don't
think is causing the problem.
# Get the information necessary to install each repository.
url = suc.url_join( tool_shed_url,
'repository/get_repository_information?
repository_ids=%s&changeset_revisions=%s' % \
( repository_ids, changeset_revisions ) )
Thanks,
Greg Von Kuster
On Sep 13, 2013, at 10:59 AM, Richard Burhans <r...@bx.psu.edu> wrote:
Folks,
After checking a little closer last night, I have one clarification.
The Tool Shed does use POSTs as well as GETs, but even when it's
POSTing, the encoded dictionaries (encoded_repo_info_dicts) are part
of the query string. The encoded_repo_info_dicts should probably
become part of the POSTed data.
-rico
On Sep 12, 2013, at 3:55 PM, Richard Burhans wrote:
When installing Galaxy Tool Shed repositories into a local Galaxy
instance, tool shed dependecies are exchanged as an encoded python
dictionary via the HTTP GET method.
If a tool shed repository contains enough dependencies, the length
of this encoded dictionary can become extremely long. In my case,
it became long enough for my Apache server to choke on it:
[Wed Sep 11 15:34:53 2013] [error] [client XXX.XXX.XXX.XXX]
request failed: URI too long (longer than 8190)
The HTTP standard does not place a limit on the length of a URI,
but the de facto limit is 2000 characters. See the following
stackoverflow answer:
http://stackoverflow.com/a/417184
I would recommend either:
1) communicating toolshed dependencies via an HTTP POST
2) redesigning tool shed <-> instance communications
As a stopgap measure, I've modified lib/tool_shed/util/
encoding_util.py
to use a more compact encoding scheme. I'm attaching a copy in
case someone else runs into the same problem.
-rico<encoding_util.py>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
