Re: setup.ini dependency graph?
Charles Wilson writes: > On 11/4/2013 11:00 AM, Christopher Faylor wrote: >> On Mon, Nov 04, 2013 at 09:32:10AM -0500, Charles Wilson wrote: >>> I've got a few cleanups, and then I'll share the result. It's already >>> helped me generate a few re-packaging requests I plan to post over on >>> cygwin-apps... >> >> Is this packagable? It sounds pretty interesting. > > Probably. I could put it in cygutils, or standalone (like the new > cygcheck-leaves package is a standalone utility). Interesting, indeed. I've been working on another script that creates install directories for offline installation (using multiple package repositories and a config file to control package selection) that I hope to get polished for public release. It doesn't explicitly produce a dependency graph (I do have some optional debug output that could be used for this), but it wouldn't be difficult to bolt on (I already have a hash for the forward dependencies). >> Would it be crazy to generate this and make it available on the cygwin >> web site? Or would the dependency graph generation overload >> sourceware.org? > > The basic processing to generate a .dot file is pretty simple really; > just string comparisons and hash manipulations. But as Ryan already > pointed out, generating the actual graph in whatever format, is > probably compute intensive. It might be easier to keep things a bit more simple: the most common question is to get the direct dependencies of a package and the direct dependents (who is requiring this package). This graph is almost trivial and could be followed interactively up or down the dependency chain with little effort. It wouldn't look as pretty as a full dependency graph on an 8k display, but it'd be very useful. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 04/11/13 17:30, Charles Wilson wrote: On 11/4/2013 11:00 AM, Christopher Faylor wrote: On Mon, Nov 04, 2013 at 09:32:10AM -0500, Charles Wilson wrote: I've got a few cleanups, and then I'll share the result. It's already helped me generate a few re-packaging requests I plan to post over on cygwin-apps... Is this packagable? It sounds pretty interesting. Probably. I could put it in cygutils, or standalone (like the new cygcheck-leaves package is a standalone utility). Would it be crazy to generate this and make it available on the cygwin web site? Or would the dependency graph generation overload sourceware.org? The basic processing to generate a .dot file is pretty simple really; just string comparisons and hash manipulations. But as Ryan already pointed out, generating the actual graph in whatever format, is probably compute intensive. See attached... My goodness - you have been busy! You've taken some hacky kludgy perl that I threw together in five minutes flat and turned it into a proper programme. Tip of the hat to you, sir... Dave. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 04/11/2013 11:00 AM, Christopher Faylor wrote: On Mon, Nov 04, 2013 at 09:32:10AM -0500, Charles Wilson wrote: On 10/30/2013 9:51 AM, Ryan Johnson wrote: On 30/10/2013 8:48 AM, Charles Wilson wrote: Yeah; even for my stripped-down version, I need to pre-process the setup.ini and remove all mentions of cygwin, libstdc++6, libgcc1, etc. The ncurses DLLs are also a huge nexus. (It's probably easier to exclude those nodes by mucking with the perl, but...) Quick question: do you have 1+ known-big-unwanted packages and need to know who's pulling them in, or are you hoping to take some cut of the graph that gets as many desirable packages as possible given the space constraints? The graph-building script here is good for the latter, but I had the impression you were doing the former; if so, my script might get you to an answer faster by avoiding information overload. A combination of the two, actually. I've used both David's script and yours in concert. In addition, I've modified David's script to color the nodes based on origination, and to exclude or collapse 'Base' and/or 'required-by-Base' packages. I've got a few cleanups, and then I'll share the result. It's already helped me generate a few re-packaging requests I plan to post over on cygwin-apps... Is this packagable? It sounds pretty interesting. Would it be crazy to generate this and make it available on the cygwin web site? Or would the dependency graph generation overload sourceware.org? Throwing graphviz at a full cygwin package dependency graph would make a pretty effective DoS attack. Smaller graphs are cheaper, but still consume non-trivial compute. Given how slowly the online regexp package search goes, I'd hesitate to give users more ways to overload the server... What about calling out to graphviz from setup.exe (if found in %PATH%), as a replacement/supplement for the flat list of dependencies it currently reports? That would put all processing on the client, and limit the "big data" problem, as the graph only contains packages a user is currently trying to install. Alternatively, cygcheck could gain a new -g option to dump subsets of a dependency graph, extracted from setup.ini, in some appropriate format like .dot: `cygcheck -D -g python' would emit the graph of packages that python depends on `cygcheck -R -g texlive,xorg-server' would build a braph of packages that pull in either of texlive or xorg-server (reverse dependencies) -D -R -g would follow dependencies in both directions, and -g would be shorthand for -D -g; probably -D or -R by itself implies -g. The actual work could be done by calling out to a scripting language that ships with cygwin. Awk would probably be able to, and perl certainly could. /daydreaming Ryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On Mon, Nov 04, 2013 at 09:32:10AM -0500, Charles Wilson wrote: >On 10/30/2013 9:51 AM, Ryan Johnson wrote: >> On 30/10/2013 8:48 AM, Charles Wilson wrote: >>> Yeah; even for my stripped-down version, I need to pre-process the >>> setup.ini and remove all mentions of cygwin, libstdc++6, libgcc1, etc. >>> The ncurses DLLs are also a huge nexus. (It's probably easier to >>> exclude those nodes by mucking with the perl, but...) >> Quick question: do you have 1+ known-big-unwanted packages and need to >> know who's pulling them in, or are you hoping to take some cut of the >> graph that gets as many desirable packages as possible given the space >> constraints? The graph-building script here is good for the latter, but >> I had the impression you were doing the former; if so, my script might >> get you to an answer faster by avoiding information overload. > >A combination of the two, actually. I've used both David's script and >yours in concert. In addition, I've modified David's script to color >the nodes based on origination, and to exclude or collapse 'Base' and/or >'required-by-Base' packages. > >I've got a few cleanups, and then I'll share the result. It's already >helped me generate a few re-packaging requests I plan to post over on >cygwin-apps... Is this packagable? It sounds pretty interesting. Would it be crazy to generate this and make it available on the cygwin web site? Or would the dependency graph generation overload sourceware.org? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 10/30/2013 9:51 AM, Ryan Johnson wrote: On 30/10/2013 8:48 AM, Charles Wilson wrote: Yeah; even for my stripped-down version, I need to pre-process the setup.ini and remove all mentions of cygwin, libstdc++6, libgcc1, etc. The ncurses DLLs are also a huge nexus. (It's probably easier to exclude those nodes by mucking with the perl, but...) Quick question: do you have 1+ known-big-unwanted packages and need to know who's pulling them in, or are you hoping to take some cut of the graph that gets as many desirable packages as possible given the space constraints? The graph-building script here is good for the latter, but I had the impression you were doing the former; if so, my script might get you to an answer faster by avoiding information overload. A combination of the two, actually. I've used both David's script and yours in concert. In addition, I've modified David's script to color the nodes based on origination, and to exclude or collapse 'Base' and/or 'required-by-Base' packages. I've got a few cleanups, and then I'll share the result. It's already helped me generate a few re-packaging requests I plan to post over on cygwin-apps... -- Chuck -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 30/10/2013 8:48 AM, Charles Wilson wrote: On 10/26/2013 5:40 AM, David Stacey wrote: On 25/10/13 17:12, Charles Wilson wrote: Oooo - this sounds like fun. I've knocked up some (very bad) perl that gives you what you need. It generates a graphviz file that you can pipe to 'dot' to generate the dependency graph in whatever format you require. Put the perl script and your 'setup.ini' file in the same directory and type: ./graph_setup_ini.pl | dot -Tpdf -osetup.pdf Thanks, that worked well. Your problem here is Big Data: Cygwin has 3041 packages, and any dependency graph with this number of nodes is going to look a mess. It also takes a while to process the data. Oh, and some PDF viewers won't display the output file (LibreOffice Draw was the only tool I have that managed it). However, if your starting point is a stripped down Cygwin then you might be OK. Yeah; even for my stripped-down version, I need to pre-process the setup.ini and remove all mentions of cygwin, libstdc++6, libgcc1, etc. The ncurses DLLs are also a huge nexus. (It's probably easier to exclude those nodes by mucking with the perl, but...) Quick question: do you have 1+ known-big-unwanted packages and need to know who's pulling them in, or are you hoping to take some cut of the graph that gets as many desirable packages as possible given the space constraints? The graph-building script here is good for the latter, but I had the impression you were doing the former; if so, my script might get you to an answer faster by avoiding information overload. Ryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 10/26/2013 5:40 AM, David Stacey wrote: On 25/10/13 17:12, Charles Wilson wrote: Oooo - this sounds like fun. I've knocked up some (very bad) perl that gives you what you need. It generates a graphviz file that you can pipe to 'dot' to generate the dependency graph in whatever format you require. Put the perl script and your 'setup.ini' file in the same directory and type: ./graph_setup_ini.pl | dot -Tpdf -osetup.pdf Thanks, that worked well. Your problem here is Big Data: Cygwin has 3041 packages, and any dependency graph with this number of nodes is going to look a mess. It also takes a while to process the data. Oh, and some PDF viewers won't display the output file (LibreOffice Draw was the only tool I have that managed it). However, if your starting point is a stripped down Cygwin then you might be OK. Yeah; even for my stripped-down version, I need to pre-process the setup.ini and remove all mentions of cygwin, libstdc++6, libgcc1, etc. The ncurses DLLs are also a huge nexus. (It's probably easier to exclude those nodes by mucking with the perl, but...) -- Chuck -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: why do psutils & libtool packages need ~25+MB of latex (was Re: setup.ini dependency graph?)
On 10/27/2013 1:00 AM, Linda Walsh wrote: On 10/25/2013 5:29 PM, Ryan Johnson wrote: On 25/10/2013 12:12 PM, Charles Wilson wrote: Does anybody have a script or a tool that can parse a setup.ini and generate a dependency graph? I'm using pmcyg to create a stripped-down standalone installation CD and it's too big, so I'm trying to figure out where the culprit is that's pulling in so much stuff... I threw one together a while back when dealing with packages trying to pull in latex: http://cygwin.com/ml/cygwin/2012-03/msg00242.html --- I keep trying to get rid of latex and it keeps coming back... I think I figured out why... psutils? libtool? If you look at setup.ini, you'll see that psutils is an obsolete package, which has been replaced by texlive-collection-fontutils. You must have once installed psutils for some reason. If you don't need its functionality, uninstall it and all the texlive packages that it brought in. You may have other obsolete packages installed that are also bringing in texlive packages. Just run setup and uncheck the "Hide obsolete packages" box. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
why do psutils & libtool packages need ~25+MB of latex (was Re: setup.ini dependency graph?)
On 10/25/2013 5:29 PM, Ryan Johnson wrote: On 25/10/2013 12:12 PM, Charles Wilson wrote: Does anybody have a script or a tool that can parse a setup.ini and generate a dependency graph? I'm using pmcyg to create a stripped-down standalone installation CD and it's too big, so I'm trying to figure out where the culprit is that's pulling in so much stuff... I threw one together a while back when dealing with packages trying to pull in latex: http://cygwin.com/ml/cygwin/2012-03/msg00242.html --- I keep trying to get rid of latex and it keeps coming back... I think I figured out why... psutils? libtool? Both need latex? I think I'm must really be missing some features in psutils -- and doesn't libtool put together binary libraries? Why would it need latex in order to run? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 25/10/13 17:12, Charles Wilson wrote: Does anybody have a script or a tool that can parse a setup.ini and generate a dependency graph? I'm using pmcyg to create a stripped-down standalone installation CD and it's too big, so I'm trying to figure out where the culprit is that's pulling in so much stuff... Oooo - this sounds like fun. I've knocked up some (very bad) perl that gives you what you need. It generates a graphviz file that you can pipe to 'dot' to generate the dependency graph in whatever format you require. Put the perl script and your 'setup.ini' file in the same directory and type: ./graph_setup_ini.pl | dot -Tpdf -osetup.pdf I did this in Fedora 19; if you want this to work in Cygwin then you'll need to install graphviz from CygwinPorts. Your problem here is Big Data: Cygwin has 3041 packages, and any dependency graph with this number of nodes is going to look a mess. It also takes a while to process the data. Oh, and some PDF viewers won't display the output file (LibreOffice Draw was the only tool I have that managed it). However, if your starting point is a stripped down Cygwin then you might be OK. Hope this helps, Dave. graph_setup_ini.pl Description: Perl program -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: setup.ini dependency graph?
On 25/10/2013 12:12 PM, Charles Wilson wrote: Does anybody have a script or a tool that can parse a setup.ini and generate a dependency graph? I'm using pmcyg to create a stripped-down standalone installation CD and it's too big, so I'm trying to figure out where the culprit is that's pulling in so much stuff... I threw one together a while back when dealing with packages trying to pull in latex: http://cygwin.com/ml/cygwin/2012-03/msg00242.html It doesn't build a full graph, but if you name the offending package it tells you who depends on it. HTH, Ryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
