Re: Requires: python(foo) = bar

2012-05-12 Thread Anders F Björklund
Jeffrey Johnson wrote:

 I used http://docs.python.org/library/modulefinder.html
 and then filtered out some boring ones like sys and os.
 
 
 Yes modulefinder included in python is the right starting point.
 
 I'd stub the Provides: rather than filter the Requires: because WYSIWYG
 is far easier maintenance than discussing The One True Meaning of
   boring

Probably a good idea, I think it was only for testing purposes...

Once it works for the python installation, it should be automatic.
(i.e. all those system modules would be provided automatically too)

It was a couple of years ago, so my recollection is a little blurred.

 And the verification could/should/would proceed by capturing reality
 (i.e. overloading builtins.__import__ to log what modules are actually
 loaded for comparison with python(foo) dependencies in *.rpm headers)
 
 Any interest? If so, the starting point would be to find the 2-3 best
 python dependency graphs and have a go with sed(1) on the generated
 *.dot spewage to see how good the dependencies being found actually are.
 
 The --requires and --provides looked reasonable enough,
 but it would need some tweaking and testing before use.
 
 
 Scale of 1 - 10 (10 being perfect) can you rate your  tweaking and testing?

Well, it was a proof-of-concept and as such didn't work as a drop-in.

- It took args rather than stdin, so that should be fixed.
- Only takes one argument, so you had to do e.g. xargs -n 1
- You get duplicated dependencies, if running it in a loop
- Seems to be listing some empty python() dependencies ?

- All dependencies are unversioned, so only works for one python.
  You would need something like python2.6 versus python2.7 etc.
- It also doesn't work for Python 3000, so needs a port there
- And there is no versioning on the python modules themselves

But with the easy ones fixed, it should be comparable with perldeps ?

 There's also http://rpm5.org/cvs/fileview?f=rpm/scripts/pythoneggs.py
 
 
 Yes this too is reasonable but is merely a transform from one sewage to 
 another
 (with the usual impedance match difficulties for tested fubar semantics).
 
 (aside)
 Personally I'd rather see pythoneggs.sh rather than python eggs.py: there's
 nothing in that script that MUST have python afaict.

No, the .egg-info format is probably not so hard to parse explicitly.

 The hardest problem is finding a useful semantic for a python dependency
 graph. Otherwise there's the usual metric tone butt load of
   Disablers! Filtering! Different!
 RFE's that swarm like flies on rotten meat whenever RPM features are 
 suggested.
 
 A parser for *.dot is KISS wrto the bike shed discussion

People were unhappy enough with perldeps, to avoid adding pythondeps.

--anders

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-12 Thread Jeffrey Johnson

On May 12, 2012, at 4:04 AM, Anders F Björklund a...@rpm5.org wrote:

 Jeffrey Johnson wrote:
 
 I used http://docs.python.org/library/modulefinder.html
 and then filtered out some boring ones like sys and os.
 
 
 Yes modulefinder included in python is the right starting point.
 
 I'd stub the Provides: rather than filter the Requires: because WYSIWYG
 is far easier maintenance than discussing The One True Meaning of
  boring
 
 Probably a good idea, I think it was only for testing purposes...
 
 Once it works for the python installation, it should be automatic.
 (i.e. all those system modules would be provided automatically too)
 
 It was a couple of years ago, so my recollection is a little blurred.
 

Heh, mine too. Drilling in
Requires: python(…)
been on the todo++ all century.

 And the verification could/should/would proceed by capturing reality
 (i.e. overloading builtins.__import__ to log what modules are actually
 loaded for comparison with python(foo) dependencies in *.rpm headers)
 
 Any interest? If so, the starting point would be to find the 2-3 best
 python dependency graphs and have a go with sed(1) on the generated
 *.dot spewage to see how good the dependencies being found actually are.
 
 The --requires and --provides looked reasonable enough,
 but it would need some tweaking and testing before use.
 
 
 Scale of 1 - 10 (10 being perfect) can you rate your  tweaking and 
 testing?
 
 Well, it was a proof-of-concept and as such didn't work as a drop-in.
 
 - It took args rather than stdin, so that should be fixed.
 - Only takes one argument, so you had to do e.g. xargs -n 1

lib/rpmfc.c is file-by-file so that dependencies can be attached
to the file they came from, not a problem.

 - You get duplicated dependencies, if running it in a loop

lib/rpmfc.c will do uniqification (and filtering).

 - Seems to be listing some empty python() dependencies ?
 

Easy enough to filter. Likely some python bug that rpm will be
blamed for.

 - All dependencies are unversioned, so only works for one python.
  You would need something like python2.6 versus python2.7 etc.

There's module versioning and then there is python versioning. The
feebleness of
Requires: python(abi) = X.Y
will handle for the next decade when the conversion to python 3.Y
will inevitably be completed.

 - It also doesn't work for Python 3000, so needs a port there
 - And there is no versioning on the python modules themselves
 

Your memory isn't so blurry ;-)

 But with the easy ones fixed, it should be comparable with perldeps ?
 

But not based on *RE's scraping import syntax out of source code.

 There's also http://rpm5.org/cvs/fileview?f=rpm/scripts/pythoneggs.py
 
 
 Yes this too is reasonable but is merely a transform from one sewage to 
 another
 (with the usual impedance match difficulties for tested fubar semantics).
 
 (aside)
 Personally I'd rather see pythoneggs.sh rather than python eggs.py: there's
 nothing in that script that MUST have python afaict.
 
 No, the .egg-info format is probably not so hard to parse explicitly.
 
 The hardest problem is finding a useful semantic for a python dependency
 graph. Otherwise there's the usual metric tone butt load of
  Disablers! Filtering! Different!
 RFE's that swarm like flies on rotten meat whenever RPM features are 
 suggested.
 
 A parser for *.dot is KISS wrto the bike shed discussion
 
 People were unhappy enough with perldeps, to avoid adding pythondeps.
 

There will always be issues with soft dependencies when
module loading happens conditionally on only some code paths.

Thanks for details.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Anders F Björklund
Jeffrey Johnson wrote:

 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.

There was a half-hearted attempt at this, earlier on:
http://afb.users.sourceforge.net/rpm/pythondeps.py

 A google search for python module dependency graph shows
 some reasonably serious efforts to create reliable (i.e. not
 *RE's scraping use … statements out of *.pl) dependencies.

I used http://docs.python.org/library/modulefinder.html
and then filtered out some boring ones like sys and os.


 And the verification could/should/would proceed by capturing reality
 (i.e. overloading builtins.__import__ to log what modules are actually
 loaded for comparison with python(foo) dependencies in *.rpm headers)
 
 Any interest? If so, the starting point would be to find the 2-3 best
 python dependency graphs and have a go with sed(1) on the generated
 *.dot spewage to see how good the dependencies being found actually are.

The --requires and --provides looked reasonable enough,
but it would need some tweaking and testing before use.

There's also http://rpm5.org/cvs/fileview?f=rpm/scripts/pythoneggs.py

--anders

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Jeffrey Johnson

On May 11, 2012, at 2:32 PM, Anders F Björklund wrote:

 Jeffrey Johnson wrote:
 
 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.
 
 There was a half-hearted attempt at this, earlier on:
 http://afb.users.sourceforge.net/rpm/pythondeps.py
 

Nice!

 A google search for python module dependency graph shows
 some reasonably serious efforts to create reliable (i.e. not
 *RE's scraping use … statements out of *.pl) dependencies.
 
 I used http://docs.python.org/library/modulefinder.html
 and then filtered out some boring ones like sys and os.
 

Yes modulefinder included in python is the right starting point.

I'd stub the Provides: rather than filter the Requires: because WYSIWYG
is far easier maintenance than discussing The One True Meaning of
boring

 
 And the verification could/should/would proceed by capturing reality
 (i.e. overloading builtins.__import__ to log what modules are actually
 loaded for comparison with python(foo) dependencies in *.rpm headers)
 
 Any interest? If so, the starting point would be to find the 2-3 best
 python dependency graphs and have a go with sed(1) on the generated
 *.dot spewage to see how good the dependencies being found actually are.
 
 The --requires and --provides looked reasonable enough,
 but it would need some tweaking and testing before use.
 

Scale of 1 - 10 (10 being perfect) can you rate your  tweaking and testing?

 There's also http://rpm5.org/cvs/fileview?f=rpm/scripts/pythoneggs.py
 

Yes this too is reasonable but is merely a transform from one sewage to another
(with the usual impedance match difficulties for tested fubar semantics).

(aside)
Personally I'd rather see pythoneggs.sh rather than python eggs.py: there's
nothing in that script that MUST have python afaict.

The hardest problem is finding a useful semantic for a python dependency
graph. Otherwise there's the usual metric tone butt load of
Disablers! Filtering! Different!
RFE's that swarm like flies on rotten meat whenever RPM features are suggested.

A parser for *.dot is KISS wrto the bike shed discussion

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Tim Mooney

In regard to: Requires: python(foo) = bar, Jeffrey Johnson said (at 2:11pm...:


Matthew Dawkins mentioned an interest in better python module
dependencies in RPM earlier today.


I'm still running rpm 5.1.9 and meaning to upgrade, but I've been meaning
to ask about ruby dependencies.  For better or worse, ruby is becoming
a big party of my life.

5.1.9 isn't doing anything for automatic ruby module dependencies, has
that improved with any of the more recent RPM releases?  Ruby complicates
things somewhat over other scripting languages because of the
not-yet-complete transition from standard modules to gems, so automatic
dependency tracking has to do some extra work for ruby, I think.

Also, if I were looking to settle on a particular stable version of rpm
for the next year or so, what's the recommended stable version to pick?
Just 5.4.latest, or something earlier?


All of these are python scripts are generating *.dot output. So one approach
to better
Requires: python(foo) = bar
would be to write a parser for *.dot format (which is pretty damn trivial, even
sed(1) is almost enough) to automagically generate better python module
dependencies.


I wrote a brain-dead BuildRequires dependency generator for all my local
specs that generated a .dot file, but had real trouble getting graphviz
to display it.  I would love to know what options people are using with
graphviz to get a reasonable display of 1800+ nodes, some of which have
50-100 edges connecting them.

Tim
--
Tim Mooney tim.moo...@ndsu.edu
Enterprise Computing  Infrastructure  701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Per Øyvind Karlsen
2012/5/11 Tim Mooney tim.moo...@ndsu.edu:
 In regard to: Requires: python(foo) = bar, Jeffrey Johnson said (at
 2:11pm...:


 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.


 I'm still running rpm 5.1.9 and meaning to upgrade, but I've been meaning
 to ask about ruby dependencies.  For better or worse, ruby is becoming
 a big party of my life.

 5.1.9 isn't doing anything for automatic ruby module dependencies, has
 that improved with any of the more recent RPM releases?  Ruby complicates
 things somewhat over other scripting languages because of the
 not-yet-complete transition from standard modules to gems, so automatic
 dependency tracking has to do some extra work for ruby, I think.

 Also, if I were looking to settle on a particular stable version of rpm
 for the next year or so, what's the recommended stable version to pick?
 Just 5.4.latest, or something earlier?

I introduced support for ruby gems around rpm 5.2.x,
with fairly completetion of current support in 5.3.x.

5.4.x contains new fixes that you need for ruby 1.9
though, which hasn't been backported to older
branches.

Also there's some new helpers and macros
available to make it easier to auto-generate
ruby packages from gems, adapted to
rpm5: http://gitorious.org/+rpm5distro/rpm5distro/gem2rpm5

This work has been carried out in Mandriva Linux,
so if you need to look around for additional info
on, you should probably try visit our wiki, mailing
lists, repositories etc. :)

--
Regards,
Per Øyvind
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Jeffrey Johnson

On May 11, 2012, at 2:46 PM, Tim Mooney tim.moo...@ndsu.edu wrote:

 In regard to: Requires: python(foo) = bar, Jeffrey Johnson said (at 2:11pm...:
 
 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.
 
 I'm still running rpm 5.1.9 and meaning to upgrade, but I've been meaning
 to ask about ruby dependencies.  For better or worse, ruby is becoming
 a big party of my life.
 
 5.1.9 isn't doing anything for automatic ruby module dependencies, has
 that improved with any of the more recent RPM releases?  Ruby complicates
 things somewhat over other scripting languages because of the
 not-yet-complete transition from standard modules to gems, so automatic
 dependency tracking has to do some extra work for ruby, I think.
 

There's rubygems.rb written by Per Oyvind in production
use in Mandriva for more than a year now and I haven't
seen/heard serious issues (but I can't easily tell how
much ruby use there is: I do know that ROSA's ABF is
making some serious use of ruby 1.9.3 but using rvm
not rpm for installation. I looked a bit at rvm
and was quite impressed that it succeeded so well.
Meanwhile the very ida of running multiple versions
of ruby and modules out of a per-user shadow tree
in ~/.rvm drove me to some serious drinking to forget
what I saw ;-)

 Also, if I were looking to settle on a particular stable version of rpm
 for the next year or so, what's the recommended stable version to pick?
 Just 5.4.latest, or something earlier?
 

rpm-5.3.11 is production, rpm-5.4.x will be rather active
until mid-late summer preparing for ROSA 2012.

About the only real difference between rpm-5.3 - 5.4 is that
embedded sqlite appears ready for production use, and sqlite3
for an rpmdb will likely re-appear, as well as the ability to
push package metadata into multiple other sqlite3 databases.

The embedded sqlite3 still has some minor (10-60KB) memory
leaks (technically they are not leaks because they
are not recurrent, but I'd rather not see the flaws because
I use valgrind on rpm daily).

There are buildbot's and continuous integration running at
http://harwich.jbj.org:8010
You can easily assess stability of rpm-5.4.x as I proceed.

 All of these are python scripts are generating *.dot output. So one approach
 to better
  Requires: python(foo) = bar
 would be to write a parser for *.dot format (which is pretty damn trivial, 
 even
 sed(1) is almost enough) to automagically generate better python module
 dependencies.
 
 I wrote a brain-dead BuildRequires dependency generator for all my local
 specs that generated a .dot file, but had real trouble getting graphviz
 to display it.  I would love to know what options people are using with
 graphviz to get a reasonable display of 1800+ nodes, some of which have
 50-100 edges connecting them.
 

Hehehe … yes distro everything graphs broke every display
format I tried when I looked at various RPM based linux distros
a while back.

Using graphviz on Mac OS X (used to be very very nice, dunno if
still actively distributed in *.dmg) and choosing PDF as a display
was the _ONLY_ means I could find to handle large dependency graphs.

I still think a tree map rather than nodes - edges is a
better easier to understand display format for packages
because of the ability to map multiple attributes into
size/color/adjacency/labels/shapes. At some point all you
have is a huge snarly yarn ball with dotty.

hth

73 de Jeff
 Tim
 -- 
 Tim Mooney tim.moo...@ndsu.edu
 Enterprise Computing  Infrastructure  701-231-1076 (Voice)
 Room 242-J6, IACC Building 701-231-8541 (Fax)
 North Dakota State University, Fargo, ND 58105-5164
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Jeffrey Johnson

On May 11, 2012, at 2:46 PM, Tim Mooney tim.moo...@ndsu.edu wrote:

 In regard to: Requires: python(foo) = bar, Jeffrey Johnson said (at 2:11pm...:
 
 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.
 
 I'm still running rpm 5.1.9 and meaning to upgrade, but I've been meaning
 to ask about ruby dependencies.  For better or worse, ruby is becoming
 a big party of my life.
 

Specific questions:

1) Would rpm-ruby bindings be useful to you? The existing GPL
project floating about are rather interesting: an entire build
system was written in some JA distro and the methods
are much more carefully thought through than what I
typically see with RPM bindings in various languages
I need to see installed NEVRA from an rpmdb!
and all the other functionalities of package management are ignored.
That isn't/wasn't true in the ruby bindings: writing an
entire build system in ruby takes some serious efforts.

(aside)
Sadly GPL prevents usage (and interest) in the existing ruby
bindings on the LGPL lunatic fringe of FL/OSS @rpm5.org. And
there is negative interest in @rpm5.org support @redhat.com and @suse.com.
*shrug*

There's also some very preliminary (5 days effort) work I did to
assess the feasibility of embedded ruby that has been reworked
to some extent info bindings (the original intent was embedded
interpreter module, these are very different usage cases).

There's some serious impediments to embedding ruby because
allocations are all on stack, making it quite tricky to
instantiate multiple embedded interpreters reliably. The
Great Divide between 1.8.x - 1.9.y, and the rather poor
support for linking in the needed libraries for an embedded
interpreter are rather painful too. But I think I know some
ways to attack all of these issues …

2) Are you doing DevOPS? I'd like to see a unification of
configuration - package
management, actually looked at puppet/chef/cfengine 6 weeks
ago. cfengine would be easiest to link into RPM and has
definite design strengths (i.e. I like promises). But
clearly PUPPET! PUPPET! PUPPET! is the loudest chant
from the Linux advocates everywhere.

The remote hosting with chef was also quite interesting
(though the business model may not pay off, and chef
is still mostly playing catchup to puppet).

Meanwhile I was underwhelmed with most of what I saw
with DevOPS: simple tasks became hugely complex and
there were no apparent complex tasks that the tools simplified
enough to interest me. But that was merely a drive-by
review of current state-of-the-art; I'm sure there
are features/benefits that I entirely missed in all
of the 3 DevOPS applications (2 in ruby) that I looked at.

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Tim Mooney

In regard to: Re: Requires: python(foo) = bar, Jeffrey Johnson said (at...:



On May 11, 2012, at 2:46 PM, Tim Mooney tim.moo...@ndsu.edu wrote:


In regard to: Requires: python(foo) = bar, Jeffrey Johnson said (at 2:11pm...:


Matthew Dawkins mentioned an interest in better python module
dependencies in RPM earlier today.


I'm still running rpm 5.1.9 and meaning to upgrade, but I've been meaning
to ask about ruby dependencies.  For better or worse, ruby is becoming
a big party of my life.



Specific questions:

1) Would rpm-ruby bindings be useful to you?


Maybe some day, but at this point, not really.  I don't actually want to
interface with rpm from ruby (yet, and maybe not ever), I just want rpm
to automatically figure out all the junk (module or gem) that is required
when I rpm install something like rubygem-puppet-lint.


2) Are you doing DevOPS? I'd like to see a unification of
configuration - package
management, actually looked at puppet/chef/cfengine 6 weeks
ago.


Yes, that's what's driving ruby in our environment: puppet.


cfengine would be easiest to link into RPM and has
definite design strengths (i.e. I like promises). But
clearly PUPPET! PUPPET! PUPPET! is the loudest chant
from the Linux advocates everywhere.


It does have tremendous momentum.  It's changing a lot and is
a serious moving target, though.  A lot of what was done in our
environment when our primary puppet advocate was beginning to deploy
puppet here will need to be reworked considerably for the forthcoming
puppet 3.0.


Meanwhile I was underwhelmed with most of what I saw
with DevOPS: simple tasks became hugely complex and
there were no apparent complex tasks that the tools simplified
enough to interest me. But that was merely a drive-by
review of current state-of-the-art; I'm sure there
are features/benefits that I entirely missed in all
of the 3 DevOPS applications (2 in ruby) that I looked at.


I'm hard pressed to say that you're wrong;  There is a lot (I mean *a
lot*) of work that needs to be invested up front to get any benefits
out of devops.  If there's much variation in one's environment (as there
is in an academic environment like mine) that also adds a lot of
complexity, and it's complexity that one isn't necessarily prepared to
handle when they're initially rolling out something like puppet.

The benefits start to become pretty significant once you have a good
portion of your hosts included in your CM infrastructure.  It's really
nice to be able to make one change (like remove a sysadmin when he leaves)
in one spot and have it automatically applied to your entire environment
automatically.  Or know that your entire web farm is configured
identically *and* will be re-configured to be identical even if an admin
makes a change just for testing on just one of the web servers, and then
forgets to either undo or propagate it.

It's no panacea, but once you get critical mass, the benefits do start
to become much more apparent.

Tim
--
Tim Mooney tim.moo...@ndsu.edu
Enterprise Computing  Infrastructure  701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Tim Mooney

In regard to: Re: Requires: python(foo) = bar, Jeffrey Johnson said (at...:


Meanwhile the very ida of running multiple versions
of ruby and modules out of a per-user shadow tree
in ~/.rvm drove me to some serious drinking to forget
what I saw ;-)


Ruby, in general, has inspired beverage consumption in me.


rpm-5.3.11 is production, rpm-5.4.x will be rather active
until mid-late summer preparing for ROSA 2012.


I'll give 5.3.11 a whirl in a couple weeks.  Thanks for the pointer.


Using graphviz on Mac OS X (used to be very very nice, dunno if
still actively distributed in *.dmg) and choosing PDF as a display
was the _ONLY_ means I could find to handle large dependency graphs.


I was hoping for PDF display, but that was a no-go for me on both
Solaris 10 and Linux.  The only option that actually worked at all was
png, and it wasn't very usable.


I still think a tree map rather than nodes - edges is a
better easier to understand display format for packages
because of the ability to map multiple attributes into
size/color/adjacency/labels/shapes. At some point all you
have is a huge snarly yarn ball with dotty.


That's exactly what I had, and I was only tracking manually specified
BuildRequires dependencies on my Solaris system.  It was a mess...

Tim
--
Tim Mooney tim.moo...@ndsu.edu
Enterprise Computing  Infrastructure  701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Mark Hatle

On 5/11/12 1:11 PM, Jeffrey Johnson wrote:

Matthew Dawkins mentioned an interest in better python module
dependencies in RPM earlier today.

A google search for python module dependency graph shows
some reasonably serious efforts to create reliable (i.e. not
*RE's scraping use … statements out of *.pl) dependencies.


I'd love better dependencies, but most of the time the implementations I see are 
focused around attempting to run the module in python and having it return back 
usage information.


This doesn't always work for me in my cross-install case.  (Note, I can 
certainly make it a requirement that the same version of python be available on 
the host (build system) as on the target system.  But I don't believe that is 
enough.)


--Mark


Perhaps best-of-breed (based on 10 mins of surfing) appears to be snake food
http://furius.ca/snakefood/
There's also (perhaps more recent/active)
http://pypi.python.org/pypi/modulegraph/

There's also (useful for verifying but not for generating dependencies
because it requires loading modules)
http://www.indelible.org/ink/python-reloading/
with a perhaps even simpler script here

http://code.activestate.com/recipes/577222-create-module-dependency-graph/

All of these are python scripts are generating *.dot output. So one approach
to better
Requires: python(foo) = bar
would be to write a parser for *.dot format (which is pretty damn trivial, even
sed(1) is almost enough) to automagically generate better python module
dependencies.

And the verification could/should/would proceed by capturing reality
(i.e. overloading builtins.__import__ to log what modules are actually
loaded for comparison with python(foo) dependencies in *.rpm headers)

Any interest? If so, the starting point would be to find the 2-3 best
python dependency graphs and have a go with sed(1) on the generated
*.dot spewage to see how good the dependencies being found actually are.

JMHO, dunno if I have time to look at python(…) reps for a couple of months ...

73 de Jeff


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


Re: Requires: python(foo) = bar

2012-05-11 Thread Jeffrey Johnson

On May 11, 2012, at 4:12 PM, Mark Hatle mark.ha...@windriver.com wrote:

 On 5/11/12 1:11 PM, Jeffrey Johnson wrote:
 Matthew Dawkins mentioned an interest in better python module
 dependencies in RPM earlier today.
 
 A google search for python module dependency graph shows
 some reasonably serious efforts to create reliable (i.e. not
 *RE's scraping use … statements out of *.pl) dependencies.
 
 I'd love better dependencies, but most of the time the implementations I see 
 are focused around attempting to run the module in python and having it 
 return back usage information.
 
 This doesn't always work for me in my cross-install case.  (Note, I can 
 certainly make it a requirement that the same version of python be available 
 on the host (build system) as on the target system.  But I don't believe that 
 is enough.)
 

SUre module loading works in cross-install: qemu (and that need not be
deathly slow for something as simple as extracting a dep graph)

Meanwhile some of the links do _NOT_ require loading afaik. Which
is an improvement on state-of-the-art in python if true (I haven't tried).

73 de Jeff
 --Mark
 
 Perhaps best-of-breed (based on 10 mins of surfing) appears to be snake 
 food
  http://furius.ca/snakefood/
 There's also (perhaps more recent/active)
  http://pypi.python.org/pypi/modulegraph/
 
 There's also (useful for verifying but not for generating dependencies
 because it requires loading modules)
  http://www.indelible.org/ink/python-reloading/
 with a perhaps even simpler script here
  
 http://code.activestate.com/recipes/577222-create-module-dependency-graph/
 
 All of these are python scripts are generating *.dot output. So one approach
 to better
  Requires: python(foo) = bar
 would be to write a parser for *.dot format (which is pretty damn trivial, 
 even
 sed(1) is almost enough) to automagically generate better python module
 dependencies.
 
 And the verification could/should/would proceed by capturing reality
 (i.e. overloading builtins.__import__ to log what modules are actually
 loaded for comparison with python(foo) dependencies in *.rpm headers)
 
 Any interest? If so, the starting point would be to find the 2-3 best
 python dependency graphs and have a go with sed(1) on the generated
 *.dot spewage to see how good the dependencies being found actually are.
 
 JMHO, dunno if I have time to look at python(…) reps for a couple of months 
 ...
 
 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org