Hi, That definitely helped, thanks for the response!
Haven't had time to look at this in depth, so I feel like I'm not qualified to know if my own suggestion is actually helpful. But looking at osv-apps/python-from-host/Makefile, it seems like nix could do things a bit more cleanly. Nix does manage both the shared libraries and any python dependencies, so rather than having two runs of manifest_from_host.sh and an rsync, I can just tell nix I need python and some pip packages. Running a single command will then give me every path I need. Tested this with python3.8 and Flask and it worked. Personally I find that workflow a little simpler than having to figure out what directories pip packages are installed in. Another advantage would be that nixpkgs has 60,000 packages, and although I'm sure there's plenty of compatibility issues, I think at times it would be a lot easier to take advantage of the work that's already been done to create all those packages, like in this case Flask works without modification. Haven't used mpm much, so maybe it does things nix can't, but I would guess nix could solve a lot of the same problems. Sorry if I'm making suggestions about something I don't understand, just thought I'd bounce my ideas off someone who does understand them. Happy to explain anything in more depth or demo what I mean. Thanks, Matthew On Saturday, November 7, 2020 at 10:15:28 PM UTC-7 [email protected] wrote: > Hi Matthew, > > I am not familiar with nix and how exactly it would fit. If you look at > the osv-apps repo there are many examples of python 2/3 apps. All of those > are driven by module.py and optional makefiles to do a job of collecting > relevant files to the final OSv image as scripts/build, scripts/module.py > orchestrates it all. Alternatively, there is capstan with its package* > command and *.mpm archives. I am not sure where and how nix would fit into > this. > > Now the purpose of manifest_from_host.sh is quite simple - given a Linux > shared library/-ies or executable or a directory with those, find all > *dependant* shared libraries based on information in DT_NEEDED elf header. > As you can see it is not specific to Python. On other hand > manifest_from_host.sh is not indended and can not find all other > dependencies (*.py., *.pyc, etc files) full Python runtime needs. My sense > is that would still need to run manifest_from_host.sh against files built > by nix but I might be wrong. > > Another alternative to building OSv images could be using Docker images > and unpacking them to create corresponding OSv image. For an example look > at > https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker > > which uses undocker tool. Another alternative would be then to use Python > docker image in a similar way. And possibly combine it with capstan. > > I hope it helps a bit, > Waldek > > > > On Tuesday, November 3, 2020 at 10:03:20 AM UTC-5 Matthew Kenigsberg wrote: > >> >> Hi, >> >> Is there a recommended way to add pip packages to osv images? >> >> Was trying to figure that out and I also have a suggestion: I started >> using nix, which is a package manager, and it seems like it could be a >> really good tool for the job. It keeps track of every dependency for a >> piece of software, and copying all the files to run an image with python38 >> and some pip packages only takes one command. I think using nix could >> also make manifest_from_host.sh unnecessary (?) >> >> Anyways, is there an easier way to be using pip packages? >> >> Thanks, >> Matthew >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/fae5d0a1-9886-41ea-8156-0d7ee2f68059n%40googlegroups.com.
