Please test: Packaging using stripped parl(dyn)
Hi list, after some discussion with Roderich, I dove into the PAR::Packer sources and changed some rather integral parts. Here's what happened: - We had lots of bugs due to XS modules being built into 'parl' being insufficient for the packaged code (Scalar::Util::refaddr()) or incompatible with the packaged XS modules. Note: Distinguish between the a) system PAR was built on, b) system pp is used on and c) system the resulting binary is used on. - A possible solution is to use the modules found on the packaging system (b) instead of what was built into parl on system a). - That either requires rebuilding parl for every call to pp on b) (bad because b) might lack the required tools) or using a stripped down parl. - A stripped down parl wouldn't work as a standalone PAR loader. == parl can't be modified. - Solution: -- Leave parl as it is and embed a stripped down version into a pure data module (PAR::StrippedPAR::Static and PAR::StrippedPAR::Dynamic respectively) which is created at PAR build time. -- Extract that parl-without-embedded-modules for each pp call and use it for packaging binary executables. The reason I chose to embed the stripped parl into a module over adding another (two: static dynamic) executable files to scripts/ was that the latter approach might have caused confusion when strippedparl or so wouldn't work as advertised. I'm going to commit these changes to SVN now. I strongly urge all of you to test these changes thoroughly. They're somewhat extensive and I would hate if the blew up in my face when I make the 0.958 release. You can get the latest SVN with this command: svn checkout http://svn.openfoundry.org/par/trunk Cheers, Steffen
Re: Please test: Packaging using stripped parl(dyn)
Me again! Steffen Mueller schrieb: I'm going to commit these changes to SVN now. I strongly urge all of you to test these changes thoroughly. They're somewhat extensive and I would hate if the blew up in my face when I make the 0.958 release. For your information, I have just tested these changes on win32 and they fail. I'll investigate asap. Probably tonight. If anybody is willing to help debug this on win32, you're very welcome. :) Steffen
Re: Undefined subroutine Scalar::Util::refaddr
On Mon, 2006-11-06 at 14:38 +0100, Steffen Mueller wrote: Roderich Schupp schrieb: [...] You could try what happens for system() (don't remember). If system() doesn't flash a DOS box with --gui, you could replace the backticks by using IPC::Run3 (that allows you to catch stdout/stderr into a string, but way more flexible as backticks), because it uses system() internally. If system() doesn't work, maybe some advanced use of Win32::Process and/or Win32::Console might to the trick? Make sure you use the multi-argument form of system() because that should avoid the console! IPC::Run3 flashes - so that didn't help. system() might be ok, but without a shell I can't redirect output - or at least I don't know how to. Peter
Re: Please test: Packaging using stripped parl(dyn)
Hi Glenn, instead of replying to each point, I'll write a coherent reply. I hope you don't mind. It mostly boils down to this valid question Glenn asked: If the stripped parl has a chance of working when executed by pp, why would in not be able to work in all cases of execution? Let me put the long answer to this: A pp-ed binary is basically: - Some C loading code + a modified perl == parl - PAR (or rather par.pl!) and its dependencies embedded into the above == stand-alone parl - A .par file appended to the above == standalone program Executing this, the first stage extracts the second and uses it to load the third. Removing the second part (par.pl and friends) should stop it from working as expected. That's what I'm seeing on win32 but not on linux. Why? Bear with me: Roderich and I tried using a parl without the second part on linux. It worked just fine. For the reasons laid out above, it shouldn't have, but it did anyway. So I tried it out and implemented the new behaviour. Understanding what happens during make() in myldr/ was a good part of the work involved. That's not lost. The experiment worked flawlessly on my linux. Why? Because parl==perl+C has @INC paths compiled into it. It picked up stuff from my installed perl. I don't mind that, of course, for uses of parl via pp which happens on a fully configured perl at all times. When building executables from parl+C+.par but without par.pl+dependencies and running these executables on the same system, parl *still* picked out the dependencies from my system. Here is my big, big error. I didn't expect it to. I recently changed the code in the packaged main.pl so that @INC is cleared if you recall. parl picks up the libs, then runs a new interpreter for main.pl which clears *only its own @INC* and runs the user code. Hence the user code doesn't see the hardwired @INC, but parl does. This is also the reason things blew up on win32: perl doesn't have hardcoded @INC there. So. The easy way doesn't work. I can either do the embedding of the par.pl+deps code into parl for every pp call or I can roll back svn to the state a few commits ago and call it a broken experiment. I'm not sure how practical the former is. I'd have to think that through (this time for real). Steffen P.S: Now that you bore with me, I can tell you the short answer: I expected parl not to have perl's compiled-in @INC because I removed it in main.pl. I was wrong.
Re: Please test: Packaging using stripped parl(dyn)
Glenn Linderman schrieb: [...] So most of the extra run-time cost of pp in the above sequence could be eliminated by doing it at installation time, or at a later re-run this if you upgrade XS modules. If this sort of packaging of the build time works for all pre-built installation techniques, then you have the current performance of pp/par, the bug fixed, and the cost was a more complex build scheme, and a slower installation. Quite frankly, I am more than happy to trade slower package generation for less frustrated users. If they have to run some weird script (as root, no less!) every time they upgrade one of the modules which are packaged into parl, they won't do that. They won't because they wouldn't know or if they knew, they wouldn't remember by the time they upgrade. Else, we could just tell them to install a newer PAR binary anyway. But most users won't even surface here. They'll just be annoyed and buy perl2exe or perlapp. Furthermore, running scripts at installation time for all the gazillions of binary package managers is a daunting and certainly not a rewarding task. That you can only think of PPM being a package manager shows that you're working on Windows only. (Almost) Every Linux has its own package manager. Additionally, you can install .par PAR packages. The PAR Makefile.PL does this if no compiler is present, for example. (Though I control the tool in this case.) I will not inflict running some tool on upgrades on the users and I won't inflict dealing with installation-time-of-binary-packages scripts on me. Sorry, but this doesn't work. The whole point of this exercise is to make PAR easier for the user and to remove things they need to remember when administrating a PAR installation. Steffen
Linux DBD::MySQL client library issue
I created an application that uses DBD::MySQL. I add the MySQL client library to the PAR with '-l /opt/mysql/lib/mysql/libmysqlclient.so.15'. When I do the 'pp' on my Solaris 8 system the executable runs and completes without issue, however, when I do the same command line on Linux, things get interesting. The first time I run it on Linux, it complains about not being able to find the library file and exits, if I rerun the executable it runs just fine and completes. I can also change my shell's LD_LIBRARY_PATH after the initial run and it will work. If I look in the '/tmp/par-root/cache-*' file I see the libmysqlclient.so file in the root of the cache both before and after the initial run. In a test script I can see that LD_LIBRARY_PATH and others are being set correctly. It's almost like DBD::MySQL is called before the LD_LIBRARY_PATH is set. If call it without the '-l' and the libmysqlclient.so.15 file is in '/ot/mysql/lib/mysql' then the PAR file runs without issue from the start. I have done like the FAQ stated and removed the link and copied the file to the real name. Initial Run: [EMAIL PROTECTED] ./test_prog Can't load '/tmp/par-root/cache-3bb398dbc6f8440b27f0e710c6cdba3e286c7562/726cee52.so' for module DBD::mysql: libmysqlclient.so.15: cannot open shared object file: No such file or directory at /opt/perl/lib/5.8.4/i686-linux/DynaLoader.pm line 230. at ../blib/lib/PAR/Heavy.pm line 107 Compilation failed in require at (eval 11) line 157. BEGIN failed--compilation aborted at (eval 11) line 157. [EMAIL PROTECTED] [EMAIL PROTECTED] ./test_prog [EMAIL PROTECTED] I've not seen this issue in the mailing list archives, so I believe this to be new. PAR Version: 0.957 Perl Version: 5.8.4 MySQL 5.0.24 Eric Brinkley
