RE: Improving speed of par app startup
Well, sort of. I need a solution that is portable across Solaris, Linux, Windows, MacOSX, HP-UX, etc. with minimal fuss. A pp app gives me portability and simplicity of installation (i.e. no installation required). The user can simply copy the binaries anywhere they like and it just works. I don't mind requiring an installer for server components, but anything that requires extra installation steps for every user of the system is an impediment to user adoption. Batch files and vbscripts are problematic if you are using cygwin instead of cmd as the shell. To get it all working on Windows would require that I create and maintain a .bat file that calls a .vbs file, plus a shell script for cygwin and unix users, plus the original .pl file and all its supporting files. This is doable and would certainly resolve the space and performance concerns, but it's a lot more complicated than the self-contained binary approach. >From a supportability standpoint pp does pretty much exactly what I want, ignoring the performance and disk space issues we've been discussing. --Scott -Original Message- From: Roderich Schupp [mailto:[EMAIL PROTECTED] [...] Now I think I understand what you really want: (1) a mechanism to determine everything that's used by your application (2) arrange all this stuff below some given directory (3) have it work when started from there even when the given directory has been moved around [...] -
Re: Improving speed of par app startup
On Nov 28, 2007 6:39 PM, Scott Stanton <[EMAIL PROTECTED]> wrote: > I'm trying to ship a self contained application to our customers. We > are not allowed to dictate to the customer what versions of perl and > installed packages they may have on their machine outside of our > installation directory. They may not even have an installed perl. We > certainly can't rely on them having compilers, etc. For us, the point > of wrapping the apps is to provide a single self-contained binary for > each tool that can be copied to any directory without external > dependencies. > > We don't know what directory they will install our application into when > we build the application, so a single fixed cache directory doesn't > really work unless it's settable after the fact. Now I think I understand what you really want: (1) a mechanism to determine everything that's used by your application (2) arrange all this stuff below some given directory (3) have it work when started from there even when the given directory has been moved around The fact that pp produces a self-extracting executable is irrelevant for this goal except that this achieves (1), (2) (with stuff being just the executable) and (3) (if the cache directory could be changed after building the executable). But you could instead simply ship a zip file to your customers that they can unzip anywhere and then run the application from there as is (if it could be made "self-relocatable") or apply a separate relocation step as part of the installation procedure? In fact, wouldn't the following work: Assuming that you already have (1) and (2) with - all Perl modules (including glue dlls) in blib-style layout under blib/ - all other needed dlls under lib/ - your perl scripts under scripts/ - the perl executable perl.exe copied into the toplevel directory Now wrap every scripts/foo.pl with foo.bat that does the following: - determine the directory it is run from (I think current Windows scripting host will give you that), say $here - execute $here/perl.exe $here/prolog.pl $here $here/scripts/foo.pl rest-of-original-arguments And prolog.pl does: my $here = shift @ARGV my $script = shift @ARGV set library search path to "$here/lib" clear @INC use blib "$here/blib" # but "by hand", don't actually use "use" or "blib" or any other module do $script; Cheers, Roderich -
PAR::Packer build error on OSX 10.5
Hello. I'm having difficulty building PAR::Packer. I'm running OSX 10.5. I've downloaded the svn from http://svn.openfoundry.org/par/PAR-Packer/trunk/ and am trying to compile. However, I get the following: [EMAIL PROTECTED]:~/Install/Sources/par-packer $ make include /Users/adodge/Install/Sources/par-packer/inc/Module/Install.pm include inc/Module/Install/PAR.pm include inc/Module/Install/Base.pm cp lib/App/Packer/PAR.pm blib/lib/App/Packer/PAR.pm cp lib/PAR/Packer.pm blib/lib/PAR/Packer.pm cp lib/PAR/Filter/Obfuscate.pm blib/lib/PAR/Filter/Obfuscate.pm cp lib/PAR/Filter/PodStrip.pm blib/lib/PAR/Filter/PodStrip.pm cp lib/PAR/StrippedPARL/Base.pm blib/lib/PAR/StrippedPARL/Base.pm cp lib/PAR/Filter.pm blib/lib/PAR/Filter.pm cp lib/PAR/Filter/PatchContent.pm blib/lib/PAR/Filter/PatchContent.pm cp lib/PAR/StrippedPARL/Dynamic.pm blib/lib/PAR/StrippedPARL/Dynamic.pm cp lib/PAR/Filter/Bytecode.pm blib/lib/PAR/Filter/Bytecode.pm cp lib/pp.pm blib/lib/pp.pm cp lib/PAR/Filter/Bleach.pm blib/lib/PAR/Filter/Bleach.pm cp lib/PAR/StrippedPARL/Static.pm blib/lib/PAR/StrippedPARL/Static.pm perl sha1.c.PL sha1.c cc -c -arch i386 -arch ppc -g -pipe -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -Wdeclaration-after-statement -I/usr/local/include -I/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE sha1.c perl ./file2c.pl /System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/libperl.dylib my_perl.c load_me_0 1 3 cc -c -arch i386 -arch ppc -g -pipe -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -Wdeclaration-after-statement -I/usr/local/include -I/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE main.c perl ./file2c.pl ../script/par.pl my_par_pl.c load_me_2 1 cc -c -arch i386 -arch ppc -g -pipe -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -Wdeclaration-after-statement -I/usr/local/include -I/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE my_par_pl.c cc -mmacosx-version-min=10.5 main.o my_par_pl.o -arch i386 -arch ppc -L/usr/local/lib /System/Library/Perl/5.8.8/darwin-thread-multi-2level/auto/DynaLoader/DynaLoader.a -L/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE -lperl -ldl -lm -lutil -lc --output ./par perl -e1 perl ./file2c.pl ./par my_par.c load_me_1 1 3 Usage: ./file2c.pl file.pl file.c c_variable make[1]: *** [my_par.c] Error 1 make: *** [subdirs] Error 2 The problem seems to be that myldr/par doesn't exist. I can't seem to find mention of this problem anywhere. Any help would be appreciated. Thank you. --Alex Dodge
RE: Improving speed of par app startup
I'm trying to ship a self contained application to our customers. We are not allowed to dictate to the customer what versions of perl and installed packages they may have on their machine outside of our installation directory. They may not even have an installed perl. We certainly can't rely on them having compilers, etc. For us, the point of wrapping the apps is to provide a single self-contained binary for each tool that can be copied to any directory without external dependencies. We don't know what directory they will install our application into when we build the application, so a single fixed cache directory doesn't really work unless it's settable after the fact. One approach I've considered is to have the installer unpack the .par file in a location within our install tree then re-par our applications with a new hard-coded cache location post-install. This would probably work well for some of our server components, but we have user tools that are likely to be accessed from machines other than the installation machine by way of a network mount. Those tools would have no fixed location to refer to, so would need to unpack to a per-user cache on startup. Basically, the way perlapp works seems to address all of our concerns, so I'd like to see if we can get equivalent functionality into par. I'm not suggesting abandoning other modes of operation, just adding the ability to use a per-user shared cache to avoid most of the overhead. --Scott -Original Message- From: Roderich Schupp [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 8:57 AM To: Scott Stanton Cc: Steffen Mueller; par@perl.org Subject: Re: Improving speed of par app startup On Nov 28, 2007 5:42 PM, Scott Stanton <[EMAIL PROTECTED]> wrote: > The intent isn't to avoid name clashes within a single application, it's > to avoid version clashes across applications with a shared cache > directory. Instead of creating a separate cache- directory for > every application, you can share a single cache directory as long as ...all your aplications come from the same source (and are built using the exact same build environment). But can't you have that already by specifying an explicit cache directory during pp'ing? If you're concerned with /tmp size and have lots of applications, why not just bite the bullet and install the necessary stuff on each client? Cheers, Roderich
Re: Improving speed of par app startup
On Nov 28, 2007 5:42 PM, Scott Stanton <[EMAIL PROTECTED]> wrote: > The intent isn't to avoid name clashes within a single application, it's > to avoid version clashes across applications with a shared cache > directory. Instead of creating a separate cache- directory for > every application, you can share a single cache directory as long as ...all your aplications come from the same source (and are built using the exact same build environment). But can't you have that already by specifying an explicit cache directory during pp'ing? If you're concerned with /tmp size and have lots of applications, why not just bite the bullet and install the necessary stuff on each client? Cheers, Roderich
RE: Improving speed of par app startup
I've seen slow behavior on Windows XP SP2 and Windows 2003. My machine has the Trend Micro anti-virus client running on it. I've tried disabling the anti-virus software and it definitely speeds things up, although it's still quite a bit slower than Linux. Unfortunately telling our users that they have to disable their anti-virus software before they use our applications is not an option. --Scott -Original Message- From: Dr.Ruud [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 2:06 AM To: par@perl.org Subject: Re: Improving speed of par app startup "Scott Stanton" schreef: > It seems like Windows > is very bad at creating lots of small files Which Windows version? Anti-virus involved? -- Affijn, Ruud "Gewoon is een tijger."
RE: Improving speed of par app startup
The intent isn't to avoid name clashes within a single application, it's to avoid version clashes across applications with a shared cache directory. Instead of creating a separate cache- directory for every application, you can share a single cache directory as long as files are given reasonably unique names. With an md5 sum, you can be pretty confident that you won't have name collisions for .pm files, but for secondary .dll files, you can't change the name. The solution is to put them in uniquely named subdirectories. This also allows subsequent releases of one application to contain updated versions of the same library without problems. The general problem I'm concerned with is how much /tmp space a collection of par apps end up consuming. If you multiply the unpacked size by the number of distinct applications and the number of users of the system, you very quickly run out of space in /tmp. When I was using perlapp, I didn't have this problem because all of the apps shared a common cache per user. The vast majority of the unpacked files can be shared without a problem. For now, I have to use --clean to avoid the problem, but this is slower than I'd like. --Scott -Original Message- From: Roderich Schupp [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 12:42 AM To: Scott Stanton Cc: Steffen Mueller; par@perl.org Subject: Re: Improving speed of par app startup On Nov 27, 2007 11:05 PM, Scott Stanton <[EMAIL PROTECTED]> wrote: > > > Steffen Mueller wrote: > > > That's actually a good idea: Avoiding dll file name clashes using a > > directory for each external (non-perl-module) dll. Currently, we > extract > > the external dlls under their original name, IIRC. However, I don't > know > > how you can make those directories accessible to the dll loader > without > > appending *each* directory to $PATH. Doing that in a long path ($TEMP > is > > long on Windows, isn't it?) might quickly result in hitting the length > > limitations of env. vars on Windows. > > Long paths might be an issue, although if we're talking just a handful > of libraries, in practice it might work just fine. Putting every dll into its own directory won't solve the name clash issue: no matter how you order this LD_LIBRARY_PATH (or whatever it is called), if there are two dlls with the same name any attempt to load such a dll will only get the first along the library search path. So you might as well put them all with their original names into one directory (and have PAR::Packer carp if it encounters a name clash). If two dlls with the same name ever worked in the unpacked case, then this means that either (1) the native OS library loader or (2) the application explicitly deviates from this simple model of operation: search the (fixed at program startup) library path until a dll with the required name is found. Cheers, Roderich
Re: Improving speed of par app startup
"Scott Stanton" schreef: > It seems like Windows > is very bad at creating lots of small files Which Windows version? Anti-virus involved? -- Affijn, Ruud "Gewoon is een tijger."
Re: Improving speed of par app startup
Hi Roderich, Roderich Schupp wrote: > Putting every dll into its own directory won't solve the name clash issue: doh! Of course. I must have been half asleep. Steffen
Re: Improving speed of par app startup
On Nov 27, 2007 11:05 PM, Scott Stanton <[EMAIL PROTECTED]> wrote: > > > Steffen Mueller wrote: > > > That's actually a good idea: Avoiding dll file name clashes using a > > directory for each external (non-perl-module) dll. Currently, we > extract > > the external dlls under their original name, IIRC. However, I don't > know > > how you can make those directories accessible to the dll loader > without > > appending *each* directory to $PATH. Doing that in a long path ($TEMP > is > > long on Windows, isn't it?) might quickly result in hitting the length > > limitations of env. vars on Windows. > > Long paths might be an issue, although if we're talking just a handful > of libraries, in practice it might work just fine. Putting every dll into its own directory won't solve the name clash issue: no matter how you order this LD_LIBRARY_PATH (or whatever it is called), if there are two dlls with the same name any attempt to load such a dll will only get the first along the library search path. So you might as well put them all with their original names into one directory (and have PAR::Packer carp if it encounters a name clash). If two dlls with the same name ever worked in the unpacked case, then this means that either (1) the native OS library loader or (2) the application explicitly deviates from this simple model of operation: search the (fixed at program startup) library path until a dll with the required name is found. Cheers, Roderich
Re: PAR: shared library dependencies
Hi Scott, Scott Stanton wrote: > I found the problem. It turns out that _add_info will add a duplicate > entry if the used_by argument differs in case from an existing package > name. It was a simple fix, but took a while to figure out a test case. > Revision 547 has the test and a fix. Thanks! Your help is much appreciated! Expect a fixed Module::ScanDeps release soon. Best regards, Steffen
