Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Rob van der Heij
On 30 January 2013 05:17, Patrick Spinler wrote: > Since no one else seems to be pointing this out, i can see at least one > potential optimization: > > This will re-open the output file and seek to the end every output. > That's a factor of 2 or 3 more syscalls every time. (possibly plus the >

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Agblad Tore
I think that echo will issue a openfile-appendrecord-closefile every time, and that costs. The way I fix these scripts is to code them in std c, it should only require a: gcc -o mypgm mypgm.c I had a similar case some years ago, it also read and write files but also did calculations via expr cmd

Re: Sending a signal along with associated data.

2013-01-30 Thread Malcolm Beattie
Thomas Anderson writes: > You are correct in that the use of signals is pretty limited and there isn't > a convenient way to pass data to the target application. POSIX real-time signals allow an int or pointer's-worth of data to be sent along with the signal which is then queued and also carries

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread John McKown
Thanks to all for the input! I _tried_ to run the script over night. I added an echo to tell me which input file I was working on. I came in this morning. It had been running from 14:00 to 06:30 (16 1/2 hours) and was still on the first input file. That ain't gonna cut it. Time to rethink. Using a

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Shane G
On Thu, Jan 31st, 2013 at 12:44 AM, John McKown wrote: > Thanks to all for the input! I _tried_ to run the script over night. I > added an echo to tell me which input file I was working on. I came in > this morning. It had been running from 14:00 to 06:30 (16 1/2 hours) > and was still on the firs

Re: What is this error?

2013-01-30 Thread Tom Huegel
Just an FYI.. XRDP works fine now on s390x..after Dans update On Tue, Jan 29, 2013 at 7:15 AM, Tom Huegel wrote: > Thanks Dan, that was quick.. Hopefully I can get to it later today. > Tom > > On Tue, Jan 29, 2013 at 6:12 AM, Dan Horák wrote: > >> Tom Huegel píše v Út 29. 01. 2013 v 04:38 -080

Re: What is this error?

2013-01-30 Thread Dan Horák
Tom Huegel píše v St 30. 01. 2013 v 06:43 -0800: > Just an FYI.. XRDP works fine now on s390x..after Dans update thanks for testing Dan > > On Tue, Jan 29, 2013 at 7:15 AM, Tom Huegel wrote: > > > Thanks Dan, that was quick.. Hopefully I can get to it later today. > > Tom > > > > On Tue, Ja

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Mark Post
>>> On 1/30/2013 at 08:44 AM, John McKown wrote: > But I may be forced into using C or C++ for > speed. Too bad I'm not a very good C programmer. Based on the number of buffer overflow exploits that have been discovered over the years, I don't think very many people are. Mark Post --

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Patrick Spinler
And I should mention, as Shane G. suggested, for simple, big, text processing stuff like this you'll get as good performance from perl as from compiled code. There's a reason big text processing genomics stuff like bioperl is written in perl I'm not a python wizard, but I have to imagine tha

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread John McKown
Well, I know that downloading the 160 Gig uncompressed data takes about 8 hours on the 10 Gig/sec Ethernet connection. I then bzip2 compress that to about 50 Meg. Which I binary upload back to z/OS for safety (since it's setting on my Linux desktop) in just a few minutes. But bzgrep can scan the co

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Patrick Spinler
Regarding i/o buffering, as Rob discusses On 1/30/13 2:40 AM, Rob van der Heij wrote: > > If the input files have a lot of 'chunks' that go to the same output > file, it might be fairly easy to gobble up the ones that go together > and write them in a single go. Based on more heuristics, you may b

Re: Speed of BASH script vs. Python vs. Perl vs. compiled

2013-01-30 Thread Chase, John
> -Original Message- > From: Linux on 390 Port On Behalf Of John McKown > > Well, I know that downloading the 160 Gig uncompressed data takes about 8 > hours on the 10 Gig/sec > Ethernet connection. I then bzip2 compress that to about 50 Meg. Which I > binary upload back to z/OS > for sa