Re: /proc/$$/fd mystery - {t,}csh vs. bash

2009-01-08 Thread Dave Johnson
Michael ODonnell writes:
> 
> 
> > Because on startup /bin/csh dups stdin/out/err to higher fds...
>   [...]
> > this means /bin/csh uses FDs 16,17,18 as it's stdin/out/err instead
> > of 0/1/2.
>   [...]
> > when running a program: after clone(), it just dup()s back to
> > stdin/out/err before exec():
> 
> Cool - thanks for the analysis.  Why might they be doing this?

from csh.h:

> /*
>  * The shell moves std in/out/diag and the old std input away from units
>  * 0, 1, and 2 so that it is easy to set up these standards for invoked
>  * commands.
>  */
> #define FSHTTY  15  /* /dev/tty when manip pgrps */
> #define FSHIN   16  /* Preferred desc for shell input */
> #define FSHOUT  17  /* ... shell output */
> #define FSHERR  18  /* ... shell diagnostics */
> #define FOLDSTD 19  /* ... old std input */


It does make it less complicated if you invoke programs with weird
redirection arguments.

If fd 0/1/2 are all unused just before you exec() then its simple to
dup/open these based on whatever redirection you want the program to
run under.  If they are used then you may have to move stuff out of
the way before setting up redirection or accidentially misplace the
shell's actual stdin/out/err while setting up redirection.

-- 
Dave
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: [GNHLUG] MerriLUG Nashua, Thur 15 Jan, Dinner at Martha's

2009-01-08 Thread Heather Brodeur
Jim Kuzdrall wrote:
> Who  : The friendly, suave, intelligent, knowledgeable MerriLUG group 
> What : Whatever interests you
> Where: Martha's Exchange
> Day  : Thur 15 Jan **Next Week**
> Time : 6:00 PM for grub (no upstairs discussion this month)
> 
> :: Overview


I would like to thank Jim Kuzdrall for all his help with announcements 
and reservations.  Without his help I would not have made it this far.

As is evidenced by the lack of presentation this month and short notice 
meetings lately, I am getting burnt out on coordinating the MerriLUG 
meetings.  Add to this that my work schedule will make it difficult for 
me to make it to the meetings for the next several months, and I have 
decided that I need to step down from this position for a while.

I hope that someone (or preferably a group of someones to share the 
work) will step up to take over in my absence, I'd really hate to see 
the group fall apart (that's actually why I took the lead in the first 
place).  If someone wants to take over indefinitely that's great, but if 
you can just commit to a few months to give me a break, that works too. 
  I _don't_ want to disappear into the sunset as have so many before me.

I am _not_ considering stepping down from the GNHLUG board at this time, 
and am hoping that I will now be able to devote more energy to the 
board, especially if someone takes over MerriLUG.

Sincerely

--Heather Brodeur
Linux _User_ and proud of it :-)
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: Python question

2009-01-08 Thread Steven W. Orr
On Tuesday, Jan 6th 2009 at 16:26 -, quoth Paul Lussier:

=>
=>Hi Folks,
=>
=>Is there a python way to get HH:MM:SS from time.localtime()  ?
=>
=>I'm trying to time how long it takes a python script to run and have
=>thus done:
=>
=>   BEGIN = time.time()
=>   
=>   END = time.time()
=>
=>   ELAPSED = END - BEGIN
=>
=>So, now I have a number like 1231265125.36
=>
=>Which is great, and I can write the code necessary to convert it, but
=>figured that this wheel is probably already invented.
=>
=>And, since I'm just learning python, all better ideas are welcome :)

Think more ooopsier.

#! /usr/bin/python
import datetime
import time

then = datetime.datetime.now()
print "then = ", then

time.sleep(5)

now = datetime.datetime.now()
print "now  = ", now

print "Elapsed = ", now - then


datetime.now() returns a datetime object and those objects allow for 
various types of artithmetic to be perfomed on them. :-)

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: /proc/$$/fd mystery - {t,}csh vs. bash

2009-01-08 Thread Michael ODonnell


> Because on startup /bin/csh dups stdin/out/err to higher fds...
[...]
> this means /bin/csh uses FDs 16,17,18 as it's stdin/out/err instead
> of 0/1/2.
[...]
> when running a program: after clone(), it just dup()s back to
> stdin/out/err before exec():

Cool - thanks for the analysis.  Why might they be doing this?
 
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: /proc/$$/fd mystery - {t,}csh vs. bash

2009-01-08 Thread Dave Johnson
Michael ODonnell writes:
>   e521:~ 514---> /bin/csh
>  e521:~> echo My PID is $$, contents of my /proc/$$/fd follow... ; ls -l 
> /proc/$$/fd
>  My PID is 27244, contents of my /proc/27244/fd follow...
>  total 0
>  lrwx-- 1 mod mod 64 2009-01-08 20:47 15 -> /dev/pts/0
>  lrwx-- 1 mod mod 64 2009-01-08 20:47 16 -> /dev/pts/0
>  lrwx-- 1 mod mod 64 2009-01-08 20:47 17 -> /dev/pts/0
>  lrwx-- 1 mod mod 64 2009-01-08 20:47 18 -> /dev/pts/0
>  lrwx-- 1 mod mod 64 2009-01-08 20:47 19 -> /dev/pts/0


Ah the wonders of strace

Because on startup /bin/csh dups stdin/out/err to higher fds...

execve("/bin/csh", ["/bin/csh"], [/* 31 vars */]) = 0
[... blah blah ...]
dup2(0, 16) = 16
fcntl64(16, F_SETFD, FD_CLOEXEC)= 0
dup2(1, 17) = 17
fcntl64(17, F_SETFD, FD_CLOEXEC)= 0
dup2(2, 18) = 18
fcntl64(18, F_SETFD, FD_CLOEXEC)= 0
dup2(16, 19)= 19
fcntl64(19, F_SETFD, FD_CLOEXEC)= 0
[... blah blah ...]
close(0)= 0
close(1)= 0
close(2)= 0

this means /bin/csh uses FDs 16,17,18 as it's stdin/out/err instead of
0/1/2.

when running a program:

after clone(), it just dup()s back to stdin/out/err before exec():

clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0xb7d266f8) = 11073
[... blah blah ...]
[pid 11073] close(0)= -1 EBADF (Bad file descriptor)
[pid 11073] dup(19) = 0
[pid 11073] fcntl64(0, F_SETFD, 0)  = 0
[pid 11073] close(1)= -1 EBADF (Bad file descriptor)
[pid 11073] dup(17) = 1
[pid 11073] fcntl64(1, F_SETFD, 0)  = 0
[pid 11073] close(2)= -1 EBADF (Bad file descriptor)
[pid 11073] dup(18) = 2
[pid 11073] fcntl64(2, F_SETFD, 0)  = 0
[... blah blah ...]
[pid 11073] execve("/bin/ls", ["ls", "-l", "/proc/11071/fd"], [/* 37 vars */]) 
= 0




-- 
Dave
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


/proc/$$/fd mystery - {t,}csh vs. bash

2009-01-08 Thread Michael ODonnell

OK - this is at least a minor case of WTF - why is it that
the list of active file descriptors shown when {t,}csh is
running shows no descriptor lower than 15 ?!?

  e521:~ 711---> /bin/bash
  e521:~ 513---> echo My PID is $$, contents of my /proc/$$/fd 
follow... ; ls -l /proc/$$/fd
 My PID is 27237, contents of my /proc/27237/fd follow...
 total 0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 0 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 1 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 2 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 255 -> /dev/pts/0
  e521:~ 514---> /bin/tcsh
 e521:~> echo My PID is $$, contents of my /proc/$$/fd follow... ; ls -l 
/proc/$$/fd
 My PID is 27240, contents of my /proc/27240/fd follow...
 total 0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 15 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 16 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 17 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 18 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 19 -> /dev/pts/0
 e521:~> /bin/bash
  e521:~ 513---> echo My PID is $$, contents of my /proc/$$/fd 
follow... ; ls -l /proc/$$/fd
 My PID is 27242, contents of my /proc/27242/fd follow...
 total 0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 0 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 1 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 2 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 255 -> /dev/pts/0
  e521:~ 514---> /bin/csh
 e521:~> echo My PID is $$, contents of my /proc/$$/fd follow... ; ls -l 
/proc/$$/fd
 My PID is 27244, contents of my /proc/27244/fd follow...
 total 0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 15 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 16 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 17 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 18 -> /dev/pts/0
 lrwx-- 1 mod mod 64 2009-01-08 20:47 19 -> /dev/pts/0


___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Ben Scott
On Thu, Jan 8, 2009 at 7:41 AM, Jerry Feldman  wrote:
> On 01/07/2009 04:10 PM, Ben Scott wrote:
>> The C standard provides no guarantees
>> about how struct's will be laid out in memory.  The compiler can do
>> (and often does) whatever it want.  As we see here, compilers often
>> align/pad struct's to optimize for the architecture they're targeting.
>
> The compiler is required to maintain the order of the elements in
> a struct.

  Ah, I didn't know that.  (Or if I did, I had forgotten.)  I stand corrected.

> Many times it is the calling standard that mandates alignment, not the
> compiler.

  Right, but sometimes when the compiler changes, how it implements
those mandates changes as well.  Back on the early days of GCC on the
Alpha, GCC was still evolving to do a better job.  I remember
something along the lines of a compiler update that resulted in data
structures compiled with version Y not being compatible with data
structures from version X, from the same source code.  Granted, the
code in question was probably not written with portability in mind,
but the programmer probably didn't think portability was important
because the code was "never" going to run on another machine.  He
didn't foresee a compiler update introducing portability concerns.

> Additionally, pragmas are not standard and are implementation defined.

  Absolutely.  I once read that an early version of GCC didn't
implement any pragma's, so the "implementation defined behavior" on
encountering a pragma was to attempt to start Emacs and run the
"psychoanalyze-pinhead" command.

  "When the compiler encounters [a given undefined construct] it is
legal for it to make demons fly out of your nose."  -- John Woods

-- Ben
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Ben Scott
On Thu, Jan 8, 2009 at 12:11 PM, Kevin D. Clark
 wrote:
> Ben Scott writes:
>> On Wed, Jan 7, 2009 at 11:31 PM,   wrote:
>>> ... function/method for accessing each structure member ...
>> ... only truly portable way, but it's cumbersome ...
>
> Yeah, it is cumbersome.  Sometimes programming is hard.

  Sometimes?  :)

  That comment of mine was mainly an observation that a plurality of
programmers are lazy (can be good or bad) plus ignorant and/or
apathetic (always bad), so cumbersome solutions tend to get not-used.
And certainly, from an engineering standpoint, I'm generally in favor
of less cumbersome solutions when possible.  Alas, "the best of all
worlds" is rarely this one.

> The thing is, you don't have to do this work until you run into an
> architecture/compiler that absolutely does not do what you want it to.
> Until then your code is fine.

  Sure.  To paraphrase Larry Wall, a program is "correct" if it gets
the job done, for whatever definition of "job" and "done" you like.
:)

  My problem is that as an IT management weenie, I'm the one who's
always getting burned by programmers who define getting the job done
as "compiler produces an executable, and it ran once on my machine
without crashing".  Not every programmer works that way, but it seems
like the plurality do.  Such experiences tend to affect one's
attitude.

> If you (not anybody in particular) think that code written in this
> manner isn't right and you'd prefer to use something else,
> stop using Linux or any other real-world OS.

  I (me in particular) would prefer to use something else.  I'd also
prefer that there was no war or hunger or misunderstanding in the
world.  :)  As I tried to imply in my previous post, reality is rarely
perfect.  :)

> ... ultra-high performance is not the primary goal, it is frequently better to
> stream/pass/store/etc. things as text.

  Indeed.  It's worth mentioning that many of the Internet protocols
are text-based for that very reason, and they won out over more
"sophisticated" alternatives.

-- Ben
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Jim Kuzdrall
On Thursday 08 January 2009 11:25, virgins...@vfemail.net wrote:
>
> While this could work for userland code, you might run into trouble
> in kernel code.  How would you access the "struct library" from the
> kernel?  And (more importantly!) how would you convince the kernel
> developers to adopt such a practice?
>
> Perhaps, if the universal<->native translation was done by a
> preprocessor, you could have say scsi_ioctl.uni -> scsi_ioctl.h,
> which could then be #included in the source.  That might be a little
> less cumbersome.
>
> I don't think having *.uni->*.h would solve endian issues, but endian
> issues generally aren't a problem in the kernel.  (With some notable
> exceptions: struct udphdr, struct nbd_request, etc.)

My outline over-complicated the solution - no pointer array is 
needed.  If the addresses of the structs are be obtained from the 
kernel while it is running, the data can be sent to or taken from them 
to an external "universal" format.  At the very worst, this would 
require a compile header which allocates extern pointers equated to the 
run-time struct addresses.  The translation functions can be compiled 
with the kernel or in a linked library, but they don't need any changes 
to the existing kernel code.

Admittedly, most of the code I have written over the years has been 
for use without an operating system - speed and real-time processing 
were always critical.  But, if I understand the problem correctly, it 
should not be necessary to mess with the kernel or the kernel 
developers.

Jim Kuzdrall 
> ___
> gnhlug-discuss mailing list
> gnhlug-discuss@mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Kevin D. Clark

Ben Scott writes:

> On Wed, Jan 7, 2009 at 11:31 PM,   wrote:

> > One way might be to expose a function/method for
> > accessing each structure member.
> 
> That's probably the only truly portable way, but it's cumbersome as
> hell.

Yeah, it is cumbersome.  Sometimes programming is hard.  It is hard
work to make sure that something works right in all cases.  Hopefully
every project worth doing has somebody on staff who cares about
details like this.

The thing is, you don't have to do this work until you run into an
architecture/compiler that absolutely does not do what you want it to.
Until then your code is fine.  If you (not anybody in particular)
think that code written in this manner isn't right and you'd prefer to
use something else, stop using Linux or any other real-world OS.

Basically, when you play with bits, you might get bitten.  If the
programmer who writes the code inserts enough self-checks into the
code and writes their code rigorously enough, everything is fine.  It
is also is a requirement that the designers of the system at least
think of interoperability problems up-front rather than after the
system gets deployed somewhere.  If the project doesn't have people
who can think in this way, then there will be problems.  Oh well.

...

This is why I was so angry at the programmer who removed the asserts
from my code in the blog posting that I cited.  What he did was
basically the equivalent of ripping out all of the gauges and
indicator lights from the dashboard of your car, all while uttering "I
didn't understand these things so I ripped them out".  Well, sometimes
indicator lights and gauges are very important.

Regards,

--kevin

PS All of this has one other collary: eventually, you arrive at the
   conclusion that if a given value has to be 
   streamed/passed/stored/whatever and ultra-high performance is not
   the primary goal, it is frequently better to
   stream/pass/store/etc. things as text.

-- 
"No, no, no, don't tug on that. You never know what it might
 be attached to."
-- Buckaroo Banzai
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Tom Buskey
On Thu, Jan 8, 2009 at 11:25 AM,  wrote:

> > From: Jim Kuzdrall 
> > Organization: Intrel
> > Date: Thu, 8 Jan 2009 08:20:12 -0500
>
> > On Thursday 08 January 2009 00:06, Ben Scott wrote:
> > > On Wed, Jan 7, 2009 at 11:31 PM,   wrote:
> > > > So what's the recommended way to do this?
> > >
> > >   I dunno that there really is any really good way.
> >
> > There is a solution to the problem, I believe, if the structs are
> > data structures that you wish to be common to several environments and
> > processors.
>
> Sure there is.  Just write every program in Java. ;^)


Or perl or python or shell

Jon Bentley's ACM column "Programming Pearls" (collected in a few books)
talks about the value of shell scripts & ASCII data files for portability.

He went from a multi user VAX to a Sun Sparc workstation to an Alpha and saw
his programs speed up considerably.  Because he didn't use binary files for
data, he didn't have to worry about endiness or 16 vs 32 bits.  Ok, this is
old stuff.

We've gone from 16 -> 32 -> 64 bits with various endieness in computer
evolution.  The next evolution will be linear to parallel (multi
core/thread/etc) and will be much harder.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


[GNHLUG] MerriLUG Nashua, Thur 15 Jan, Dinner at Martha's

2009-01-08 Thread Jim Kuzdrall
Who  : The friendly, suave, intelligent, knowledgeable MerriLUG group 
What : Whatever interests you
Where: Martha's Exchange
Day  : Thur 15 Jan **Next Week**
Time : 6:00 PM for grub (no upstairs discussion this month)

:: Overview

We will use this opportunity for some friendly conversation around the 
dining table.  Enjoy!

 >>> RSVP to Jim Kuzdrall for dinner to assure adequate seating. <<<
 !!! If you are not a "Regular Attendee" (50%), please let me know. !!!

Driving directions:
http://wiki.gnhlug.org/twiki2/bin/view/Www/PlaceMarthasExchange

Thanks,

Jim Kuzdrall
gnh...@intrel.com


___
gnhlug-announce mailing list
gnhlug-annou...@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-announce/

___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread VirginSnow
> From: Jim Kuzdrall 
> Organization: Intrel
> Date: Thu, 8 Jan 2009 08:20:12 -0500

> On Thursday 08 January 2009 00:06, Ben Scott wrote:
> > On Wed, Jan 7, 2009 at 11:31 PM,   wrote:
> > > So what's the recommended way to do this?
> >
> >   I dunno that there really is any really good way.
> 
> There is a solution to the problem, I believe, if the structs are 
> data structures that you wish to be common to several environments and 
> processors.

Sure there is.  Just write every program in Java. ;^)

> The solution starts with a program for which the structs are 
> working.  These structs are then read, one element at a time, by a 
> program that converts them to a "universal" format, creating a 
> universal struct.  These functions are readily compiled on any system 
> without any adjustments, as it turns out.
> 
> Once created, the structs go into a "universal" struct library which 
> can be used various ways, depending on your needs.  Your programs must 
> explicitely access all structs via a pointer array, constructed each 
> time the program loads.  The program which uses the struct first 
> translates the universal struct to its native format.  Then it replaces 
> the pointer the compiler put in the array with the location of the 
> translated struct.

While this could work for userland code, you might run into trouble in
kernel code.  How would you access the "struct library" from the
kernel?  And (more importantly!) how would you convince the kernel
developers to adopt such a practice?

Perhaps, if the universal<->native translation was done by a
preprocessor, you could have say scsi_ioctl.uni -> scsi_ioctl.h, which
could then be #included in the source.  That might be a little less
cumbersome.

I don't think having *.uni->*.h would solve endian issues, but endian
issues generally aren't a problem in the kernel.  (With some notable
exceptions: struct udphdr, struct nbd_request, etc.)
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Thomas Charron
On Thu, Jan 8, 2009 at 12:06 AM, Ben Scott  wrote:
> On Wed, Jan 7, 2009 at 11:31 PM,   wrote:
>> So what's the recommended way to do this?
>  I dunno that there really is any really good way.

int32_t and int64_t are your friends.  :-D

>> One way might be to expose a function/method for
>> accessing each structure member.
>  That's probably the only truly portable way, but it's cumbersome as hell.

  That's the 'proper' way to do it, for many reasons besides things
already mentioned.  By using successors to get and set variables you
can offer protection and insulation of internal variables from the
'cruel, cruel world'.

>  Maybe there's a better way and I just don't know it.  I'm far from a C 
> expert.

  In order to safely pass a data structure around, you really need to
be able to serialize it in some way, and then reistantiate the struct
after the fact.  If you don't, you get EXACTLY what you have here.
It's a nearly identical problem to little endian vs big endian
processors.   Dumping structs is a bad, bad, UBAH bad idea.

>> But a lot of existing code isn't written that way.
>  Correct.  Welcome to the real world.  It's defective (and I want my
> money back).

  Ok Meatloaf.  :-D

>  Linux and *BSD often get away with these non-portable practices
> because so much stuff comes with source and can be easily re-built.
> As long as everything is built with the same compiler and the same
> compiler options and for the same architecture, the same internal
> representations will be used, so nobody notices.  But then you have
> neat things like file formats which "change" depending on the hardware
> the program was compiled for.

  Do many modules have this issue?  I'm ignorant to what a whole lot
of modules are currently using.  Modules I've written use int32_t and
int64_t.

>  This is also a big part of the reason why binary compatibility is
> hard.  A binary package (.rpm, .deb, etc.) built in one environment
> (kernel + compiler + libraries + compiler options + build options +
> target architecture + ...) won't necessarily be compatible with a
> different environment.  RPM refuses to install subtly different
> packages for good reason.

  I shudder when I remember libc hell.  "Welcome to DLL Hell!  Now,
FOR LINUX!"..  :-D

-- 
-- Thomas
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: On portable C programming (was: libraw1394 struct layouts...)

2009-01-08 Thread Jim Kuzdrall
On Thursday 08 January 2009 00:06, Ben Scott wrote:
> On Wed, Jan 7, 2009 at 11:31 PM,   wrote:
> > So what's the recommended way to do this?
>
>   I dunno that there really is any really good way.

There is a solution to the problem, I believe, if the structs are 
data structures that you wish to be common to several environments and 
processors.

The problem is, as pointed out, that numeric values must be on 
certain byte boundaries for different processors.  Mixed in text does 
not, but it will end up on a boundary if it follows a number (float, 
double, integer, etc.).  Any mixed-in text requires padding that varies 
with the processor or compiler when the next number is reached.

The solution starts with a program for which the structs are 
working.  These structs are then read, one element at a time, by a 
program that converts them to a "universal" format, creating a 
universal struct.  These functions are readily compiled on any system 
without any adjustments, as it turns out.

Once created, the structs go into a "universal" struct library which 
can be used various ways, depending on your needs.  Your programs must 
explicitely access all structs via a pointer array, constructed each 
time the program loads.  The program which uses the struct first 
translates the universal struct to its native format.  Then it replaces 
the pointer the compiler put in the array with the location of the 
translated struct.

You can allow all systems to access the struct library with the 
universal format functions to keep the structs up to date.  Or, if the 
values are static, just translate the universal struct library to 
several native libraries, which can be loaded in binary to their 
respective programs.

So, what about these universal programs?  The problem is the numeric 
values.  A crude solution would create the universal struct entirely of 
ASCII by using printf(), strtol(), strtod(), etc.  (Don't use scanf(); 
it usually causes more problems than is solves.)

A more elegant way was published in the "C/C++ Users Journal" 
January 1995, page 33, entitled "Transferring Numeric Values Between 
Computers" by (guess who) James A. Kuzdrall.  P. J. Plauger was editior 
and was enthralled by the universality of the technique.  Neither the 
numeric format of the host or the target computer need be known - 
bigendian/littleendian, 32bit/64bit, 2,s compliment/BCD, none of that 
changes the source code of the translation programs.

The functions are fputi, fgeti, fputl, fgetl, fputf, fgetf.  fgetf 
translates to and from a float of 32-bits, which can be cast to native 
precision before storing in the native system.  If you need more float 
precision than 32-bits, the algorithm must be extended to double.

If anyone is interested, I will see if I can find my copy of the 
article in text format and the source code for the functions.  You must 
be pretty conversant with numeric structures to follow the reasoning, 
but using the functions is easy.

Jim Kuzdrall

P.S. If anyone wants this numeric transfer technique as this month's 
topic for MerriLUG, let me know before noon today, 08 Jan, when the 
notice goes out.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: libraw1394 struct layouts, i386 vs. x86_64

2009-01-08 Thread Jerry Feldman

On 01/07/2009 04:10 PM, Ben Scott wrote:

  But copying C headers around like that is also a bad idea, for the
very reason you're encountering: The C standard provides no guarantees
about how struct's will be laid out in memory.  The compiler can do
(and often does) whatever it want.  As we see here, compilers often
align/pad struct's to optimize for the architecture they're targeting.

  
Many times it is the calling standard that mandates alignment, not the 
compiler. The compiler is required to maintain the order of the elements 
in a struct. Calling standards usually require that a struct be aligned 
on the longest natural boundary (on a 64-bit system it would be 64-bit). 
Then elements themselves are aligned to their natural boundaries. In 
i386 you will find that ints are usually on 32-bit boundaries, but 
doubles are also on 32-bit boundaries eventhough doubles are 64-bits 
wide. Additionally, in some systems accessing an element (say 64-bit 
long) that is not properly aligned will cause an exception. Some systems 
will trap and perform a fixup resulting in poor performance, and others 
will cause the application to crash. In developing the assembler windows 
NT on the Alpha, the WNT object file format had to be packed.


Additionally, pragmas are not standard and are implementation defined. 
The behavior or #pragma pack on GCC/G++ may be much different than 
#pragma pack on Visual C++. Avoid pragmas if you can, otherwise wrap 
them in #ifdefs.


--
Jerry Feldman 
Boston Linux and Unix
PGP key id: 537C5846
PGP Key fingerprint: 3D1B 8377 A3C0 A5F2 ECBB  CA3B 4607 4319 537C 5846




signature.asc
Description: OpenPGP digital signature
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/