Re: pil64emu testing features
Hi Jon,
so the only remaining issue with eum64 is currently the fact that
fcntl() gives a runtime error on the Mac:
MacBook-Air:picoLispEmu jkleiser$ ./pil +
[/Users/jkleiser/.pil/history:1] File lock: Invalid argument
This happens when it tries to lock the history file of the line editor
in debug mode. But it will probably also happen in other cases where
files need to be locked (e.g. in the database).
The corresponding source is line 28 of src64/io.l:
(code 'lockFileAC)
st2 (Flock) # 'l_type'
ld (Flock L_START) 0 # Start position ('l_whence' is SEEK_SET)
shr A 16 # Get length
ld (Flock L_LEN) A # Length
do
cc fcntl(C F_SETLKW Flock) # Try to lock
This boils down to (line 19477 of src64/emu.base.c):
// lockFileAC:
2644, // 18454: st2 (Flock)
2645, // 18455: ld (Flock . 4) 0
2646, // 18456: shr A 16
2647, // 18457: ld (Flock . 12) A
2650, // 18458: cc fcntl (C 14 Flock)
The '14' here is the value of F_SETLKW on x86-32 Debian. It is system
dependent, and is perhaps different on the Mac.
The last line results in the call (line 39073):
case 2650: // cc fcntl (C 14 Flock)
A.l = (uint32_t)fcntl((int)C.n, 14, (int)((uint64_t)(unsigned
long)((uint8_t*)Data+7844)));
break;
So either the value '14' is wrong here, or the whole call to fcntl().
The value of F_SETLKW is taken from the standard output of the command
'src64/sysdefs', which is compiled in the build process on the host
system.
Anybody has an idea what is wrong? Any Mac programmer awake?
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Sun, Nov 04, 2012 at 09:55:27AM +0100, Alexander Burger wrote: This boils down to (line 19477 of src64/emu.base.c): // lockFileAC: 2644, // 18454: st2 (Flock) The exact line numbers are subject to change, of course. Actually, I mistakenly looked at an a bit outdated version. The above part is at line 19479 in the current release. The last line results in the call (line 39073): case 2650: // cc fcntl (C 14 Flock) A.l = (uint32_t)fcntl((int)C.n, 14, (int)((uint64_t)(unsigned long)((uint8_t*)Data+7844))); break; Same here. This is now at 39079. Just to avoid confusion ;-) -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Alex,
Hi Jon,
so the only remaining issue with eum64 is currently the fact that
fcntl() gives a runtime error on the Mac:
MacBook-Air:picoLispEmu jkleiser$ ./pil +
[/Users/jkleiser/.pil/history:1] File lock: Invalid argument
This happens when it tries to lock the history file of the line editor
in debug mode. But it will probably also happen in other cases where
files need to be locked (e.g. in the database).
The corresponding source is line 28 of src64/io.l:
(code 'lockFileAC)
st2 (Flock) # 'l_type'
ld (Flock L_START) 0 # Start position ('l_whence' is SEEK_SET)
shr A 16 # Get length
ld (Flock L_LEN) A # Length
do
cc fcntl(C F_SETLKW Flock) # Try to lock
This boils down to (line 19477 of src64/emu.base.c):
// lockFileAC:
2644, // 18454: st2 (Flock)
2645, // 18455: ld (Flock . 4) 0
2646, // 18456: shr A 16
2647, // 18457: ld (Flock . 12) A
2650, // 18458: cc fcntl (C 14 Flock)
The '14' here is the value of F_SETLKW on x86-32 Debian. It is system
dependent, and is perhaps different on the Mac.
Yes, this is from my Mac's ...Headers/sys/fcntl.h:
#define F_SETLKW9 /* F_SETLK; wait if blocked */
Does this 9 explain our fcntl() problem?
/Jon
The last line results in the call (line 39073):
case 2650: // cc fcntl (C 14 Flock)
A.l = (uint32_t)fcntl((int)C.n, 14, (int)((uint64_t)(unsigned
long)((uint8_t*)Data+7844)));
break;
So either the value '14' is wrong here, or the whole call to fcntl().
The value of F_SETLKW is taken from the standard output of the command
'src64/sysdefs', which is compiled in the build process on the host
system.
Anybody has an idea what is wrong? Any Mac programmer awake?
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Jon, 2650, // 18458: cc fcntl (C 14 Flock) The '14' here is the value of F_SETLKW on x86-32 Debian. It is system dependent, and is perhaps different on the Mac. Yes, this is from my Mac's ...Headers/sys/fcntl.h: #define F_SETLKW9 /* F_SETLK; wait if blocked */ Does this 9 explain our fcntl() problem? If it is 9 according to the include file on the Mac, is this also the value returned by 'src64/sysdefs', and which thus appears in the above 'fcntl' line (which is (correctly) 14 for Debian)? I mean, how do these lines look on your system? Do they show 9 instead of 14? If it is so, what else might be wrong with the call to fcntl()? Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Sun, Nov 04, 2012 at 01:07:40PM +0100, Alexander Burger wrote: If it is 9 according to the include file on the Mac, is this also the value returned by 'src64/sysdefs', and which thus appears in the above 'fcntl' line (which is (correctly) 14 for Debian)? I mean, how do these lines look on your system? Do they show 9 instead of 14? What I want to say: I don't care what the actual value of F_SETLKW is. It just has to be the _correct_ one on the system where it runs on. If not, then the question is where it goes wrong. -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
I'm not familiar with the syntax, but could this be the problem?
--- a/src64/io.lSat Nov 03 15:07:11 2012 +0100
+++ b/src64/io.lSun Nov 04 15:08:35 2012 +0100
@@ -26,7 +26,7 @@
(code 'rdLockFileC)
ld A F_RDLCK # Read lock, length 0
(code 'lockFileAC)
- st2 (Flock) # 'l_type'
+ st2 (Flock L_TYPE) # 'l_type'
ld (Flock L_START) 0 # Start position ('l_whence' is SEEK_SET)
shr A 16 # Get length
ld (Flock L_LEN) A # Length
On Nov 4, 2012, at 2:59 PM, Alexander Burger wrote:
On Sun, Nov 04, 2012 at 01:07:40PM +0100, Alexander Burger wrote:
If it is 9 according to the include file on the Mac, is this also the
value returned by 'src64/sysdefs', and which thus appears in the above
'fcntl' line (which is (correctly) 14 for Debian)? I mean, how do these
lines look on your system? Do they show 9 instead of 14?
What I want to say: I don't care what the actual value of F_SETLKW is.
It just has to be the _correct_ one on the system where it runs on. If
not, then the question is where it goes wrong.
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
After applying that, some tests are still failing. In test/lib/misc.l and
test/src/main.l there are checks for chdir that use /tmp. On Mac, /tmp is a
symlink to /private/tmp.
I would just change /tmp for /bin.
Also, test/src/ext.l and test/src/ht.l fail quite early:
bash-3.2$ ./pil lib/test.l $(/bin/pwd) -bye +
[test/src/ext.l:5] !? (ext:Snx PicoLisp is not Common Lisp)
ext:Snx -- Undefined
?
(commented ext.l loading)
bash-3.2$ ./pil lib/test.l $(/bin/pwd) -bye +
[test/src/ht.l:5] !? (ht:Prin 123äöüiÄÖÜß)
ht:Prin -- Undefined
((pipe (ht:Prin 123äöüiÄÖÜß) (line T)))
[test/src/ht.l:5] 1lt;2gt;3amp;äöült;igt;ÄÖÜß -- 'test' failed
?
On Nov 4, 2012, at 3:10 PM, Jorge Acereda wrote:
I'm not familiar with the syntax, but could this be the problem?
--- a/src64/io.lSat Nov 03 15:07:11 2012 +0100
+++ b/src64/io.lSun Nov 04 15:08:35 2012 +0100
@@ -26,7 +26,7 @@
(code 'rdLockFileC)
ld A F_RDLCK # Read lock, length 0
(code 'lockFileAC)
- st2 (Flock) # 'l_type'
+ st2 (Flock L_TYPE) # 'l_type'
ld (Flock L_START) 0 # Start position ('l_whence' is SEEK_SET)
shr A 16 # Get length
ld (Flock L_LEN) A # Length
On Nov 4, 2012, at 2:59 PM, Alexander Burger wrote:
On Sun, Nov 04, 2012 at 01:07:40PM +0100, Alexander Burger wrote:
If it is 9 according to the include file on the Mac, is this also the
value returned by 'src64/sysdefs', and which thus appears in the above
'fcntl' line (which is (correctly) 14 for Debian)? I mean, how do these
lines look on your system? Do they show 9 instead of 14?
What I want to say: I don't care what the actual value of F_SETLKW is.
It just has to be the _correct_ one on the system where it runs on. If
not, then the question is where it goes wrong.
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Jorge, I'm not familiar with the syntax, but could this be the problem? ... (code 'lockFileAC) - st2 (Flock) # 'l_type' + st2 (Flock L_TYPE) # 'l_type' Yes!! Very true! A stupid mistake on my side. I don't remember the reason, but somehow I must have been under the assumption that 'l_type' is always zero. Is it non-zero on the Mac? Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Nov 4, 2012, at 4:05 PM, Alexander Burger wrote: On Sun, Nov 04, 2012 at 03:56:04PM +0100, Alexander Burger wrote: I would just change /tmp for /bin. I see. The test does a chdir to /tmp, and then verifies that it is indeed there. But can we be sure that /bin isn't a symlink either on some systems? Then I would propose simply to use / - This shouldn't be a symlink on any system, right? :) Sure :-) -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Nov 4, 2012, at 3:56 PM, Alexander Burger wrote: There should be two libraries in lib/ (here, on a 32-bit x86-32): $ file lib/ht lib/ext lib/ht: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped lib/ext: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped Are they there? If not, wasn't there an error during the make process? No error. bash-3.2$ file lib/ext lib/ext: Mach-O 64-bit dynamically linked shared library x86_64 I guess there's a problem with the the symbol loading. nm lib/ext shows the symbols are prefixed with _: .. 1238 D _Snx -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Jorge, bash-3.2$ file lib/ext lib/ext: Mach-O 64-bit dynamically linked shared library x86_64 I guess there's a problem with the the symbol loading. nm lib/ext shows the symbols are prefixed with _: .. 1238 D _Snx Yeah, that would explain it. Crappy a.out format on the Mac. I wonder why it worked in pil32 (or didn't it?)? Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Sun, Nov 04, 2012 at 05:09:14PM +0100, Jorge Acereda wrote: Then I would propose simply to use / - This shouldn't be a symlink on any system, right? :) Sure :-) Released a new testing version with all your suggested changes. ♪♫ - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Alex, Things are looking much better now: MacBook-Air:picoLispEmu jkleiser$ (cd src64; make) emu.base.c: In function run: emu.base.c:42119: warning: format not a string literal and no format arguments emu.base.c:42174: warning: format not a string literal and no format arguments MacBook-Air:picoLispEmu jkleiser$ ./pil + : (load misc/fibo.l) - NIL : (fibo 9) - 34 : (bye) MacBook-Air:picoLispEmu jkleiser$ ./pil + misc/fibo.l : (fibo 9) - 34 /Jon -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Nov 4, 2012, at 5:27 PM, Alexander Burger wrote: On Sun, Nov 04, 2012 at 05:09:14PM +0100, Jorge Acereda wrote: Then I would propose simply to use / - This shouldn't be a symlink on any system, right? :) Sure :-) Released a new testing version with all your suggested changes. test/scr/main.l contains a similar test that should also be changed to /. ♪♫ - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Jon, Things are looking much better now: ... MacBook-Air:picoLispEmu jkleiser$ ./pil + : (load misc/fibo.l) - NIL : (fibo 9) - 34 Yes, indeed. MacBook-Air:picoLispEmu jkleiser$ ./pil + misc/fibo.l Oops. Please note that this is probably not what is intended, and should actually give an error. The '+' must be at the very end of the line: $ ./pil misc/fibo.l + In pil32: $ ./pil + misc/fibo.l + open: No such file or directory ? Strangely, pil64 (and also the emulator) doesn't signal an error here. Instead, it creates an empty file with the name +. I never noticed that, as I never tried to call it this way ;-) Let me investigate ... - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Nov 4, 2012, at 6:43 PM, Alexander Burger wrote: On Sun, Nov 04, 2012 at 06:12:36PM +0100, Jorge Acereda wrote: Works ok on 32 bits: bash-3.2$ ./pil lib/test.l $(/bin/pwd) -bye + OK Interesting. What might be the reason? Is the library format different between those versions? Or do we need some linker command line flag to get the proper format (i.e. holding 'Snx' instead of '_Snx')? I'm now doubting the underscore is the problem. I wrote the following test and it resolves properly the Snx symbol without underscores. test.c Description: Binary data I guess the best is printing dlerror() after the failed dlopen()/dlsym(), that might give some hints. Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Ok, no need to modify the sources, this is an interpreter, right? ;-) ? (native @ dlopen 'N lib/ext 9) (native @ dlopen 'N lib/ext 9) dyld: loaded: /Users/jacereda/picolisp/lib/ext dyld: unloaded: /Users/jacereda/picolisp/lib/ext - 0 ? (native @ dlerror 'S) (native @ dlerror 'S) - dlopen(lib/ext, 9): Symbol not found: _A^J Referenced from: /Users/jacereda/picolisp/lib/\ ext^J Expected in: flat namespace^J in /Users/jacereda/picolisp/lib/ext ? Perhaps the executable shouldn't be stripped? On Nov 4, 2012, at 6:59 PM, Jorge Acereda wrote: On Nov 4, 2012, at 6:43 PM, Alexander Burger wrote: On Sun, Nov 04, 2012 at 06:12:36PM +0100, Jorge Acereda wrote: Works ok on 32 bits: bash-3.2$ ./pil lib/test.l $(/bin/pwd) -bye + OK Interesting. What might be the reason? Is the library format different between those versions? Or do we need some linker command line flag to get the proper format (i.e. holding 'Snx' instead of '_Snx')? I'm now doubting the underscore is the problem. I wrote the following test and it resolves properly the Snx symbol without underscores. test.c I guess the best is printing dlerror() after the failed dlopen()/dlsym(), that might give some hints. Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
On Nov 4, 2012, at 7:35 PM, Jorge Acereda wrote: Ok, no need to modify the sources, this is an interpreter, right? ;-) ? (native @ dlopen 'N lib/ext 9) (native @ dlopen 'N lib/ext 9) dyld: loaded: /Users/jacereda/picolisp/lib/ext dyld: unloaded: /Users/jacereda/picolisp/lib/ext - 0 ? (native @ dlerror 'S) (native @ dlerror 'S) - dlopen(lib/ext, 9): Symbol not found: _A^J Referenced from: /Users/jacereda/picolisp/lib/\ ext^J Expected in: flat namespace^J in /Users/jacereda/picolisp/lib/ext ? Perhaps the executable shouldn't be stripped? Bingo, removing the stripping did the trick. All tests pass now. I think it would be better to just mark the symbols that should be exported. Depending on the unstripped executable seems a bit dirty. On Nov 4, 2012, at 6:59 PM, Jorge Acereda wrote: On Nov 4, 2012, at 6:43 PM, Alexander Burger wrote: On Sun, Nov 04, 2012 at 06:12:36PM +0100, Jorge Acereda wrote: Works ok on 32 bits: bash-3.2$ ./pil lib/test.l $(/bin/pwd) -bye + OK Interesting. What might be the reason? Is the library format different between those versions? Or do we need some linker command line flag to get the proper format (i.e. holding 'Snx' instead of '_Snx')? I'm now doubting the underscore is the problem. I wrote the following test and it resolves properly the Snx symbol without underscores. test.c I guess the best is printing dlerror() after the failed dlopen()/dlsym(), that might give some hints. Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: pil64emu testing features
Hi Joe, I had the same issue on another machine. I was able to get this working by changing the gcc args to -m32 in lib/native.l instead of -m64 (apply call L 'gcc -o (tmp Nm) -fPIC -m32 -shared -export-dynamic Good! I don't know enough about this to offer an opinion on whether -m64 is appropriate on a 32bit machine. It sounds like it's possible to cross You are very right. I removed the -m64 from lib/native.l and did some tests. It is not needed, and probably better (system independent) simply without the flag. Thanks! - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
