Re: Building pthreads egg on MacOS
Hi Lassi,
that's a good question!
There has been only so much consideration as to why raise the priority
in the first place: there is exactly one Chicken thread, hence it's a
precious resource. Let's raise priority.
In other words: I'd expect little harm without it. Just
inconsistency. :-/
Darwin being the culprit here, IMHO: in this simplest solution feels
appealing to me.
Anything I'm missing?
Am Wed, 25 Aug 2021 19:38:48 +0300
schrieb Lassi Kortela :
> `chicken-install pthreads` fails to compile the following C code on
> Mac:
>
> void
> C_pthread_pre_init(void *intres)
> {
>.
>.
>.
>if(pthread_setschedprio(pthread_self(),
> sched_get_priority_max(sched_getscheduler(0 {
> fprintf(stderr, "Failed to raise main thread priority.\n");
>}
> }
>
> because Darwin is missing pthread_setschedprio() and
> sched_getscheduler() -- even though these are POSIX standard
> functions.
>
> What should we do?
>
> https://stackoverflow.com/questions/12056491/how-to-set-thread-priority-pthreads-on-mac-os
>
> says "You can only set the priority of a thread when you create it
> using pthread_setschedparam()." The current thread (pthread_self())
> has already been created so we probably can't use
> pthread_setschedparam() on it.
>
> The simplest option would be an #ifdef to comment out the above lines
> of code on Darwin.
Re: Quiet re-imports
Am Wed, 24 Mar 2021 23:51:13 + schrieb Diego : > I think it would be nice to have a flag/parameter to quiet > "re-importing already imported identifier/syntax" warnings. These > seem rather noisy, and in many cases the user is probably aware of > potential collisions (as in (import r7rs), for example). > > In the attached patch to modules.scm, I've quieted the warnings based > on the load-verbose flag, but I don't think that's entirely > appropriate as this warning is more to do with importing than loading > (for example, you can (import r7rs), then (import scheme), which will > issue the warnings even though scheme is already loaded). > > Thoughts on any of this? Hm. Out of principle I tend to rather spam the user with warnings (at least by default) and contemplate how to circumvent the issue in the first place. /Jörg
Re: Default file creation mode for core/scsh-process sets executable bit
On Thu, 7 May 2020 16:43:47 +0200 Vasilij Schneidermann wrote: > Hello, > > I've noticed that when using process redirection in the scsh-process > egg, they're always marked as executable. Observe: > > ^_^ csi -R scsh-process -e '(run (ls -l) (> ls.txt))' > ^_^ ls -l ls.txt > -rwxr--r-- 1 wasa wasa 515 May 7 16:34 ls.txt > > I've checked the output of `umask` and it's 022, meaning that the > mode should be `0777 & !022` for directories and `0666 & !022` for > files, which are 0755 and 0644 respectively. Digging a bit further > this appears to be the fault of scsh-process using `file-open` > without the optional mode argument which otherwise defaults to 0744 > in posixunix.scm. This is inconsistent with the behavior of other > procedures such as `open-output-file` (0644). > > Which of the following options would you prefer? > > 1. Patching scsh-process to pass the correct file mask when using > `file-open`. > 2. Patching posixunix.scm to calculate a better default. I strongly lean towards patching posixunix.scm -- I'm totally aware that this could break things. But this default begs for security risks too much IMHO. Just my 0.02€ Jörg
Re: [PATCH] force finalizers only if finalizers exist
On Wed, 6 May 2020 13:13:24 -0400 John Cowan wrote: > The Right Thing is probably to eventually run the GC on a private > thread, at least when multiple threads are running at all. That way > the finalizers run on the same thread that doesn't hold any locks or > otherwise risk trouble. That's what the JVM and the CLR do. +1! Safe for interrupts. Better we send them to a dedicated thread (may, or may not be the same as the finilizer thread; my bet: the same). > > On Wed, May 6, 2020 at 5:42 AM wrote: > > > > There might be a situation where a thread is already executing > > > ##sys#run-pending-finalizers, but has run out of its time slice, > > > or gc interrupted or something. In this situation running > > > ##sys#run-pending-finalizers only once might not be enough. > > > > This is basically true, but with threads we make no effort to > > terminate them properly once the main thread exists - all unjoined > > threads will die anyway, > > regardless of what they are currently doing. > > > > Also, the interplay between threads and finalizers is more or less > > undefined - interrupts are executed in whatever thread happens to be > > active and manually forcing finalizers in a different thread than > > the main thread is already asking for trouble. This is a known > > problem of the threading/interrupt system and needs to be solved > > differently. > > > > > > felix > > > > > > > >
Re: [PATCH] Pick rest argument values directly from the argvector without consing
Thanks Peter for investing your time into this. While I did not proof-read your code, my bet is that this kind of optimizations do add up at the end of the day. On Oct 27 2019, Peter Bex wrote: On Sun, Oct 27, 2019 at 01:16:10PM +0100, Peter Bex wrote: This super simple benchmark runs 25% faster with -O3: (define (foo #!optional (a 1) (b 2)) (+ a b)) (time (let lp ((i 0)) (unless (= i 1) (foo 1 3) (lp (add1 i) I've tested some more, and the more rest arguments you add, the bigger the difference is (because more needs to be consed onto the stack). With 4 arguments (which are all supplied in the call), it is 40% faster. With 8 arguments it is 60% faster etc. Of course most procedures will only have a handful of rest arguments. Likely. Though I've seen graphics code, which had a huge amount of keyword args. Will this change speed them up too? Cheers, Peter
Re: [Chicken-hackers] [PATCH] fix #1648 (correctly)
On Oct 1 2019, felix.winkelm...@bevuta.com wrote: I don't think the -unroll-limit is that useful option to expose for the user. The -inline-limit already controls the amount of inlining. I couldn't get anything to unroll more than once without having to increase the inline-limit, which of course has other implications. The inline-limit considers the _size_ of a procedure to be inlined. The unroll-limit specifies the _number_ of times an inlinable procedure will actually be inlined at the same call site. I agree that the declaration/option is of limited use, but it's consisten with the declarations/options we have. This is unnecessarily O(n^2) in the number of performed inlines. I think we should use a hash-table for inline-history. I tested with some generated files and the results were: Case n=100: real0m1.784s vs. real 0m2.179s (1.2x) Case n=200: real0m7.495s vs. real 0m12.235s (1.6x) Case n=300: real0m17.738s vs. real 0m42.660s (2.4x) Case n=400: real0m34.975s vs. real 2m3.604s (3.5x) Left as an exercise to you, dear reader. A singly-linked list is simple and straightforward, I find hash-tables ugly and wasteful and will wory about scalability when the time arrives. IMHO single linked lists do have one drawback: they encode the assumption of a specific from of the mapping right into the code. When finally the time arrives, the code is usually hard to change. Especially if one does not know what exactly the code should do and why a linked list was chosen in the first place. I for one, tend to use balanced trees instead of plain lists when I worry about the downsides of hash tables (about those latter I share your view Felix) and still want to have obvious and clear code and retain the ability to change the implementation. So let me suggest to at least not hard code list traversals. /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Improve srfi-13 performance quite a bit by inlining optarg handling
Am Fri, 27 Sep 2019 23:27:23 +0200 schrieb felix.winkelm...@bevuta.com: > > Hi all, > > > > Attached is a relatively straightforward patch for SRFI-13. It > > changes the let-string-start+end macro (and also > > let-string-start+end2 but that isn't exported) so that it uses > > let-optionals* and the entire picking apart of the rest arg list > > into start/end can be completely inlined, as well as checking the > > values against the string length(s). > > Wonderful! Applied and tagged (0.3). Would it make sense to apply this also on Chicken 4? I see the latter living for quite a while. If nothing else than if it just was for me. :-/ ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] patch: STATICBUILD for mingw
Hi, STATICBUILD for mingw cross compiled for linux gave me errors of the kind __imp_SYMBOL being undefined. The attached patch is a bit questionable. I dunno what I'm doing here. But at least it went through. Best /Jörg --- Makefile.cross-linux-mingw 2017-12-11 17:46:55.0 +0100 +++ Makefile.cross-linux-mingw.modified 2019-07-09 21:24:24.103504850 +0200 @@ -53,7 +53,12 @@ C_COMPILER_OPTIMIZATION_OPTIONS ?= -Os -fomit-frame-pointer endif endif +ifdef STATICBUILD +# cheating a bit to get through +C_COMPILER_SHARED_OPTIONS = -DPIC -DC_BUILDING_LIBCHICKEN +else C_COMPILER_SHARED_OPTIONS = -DPIC +endif LINKER_LINK_SHARED_LIBRARY_OPTIONS = -shared LIBRARIES = -lm -lws2_32 LIBCHICKEN_SO_LINKER_OPTIONS = -Wl,--out-implib,lib$(PROGRAM_PREFIX)chicken$(PROGRAM_SUFFIX).dll.a ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] patch: fix STATICBUILD attempting to install .so's
Hi, using STATICBUILD failed in an attempt to install setup-api.so. rules.make hints to the culprit already. Attached patch made it work for me(TM). Best /Jörg --- rules.make.orig 2017-12-11 17:48:05.0 +0100 +++ rules.make 2019-07-10 15:04:50.217307880 +0200 @@ -379,12 +379,10 @@ $(lib).import.so "$(DESTDIR)$(IEGGDIR)" $(NL)) endif -# XXX Shouldn't this be part of the non-static lib part? +ifndef STATICBUILD $(foreach setup-lib,$(SETUP_API_OBJECTS_1),\ $(INSTALL_PROGRAM) $(INSTALL_PROGRAM_EXECUTABLE_OPTIONS) \ $(setup-lib).so "$(DESTDIR)$(IEGGDIR)" $(NL)) - -ifndef STATICBUILD ifneq ($(POSTINSTALL_PROGRAM),true) $(foreach prog,$(INSTALLED_PROGRAMS),\ $(POSTINSTALL_PROGRAM) $(POSTINSTALL_PROGRAM_FLAGS) \ ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] patch to make PROGAM_SUFFIX do the intented thing
Hi, I tried to use PROGRAM_SUFFIX to install c4 and c5 in parallel using the same prefix (against the documented suggestions, but it looked more coherent in my case). This failed with the include files installed in .../include/chicken$PROGAM_SUFFIX - which looks to me like the right thing to do) while csc would give ...include/chicken sans $PROGRAM_SUFFIX to the C compiler. The attached patch fixes this. (Tested so with native and cross compilation. However only using CHICKEN 4.13.) Best /Jörg --- defaults.make.orig 2019-07-12 21:30:24.578912914 +0200 +++ defaults.make 2019-07-12 21:30:35.702896795 +0200 @@ -424,7 +424,7 @@ $(call echo, >>, $@,# define C_TARGET_SHARE_HOME "$(TARGET_PREFIX)/share") $(call echo, >>, $@,#endif) $(call echo, >>, $@,#ifndef C_TARGET_INCLUDE_HOME) - $(call echo, >>, $@,# define C_TARGET_INCLUDE_HOME "$(TARGET_PREFIX)/include/chicken") + $(call echo, >>, $@,# define C_TARGET_INCLUDE_HOME "$(TARGET_PREFIX)/include/chicken$(PROGRAM_SUFFIX)") $(call echo, >>, $@,#endif) $(call echo, >>, $@,#ifndef C_TARGET_STATIC_LIB_HOME) $(call echo, >>, $@,# define C_TARGET_STATIC_LIB_HOME "$(TARGET_PREFIX)/lib") ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Always regenerate import libraries in setup-mode
Hi Evan, I'm not sure what the effect of this patch is going to be. I'm just a bit afraid. I have load of code using Chicken from Makefiles. Typically in setup mode and at the same time these Makefiles rely on the import libraries not being regenerated if they don't change. That's how the behavior came to exist in the first place. If they are rebuild, than basically almost all of about 100 oversized modules will have to be rebuild without good reason. Which can take the better part of an hour in my case. Am Mon, 1 Apr 2019 19:48:08 +1300 schrieb Evan Hanson : > Hi all, > > Here's a patch that addresses a bad interaction between chicken-do and > the code in core.scm that avoids regenerating import libraries when > they haven't changed. Basically, by skipping the import library, > target files can become stuck as "out of date" and that further > invocations of chicken-install will always rebuild them. > > For example: > > chicken-install -r regex > cd ~/.cache/chicken-install/regex > chicken-install -n # everything is built > touch regex.scm > chicken-install -n # so is built (as well as > static.o) chicken-install -n # so is built > chicken-install -n # so is built > # and so on ... > > This patch is a very simple solution to the problem, but it's not the > only way we could address the issue so if someone wants to propose > something more clever please be my guest. > > Cheers, > > Evan ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] food for thought - a tentative change proposal
Hi all, attached patch is not yet in final shape. I just want to put it up as food for though. It should apply atop of b2caefa738. If you ever scratched your head about issues of signal handlers running within threads not suspecting that, limitations wrt. synchronization primitives likes mutexes being usable from within signal handlers, this change might be for you. Once this is cleaned up and debugged, there should be no limitations as to what might be run from signal handlers. The commit message has more details. I encourage you to try this. While it does change semantics, I strongly believe it changes them for the better. Now I need to learn what damage it does. Merry Christmas /Jörg From a92bb87ab1f74e628980af01b88cdb7936c8c4ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Fri, 21 Dec 2018 14:36:06 +0100 Subject: [PATCH] Forward interrupts to dedicated thread when scheduler is loaded. This is a tentative change proposal; not in final shape. Currently interrupt handlers are run within the current thread. This is required, even if occationally hard to reason about, when there is only the primordial thread. It becomes a nightmare within a multi threaded environment, especially when interrupt handlers raise exceptions or attempt to use synchronization primitives. The text book solution is to forward interrupt handling to a (possibly high priority) thread. That's what this change does to scheduler.scm. Furthermore is cleans up after itself: - remove the dependency on scheduler from posixunix and posixwin I could not figure out why these where there. However loading scheduler defeats testing the single thread version. - Modify signal-test to load srfi-18 late in the game in order to actually test the single thread case too. - Add a now-required ##sys#dispatch-interrupts to posix's sleep. That's weird: How did this actually work before? Plus: It contains a tentative test case and a cleanup which really do not belong here. --- posixunix.scm | 6 -- posixwin.scm | 2 +- scheduler.scm | 48 +++- srfi-18.scm| 7 +-- tests/mutex-test.scm | 13 + tests/signal-tests.scm | 10 -- 6 files changed, 58 insertions(+), 28 deletions(-) diff --git a/posixunix.scm b/posixunix.scm index 11a00e07..431a0503 100644 --- a/posixunix.scm +++ b/posixunix.scm @@ -27,7 +27,7 @@ (declare (unit posix) - (uses scheduler irregex extras files ports lolevel) + (uses irregex extras files ports lolevel) (disable-interrupts) (hide group-member _get-groups _ensure-groups posix-error ##sys#terminal-check) (not inline ##sys#interrupt-hook ##sys#user-interrupt-hook)) @@ -1599,7 +1599,9 @@ EOF (define parent-process-id (foreign-lambda int "C_getppid")) -(define sleep (foreign-lambda int "C_sleep" int)) +(define (sleep x) + (let ((r ((foreign-lambda int "C_sleep" int) x))) +(if (zero? r) r (##sys#dispatch-interrupt (lambda () r) (define process-signal (lambda (id . sig) diff --git a/posixwin.scm b/posixwin.scm index 9f696e03..61cdfc29 100644 --- a/posixwin.scm +++ b/posixwin.scm @@ -63,7 +63,7 @@ (declare (unit posix) - (uses scheduler irregex extras files ports lolevel) + (uses irregex extras files ports lolevel) (disable-interrupts) (hide quote-arg-string) (not inline ##sys#interrupt-hook ##sys#user-interrupt-hook) diff --git a/scheduler.scm b/scheduler.scm index e8b48efd..81ea092b 100644 --- a/scheduler.scm +++ b/scheduler.scm @@ -230,14 +230,30 @@ EOF (##sys#make-thread (lambda () (let loop () - ;; inline from (thread-suspend! (current-thread)) - (##sys#setslot thread 3 'suspended) - (##sys#call-with-current-continuation - (lambda (return) - (##sys#setslot thread 1 (lambda () (return (##core#undefined - (##sys#schedule) ) ) - (loop))) - 'suspended 'signal-handler 10) + (let ((ct ##sys#current-thread)) + (##sys#setslot ct 3 'suspended) + (##sys#setslot ct 1 loop)) + (##sys#schedule))) + 'suspended 'signal-handler ##sys#default-thread-quantum) +(set! ##sys#interrupt-hook + (let ((oldhook ##sys#interrupt-hook)) + (lambda (reason state) + (cond + ((fx= reason 255) ; C_TIMER_INTERRUPT_NUMBER + (let ((ct ##sys#current-thread)) + (##sys#setslot ct 1 (lambda () (oldhook reason state))) + (##sys#schedule) ) ) ; expected not to return! + ((eq? thread ##sys#current-thread) + (oldhook reason state)) + (else + (let ((old (##sys#slot thread 1))) + (##sys#setslot thread 1 (lambda () (oldhook reason old))) + (##sys#setslot thread 3 'ready) + (##sys#add-in-front-of-ready-queue thread) + (cond #;((fx> (##sys#slot ##sys#pending-finalizers 0) 0) + (##sys#run-pending-finalizers state) ) + ((procedure? state) (##sys#thread-yield!) (state)) + (else (##sys#
[Chicken-hackers] semantics of thread-suspend! and thread-resume!
Hi, thread-suspend! and thread-resume! are extensions to srfi-18 provided by the srfi-18 unit respectively the srfi-18 egg. A warning in the source hints to the issue at hand: ;XXX what if thread is ready or blocked? The semantics are weekly defined and it easy to create pathological test cases like the one attached. How should we handle this? Currently I see two paths and two options: Paths: 1. Keep 'suspended as a possible thread state. thread-suspend! would have to be restricted to only suspend threads in 'ready or 'running state. Options: a) Error out on other states. This might possibly break existing code. But it will catch errors early. b) Ignore other states. This might incur a nightmare catching bugs silently ignored. 2. Make thread-suspend! and thread-resume! in-/decrement the priority of the subject thread starting at zero. Rules of the game: I. Any thread with a negative priority is not eligible to run. II. Future/alternative schedulers might schedule threads with a higher priority value first. For now anything >= 0 is ready to go. Anything I missed? Which way should we choose? Best /Jörg diff --git a/tests/mutex-test.scm b/tests/mutex-test.scm index ee7c2de1..97e24e81 100644 --- a/tests/mutex-test.scm +++ b/tests/mutex-test.scm @@ -225,6 +225,18 @@ Slot Type Meaning (print "Forcing primordial broke mutex-unlock! result: " r (mutex-specific mux)) (test-exit 1))) +(let* ((mux (let ((m (make-mutex 'done))) (mutex-lock! m) m)) + (t (thread-start! + (lambda () + (print "Waiting for mutex " mux " in " (current-thread)) + (let ((r (mutex-lock! mux))) + (print "mutex-lock! succeeded on locked mutex " mux " result " r) + (if r (test-exit 1))) + (thread-yield!) + (thread-suspend! t) + (thread-resume! t) + (print "Thread " t " now in state " (thread-state t))) + (cond-expand (dribble (define-for-syntax count 0) (define-syntax trail ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] [PATCH] fix issues regarding force-primordial
Hi all, up to now the primordial thread must sit in thread-join! for force-primordial to do the right thing. It would be the wrong solution to go after all synchronization primitives adding an alternative micro-threading to dispatch signals. This would hopelessly complicate the code. Hence the alternative solution to dispatch signals to a dedicated thread. 0001 and 0002 apply on chicken 4.13 after the patches I sent today, i.e., atop of a062862d. 0001 adds test cases, 0002 brings the fix and removes the special case from thread-join!. 0003 does the same for master atop of 5da7a55b. 0004 goes to srfi-18 Note: 1. That the name ##sys#force-primordial becomes a misnomer this way. It should be renamed, hidden and removed from the not-inline declaration after testing. (Which in turn will break those tests, hence I left it in for now.) 2. We /might/ want to modify this slightly to add the signal handler thread in front of the ready queue, not at the end. Best /Jörg From 3c40b133e7464dfd4eb9e2c7c1d2a1e8e2b47096 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Wed, 19 Dec 2018 17:01:09 +0100 Subject: [PATCH 1/2] Add test cases raising issues wrt. force-primordial. --- tests/mutex-test.scm | 48 1 file changed, 48 insertions(+) diff --git a/tests/mutex-test.scm b/tests/mutex-test.scm index 035d7092..ee7c2de1 100644 --- a/tests/mutex-test.scm +++ b/tests/mutex-test.scm @@ -177,6 +177,54 @@ Slot Type Meaning (print "Got " mux " state " (mutex-state mux) " expected " (current-thread) "\n") (test-exit 1)) +(let* ((job (lambda () + (do ((i 1 (add1 i))) + ((= i 10) i) + (##sys#force-primordial) + (thread-yield! + (t (thread-start! job)) + (r (thread-join! t))) + (when (not (equal? r 10)) + (print "Forcing primordial broke thread-join! result: " r) + (test-exit 1))) + +(let* ((mux (make-mutex 'done)) + (job (lambda () + (do ((i 1 (add1 i))) + ((= i 10) + (mutex-specific-set! mux i) + (mutex-unlock! mux)) + (##sys#force-primordial) + (thread-yield! + (t (begin + (mutex-lock! mux) + (thread-start! job))) + (r (mutex-lock! mux))) + (when (not (and (eq? r #t) + (equal? (mutex-specific mux) 10))) + (print "Forcing primordial broke mutex-lock! result: " r (mutex-specific mux)) + (test-exit 1))) + +(let* ((mux (make-mutex 'done)) + (cv (make-condition-variable)) + (job (lambda () + (do ((i 1 (add1 i))) + ((= i 10) + (mutex-lock! mux) + (mutex-specific-set! mux i) + (condition-variable-signal! cv) + (mutex-unlock! mux)) + (##sys#force-primordial) + (thread-yield! + (t (begin + (mutex-lock! mux) + (thread-start! job))) + (r (mutex-unlock! mux cv))) + (when (not (and (eq? r #t) + (equal? (mutex-specific mux) 10))) + (print "Forcing primordial broke mutex-unlock! result: " r (mutex-specific mux)) + (test-exit 1))) + (cond-expand (dribble (define-for-syntax count 0) (define-syntax trail -- 2.11.0 From b2caefa73890fb7e502c0323d81aa36c1e60685b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Wed, 19 Dec 2018 17:39:23 +0100 Subject: [PATCH 2/2] Use dedicated thread instead of forcing primordial for signals. --- scheduler.scm | 27 +++ srfi-18.scm | 20 ++-- 2 files changed, 29 insertions(+), 18 deletions(-) diff --git a/scheduler.scm b/scheduler.scm index de008a1d..e8b48efd 100644 --- a/scheduler.scm +++ b/scheduler.scm @@ -222,10 +222,29 @@ EOF [(eq? (##sys#slot nt 3) 'ready) (switch nt)] [else (loop2)] ) ) ) ) ) ) -(define (##sys#force-primordial) - (dbg "primordial thread forced due to interrupt") - (##sys#setislot ##sys#primordial-thread 13 #f) - (##sys#thread-unblock! ##sys#primordial-thread) ) +(define ##sys#force-primordial + (let* ((thread #f) + (install! + (lambda () + (set! thread + (##sys#make-thread + (lambda () + (let loop () + ;; inline from (thread-suspend! (current-thread)) + (##sys#setslot thread 3 'suspended) + (##sys#call-with-current-continuation + (lambda (return) + (##sys#setslot thread 1 (lambda () (return (##core#undefined + (##sys#schedule) ) ) + (loop))) + 'suspended 'signal-handler 10) +(install!) +(lambda () + (dbg "signal thread forced due to interrupt") + ;; inline from thread-resume! + (unless (eq? (##sys#slot thread 3) 'suspended) (install!)) + (##sys#setslot thread 3 'ready) + (##sys#add-to-ready-queue thread))) ) (define ready-queue-head '()) (define ready-queue-tail '()) diff --git a/srfi-18.scm b/srfi-18.scm index dbb572bb..5f06798f 100644 --- a/srfi-18.scm +++ b/srfi-18.scm @@ -181,20 +181,12 @@ (##sys#make-structure 'condition '(uncaught-exception) (list '(uncaugh
[Chicken-hackers] some patches for scheduler and srfi-18
Hi all,
attached several patches I owe you.
Patches 0001-0004 should apply on chicken 4.13, i.e. 68eeaaef3fc.
Patches 0005 and 0006 are against master (9f55823852).
Patches 0007 and 0008 are against srfi-18 egg source.
0002 - Fix 1564: This establishes the invariant that slot #11, the
blocking object and thread being in the relevant blocking queue are
always in sync. This eases reasoning about the code and fixes the
bug. Added advantage: Having a single spot to clear blocking objects
might allow eggs to introduce additional objects to block on.
0003 - Test case regarding abandoned mutex handling: This warrents
doube checking wrt. srfi-18. The test checks for my understanding of
srfi-18 and fails without 0004 applied.
0004- Change abandoned mutexs state according to srfi-18: This passes
the test cases added by 0003. Additionally it establishes a similar
invariant as 0002 wrt. slot #4 and the timeout queue. Contains
consistency checks to be phased out after testing.
Best
/Jörg
From 7288c18082a6334be0548e2e23ca13921f99076f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?=
Date: Tue, 18 Dec 2018 14:26:03 +0100
Subject: [PATCH 2/4] Add test case catching #1564 almost for sure.
---
tests/mutex-test.scm | 23 +++
1 file changed, 23 insertions(+)
diff --git a/tests/mutex-test.scm b/tests/mutex-test.scm
index 873e812c..738e73d3 100644
--- a/tests/mutex-test.scm
+++ b/tests/mutex-test.scm
@@ -55,6 +55,29 @@ Slot Type Meaning
(print "Got " mux1 " state " (mutex-state mux1) " expected " owner1 "\n")
(test-exit 1)
+(let ((m1 (make-mutex)))
+ ;; This fails if we manage to sort primorial before t1 and unleash
+ ;; both in one turn.
+ (define (sys-thread-sleep! limit)
+;; a copy from srfi-18 which expects pre-computed goal time.
+(##sys#call-with-current-continuation
+ (lambda (return)
+ (let ((ct ##sys#current-thread))
+ (##sys#setslot ct 1 (lambda () (return (##core#undefined
+ (##sys#thread-block-for-timeout! ct limit)
+ (##sys#schedule) ) ) ) )
+ #;(print "mutex state changes atomically wrt. blocking queues")
+ (mutex-lock! m1)
+ (let ((t1 (thread-start! (lambda () (mutex-lock! m1 0.1)
+#;(print "have t1 it wait for m1")
+(thread-yield!)
+(let* ((to (##sys#slot t1 4))
+ (hit (- to 0.0001)))
+ #;(print "waiting ever so slightly less than to " to " i.e. " hit "\n")
+ (sys-thread-sleep! hit))
+;; catch inconsistent state
+(mutex-unlock! m1)))
+
(set! mux1 (make-mutex 'unlock-leaves-no-memory-leak))
(mutex-lock! mux1)
(mutex-unlock! mux1)
--
2.11.0
From b6837b2c94feb5f8348965f538b5a45bf01a7506 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?=
Date: Mon, 3 Dec 2018 21:06:26 +0100
Subject: [PATCH 1/4] Fix 1564 internal scheduler error.
---
scheduler.scm | 80 ++-
1 file changed, 41 insertions(+), 39 deletions(-)
diff --git a/scheduler.scm b/scheduler.scm
index 0b292f7f..a1a03293 100644
--- a/scheduler.scm
+++ b/scheduler.scm
@@ -34,7 +34,7 @@
;; This isn't hidden ATM to allow set!ing it as a hook/workaround
; ##sys#force-primordial
fdset-set fdset-test create-fdset stderr
- ##sys#clear-i/o-state-for-thread! ##sys#abandon-mutexes)
+ ##sys#thread-clear-blocking-state! ##sys#abandon-mutexes)
(not inline ##sys#interrupt-hook ##sys#force-primordial)
(unsafe)
(foreign-declare #<= now tmo1) ; timeout reached?
(begin
(##sys#setislot tto 13 #t) ; mark as being unblocked by timeout
- (##sys#clear-i/o-state-for-thread! tto)
+ (##sys#thread-clear-blocking-state! tto)
(##sys#thread-basic-unblock! tto)
(loop (cdr lst)) )
(begin
@@ -335,17 +335,9 @@ EOF
(define (##sys#thread-kill! t s)
(dbg "killing: " t " -> " s ", recipients: " (##sys#slot t 12))
(##sys#abandon-mutexes t)
- (let ((blocked (##sys#slot t 11)))
-(cond
- ((##sys#structure? blocked 'condition-variable)
- (##sys#setslot blocked 2 (##sys#delq t (##sys#slot blocked 2
- ((##sys#structure? blocked 'thread)
- (##sys#setslot blocked 12 (##sys#delq t (##sys#slot blocked 12) )
(##sys#remove-from-timeout-list t)
- (##sys#clear-i/o-state-for-thread! t)
+ (##sys#thread-clear-blocking-state! t)
(##sys#setslot t 3 s)
- (##sys#setislot t 4 #f)
- (##sys#setislot t 11 #f)
(##sys#setislot t 8 '())
(let ((rs (##sys#slot t 12)))
(unless (null? rs)
@@ -353,13 +345,15 @@ EOF
(lambda (t2)
(dbg " checking: " t2 " (" (##sys#slot t2 3) ") -> " (##sys#slot t2 11))
(when (eq? (##sys#slot t2 11) t)
- (##sys#thread-basic-unblock! t2) ) )
- rs) ) )
- (##sys#setislot t 12 '()) )
+ (##sys#thread-unblock! t2) ) )
+ rs)
+ (##sys#setislot t 12 '()) ) ) )
(define (##sys#thread-basic-unblock! t)
(dbg "unblocking: " t)
- (##sys#setislot t 11 #f) ; (FD . RWFLAGS) | # | #
+ #;(if (
[Chicken-hackers] regarding 1564
Hi all, I just noticed that the fix for #1564 would introduce a fresh bug. Stupid me has sent out a version where the debug code was commented out. A patch will take several days to come around, sorry. The issue is in scheduler.scm ##sys#thread-basic-unblock! It used to begin (define (##sys#thread-basic-unblock! t) (dbg "unblocking: " t) (##sys#setislot ct 11 #f) me changed that to (define (##sys#thread-basic-unblock! t) (dbg "unblocking: " t) #;(if (##sys#slot t 11) ;; remove this case after testing (##sys#error '##sys#thread-basic-unblock! "Internal scheduler error: unclean unblock" (##sys#slot t 11))) The idea of the change was to collect all changes to slot #11 of a thread at a single spot. Namely, ##sys#thread-clear-blocking-state! . This time the removals only. Doing so would have the advantage that we could export the procedure from the scheduler for eggs to overwrite it with versions aware of custom blocking objects. Though when I went over the consequence of the fix - the removal of the now possibly obsolete code in srfi-18, I learned that I should have left the comment out, then it would have caught the issue. The proposed fix is to keep both for now: (define (##sys#thread-basic-unblock! t) (dbg "unblocking: " t) (##sys#setislot ct 11 #f) ;; required by condition-variable-*! (if (##sys#slot t 11) ;; remove this case after testing (##sys#error '##sys#thread-basic-unblock! "Internal scheduler error: unclean unblock" (##sys#slot t 11))) Later we should decide how to handle slot #11. Either (for performance) manage the blocking queue inline in srfi-18 and clean slot #11 there inline too. Alternatively change condition-variable-*! to use ##sys#thread-unblock! - which would unqueue the thread and clean slot #11 - and use a version of ##sys#thread-basic-unblock! which expects slot #11 to be cleaned already. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
On Dec 3 2018, Peter Bex wrote: On Mon, Dec 03, 2018 at 10:46:38AM +0100, Jörg F. Wittenberger wrote: So for me the question remains: wouldn't it be much, much more efficient to work sort-of hand-in-hand with one of the core developers, or maybe on the list to get the remaining things (bugs and improvements) fixed and reviewed? I think this would be quite helpful. Perhaps at another hackathon we can sit down together (ideally with more than one core developer to ensure we all are on the same page and understand it). Agreed. Or maybe the list? Could take time to find a chance to meet. This is one of the ugly truths of open source collaborative development; you really have to have a good plan on how to communicate the changes you're making back to "upstream",[...] Dropping a complex patch is generally not the way to go about adding code to an existing system.[...] eyeball it for obvious mistakes and other quality issues. Too true. Plus: it depends on the culture of the project. Try to put yourself in my shoes for a moment. Up to porting to chicken, I mostly contributed to rscheme, which was a one man show. When I hit an issue, I'd send a vague patch and back came a completely rewritten one after a day or two. Though it generally where small issues or rare failures of complex optimizations going wrong. This also means that the submitted code has to be so simple that others who aren't familiar with it can study it and debug it if issues crop up (and they will, with any sizable change). The scheduler is a major pain point regarding this, since concurrency is difficult enough (or impossible?) to understand at all, regardless of the quality of the code in the scheduler (which isn't stellar to begin with). So when I evaluated chicken, I found it to be a compiler producing slightly faster code than rscheme. First tests went well. Then I invested a lot of time until I could run a more sizable piece of code. Just to run into all sorts of issues. Taking #1564 for an example. It can be quite worse than just killing the program: When I ran into it, I was not always so lucky to find the thread piled up in the waiting queue to be in a state the consistency check would complain about. When the thread was blocked for a different mutex (hence sitting in two waiting queues at the same time), the mutex-unlock! would happily unblock it - thus stealing the other mutex from the third thread holding it. This kind of poked fun at the idea to use them for synchronization. At that point it did _not_ occur to me that my code would be especially complex a thing. Not did I assume nobody else had ever run more than toy examples on load-free systems. I assumed that it was obvious how badly broken it was. (And I did not foresee this not coming up elsewhere for a decade.) Sure I felt bad to have to bring up such a huge patch. But it fixed several interrelated bugs plus two counts of Big-O reduction of complexity. I might have expected some questions, comments etc. Certainly not being completely ignored. So I tried to push it for a while. Same goes for issues, which literally went against the text book examples used to teach how not to do things like not using dummy head lists in C - something I did not believe anybody would do. At least I expected the respective patch to be welcome. Especially as it was quite a job to actually change a large file and then test the results before submitting. Eventually I took being ignored as unwarrented... ...and lost faith in the project. So at some point, merging a large change is a bit of an act of faith. It also requires trust, which needs to be built up over time by showing consistent quality patches and commitment to the project. This is the really hard bit, especially if you just want one specific feature to be added and don't have that many other things to contribute to the system simply because it works for you. Yeah, I just needed a compiler as rscheme was dead. I did not want to turn into a teacher. Little hope I had that I could ever get something fed back upstream. Hence I did no longer try to. I don't have a good solution for this, but your suggestion to walk through the code together seems like a good one to me. Agreed. Chicken is not that bad. It just has a couple of rough edges. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH: Re: Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Attached a version against master. This only appears to be correct. :-/ I messed up with the prefix, thus srfi-18 did not load and I will really not find the time to come back to the issue in a timely manner. On Dec 3 2018, Jörg F. Wittenberger wrote: Attached a patch against 4.13 master still compiling On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". (cond-expand (chicken-5 (import (chicken base)) (import (chicken time)) (import srfi-18)) (else (import chicken) (use srfi-18))) (define m (make-mutex)) (print "@@ " (current-thread) " " "lock") (mutex-lock! m) (define t (current-milliseconds)) (define (get-tosleep) (/ (floor (* 1000 (- (+ t .030) (current-milliseconds 1000)) (thread-start! (make-thread (lambda () ;; (thread-sleep! .01) (print "@@ " (current-thread) " " "lock") (let lp () (when (not (mutex-lock! m (get-tosleep))) (thread-yield!) (lp))) (print "@@ " (current-thread) " " "unlock") (mutex-unlock! m (print "@@ " (current-thread) " " "sleep") (thread-sleep! (get-tosleep)) (print "@@ " (current-thread) " " "unlock") (mutex-unlock! m) (thread-yield!) (thread-sleep! .01) (print "All ok!!") --- typical output of a failing execution: $ stdbuf -oL -eL ./t |& cat -n 1 @@ # lock 2 #: locking # 3 @@ # sleep 4 # blocks for timeout 933.0 5 scheduling, current: #, ready: (#) 6 timeout: # -> 933.0 (now: 904) 7 switching to # 8 @@ # lock 9 #: locking # 10 # blocks for timeout 933.0 11 # sleeping on mutex mutex0 12 scheduling, current: #, ready: () 13 timeout: # -> 933.0 (now: 904) 14 timeout: # -> 933.0 (now: 934) 15 timeout expired for # 16 unblocking: # 17 timeout: # -> 933.0 (now: 934) 18 timeout expired for # 19 unblocking: # 20 switching to # 21 @@ # unlock 22 #: unlocking mutex0 23 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready 27 28 Call history: 29 30 t.scm:27: chicken.base#print 31 t.scm:28: get-tosleep 32 t.scm:15: chicken.time#current-milliseconds 33 t.scm:15: scheme#floor 34 t.scm:15: scheme#/ 35 t.scm:28: srfi-18#thread-sleep! 36 t.scm:29: srfi-18#current-thread 37 t.scm:29: chicken.base#print 38 t.scm:30: srfi-18#mutex-unlock! <-- (There's an extra debug message on line 15. Add (dbg "timeout expired for " tto) in this true branch: (if (>= now tmo1) ; timeout reached? in ##sys#schedule) --- The issue mutex-unlock! makes the decision that a thread freed from the mutex's waiting list cannot be in the 'ready state. From the output above you see a case how a thread waiting on a mutex can end up being in the 'ready state. line 2: The mutex is locked by primordial thread (pt) line 4: The pt goes to sleep until 933.0 line 7: As the pt goes to sleep thread1 is scheduled to run line 10: thread1 tries to lock the mutex, but sets a timeout that happens to be at time 933.0 lines 12-14: Both threads asleep, time advances to 934 lines 15-16: pt gets put on the ready list lines 17-19: thread1 gets put on the ready list line 20: pt starts running lines 21-22: pt executes mutex-unlock! while thread1 is ready to run --- A fix Just allow the 'ready state for threads in mutex-unlock! In the patch I arbitrarily call ##sys#schedule after removing a thread from the list, but I think doing nothing would work equally well. Is this a correct fix? Sorry, I can't help with that one.. Maybe it's possible there's threads on the waiting list, but the thread that gets removed is not going to lock the mutex: There are 3 threads in this scenario, A, B and C. * A locks mutex * A sleeps until t * B tries to lock mutex until t * C tries to lock mutex * A and B are woken up at t * A unlocks mutex, frees B * B is scheduled to run as per the patch * B finds out about the timeout, gives up and starts doing something else * Now thread C is waiting on the mutex but no-one is going to free it! From 307e9d806f421bd13e4b6f30a8cdb86378b8c1dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Mon, 3 Dec 2018 22:22:05 +0100 Subject: [PATCH] Fix 1564 internal scheduler error. --- scheduler.scm | 79 +++ 1 file changed, 41 insertions(+), 38 deletions(-) diff --git a/scheduler.scm b/schedule
[Chicken-hackers] PATCH: Re: Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Attached a patch against 4.13 master still compiling On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". (cond-expand (chicken-5 (import (chicken base)) (import (chicken time)) (import srfi-18)) (else (import chicken) (use srfi-18))) (define m (make-mutex)) (print "@@ " (current-thread) " " "lock") (mutex-lock! m) (define t (current-milliseconds)) (define (get-tosleep) (/ (floor (* 1000 (- (+ t .030) (current-milliseconds 1000)) (thread-start! (make-thread (lambda () ;; (thread-sleep! .01) (print "@@ " (current-thread) " " "lock") (let lp () (when (not (mutex-lock! m (get-tosleep))) (thread-yield!) (lp))) (print "@@ " (current-thread) " " "unlock") (mutex-unlock! m (print "@@ " (current-thread) " " "sleep") (thread-sleep! (get-tosleep)) (print "@@ " (current-thread) " " "unlock") (mutex-unlock! m) (thread-yield!) (thread-sleep! .01) (print "All ok!!") --- typical output of a failing execution: $ stdbuf -oL -eL ./t |& cat -n 1 @@ # lock 2 #: locking # 3 @@ # sleep 4 # blocks for timeout 933.0 5 scheduling, current: #, ready: (#) 6 timeout: # -> 933.0 (now: 904) 7 switching to # 8 @@ # lock 9 #: locking # 10 # blocks for timeout 933.0 11 # sleeping on mutex mutex0 12 scheduling, current: #, ready: () 13 timeout: # -> 933.0 (now: 904) 14 timeout: # -> 933.0 (now: 934) 15 timeout expired for # 16 unblocking: # 17 timeout: # -> 933.0 (now: 934) 18 timeout expired for # 19 unblocking: # 20 switching to # 21 @@ # unlock 22 #: unlocking mutex0 23 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready 27 28 Call history: 29 30 t.scm:27: chicken.base#print 31 t.scm:28: get-tosleep 32 t.scm:15: chicken.time#current-milliseconds 33 t.scm:15: scheme#floor 34 t.scm:15: scheme#/ 35 t.scm:28: srfi-18#thread-sleep! 36 t.scm:29: srfi-18#current-thread 37 t.scm:29: chicken.base#print 38 t.scm:30: srfi-18#mutex-unlock! <-- (There's an extra debug message on line 15. Add (dbg "timeout expired for " tto) in this true branch: (if (>= now tmo1) ; timeout reached? in ##sys#schedule) --- The issue mutex-unlock! makes the decision that a thread freed from the mutex's waiting list cannot be in the 'ready state. From the output above you see a case how a thread waiting on a mutex can end up being in the 'ready state. line 2: The mutex is locked by primordial thread (pt) line 4: The pt goes to sleep until 933.0 line 7: As the pt goes to sleep thread1 is scheduled to run line 10: thread1 tries to lock the mutex, but sets a timeout that happens to be at time 933.0 lines 12-14: Both threads asleep, time advances to 934 lines 15-16: pt gets put on the ready list lines 17-19: thread1 gets put on the ready list line 20: pt starts running lines 21-22: pt executes mutex-unlock! while thread1 is ready to run --- A fix Just allow the 'ready state for threads in mutex-unlock! In the patch I arbitrarily call ##sys#schedule after removing a thread from the list, but I think doing nothing would work equally well. Is this a correct fix? Sorry, I can't help with that one.. Maybe it's possible there's threads on the waiting list, but the thread that gets removed is not going to lock the mutex: There are 3 threads in this scenario, A, B and C. * A locks mutex * A sleeps until t * B tries to lock mutex until t * C tries to lock mutex * A and B are woken up at t * A unlocks mutex, frees B * B is scheduled to run as per the patch * B finds out about the timeout, gives up and starts doing something else * Now thread C is waiting on the mutex but no-one is going to free it! From b6837b2c94feb5f8348965f538b5a45bf01a7506 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Mon, 3 Dec 2018 21:06:26 +0100 Subject: [PATCH] Fix 1564 internal scheduler error. --- scheduler.scm | 80 ++- 1 file changed, 41 insertions(+), 39 deletions(-) diff --git a/scheduler.scm b/scheduler.scm index 0b292f7f..a1a03293 100644 --- a/scheduler.scm +++ b/scheduler.scm @@ -34,7 +34,7 @@ ;; This isn't hidden ATM to allow set!ing it as a hook/workaround ; ##sys#force-primordial fdset-set fdset-test create-fdset stderr - ##sys#clear-i/o-state-for-thread! ##sys#abandon-mutexes) + ##sys#thread-clear-blocking-state! ##sys#abandon-mutexes) (not inline ##sys#interrupt-hook ##sys#force-primordial) (unsafe) (foreign-declare #<= now tmo1) ; timeout reached? (begin (##sys#set
Re: [Chicken-hackers] Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Thanks you so much Kon, reviewing these logs helped to confirm my feelings. Feelings, not findings. Yet. Tinkering with these scheduler/srfi-18 issues again really made me feel bad and sorry. In fact the anger has cost me the sleep of the better half of the night. Still enrages me. Whats going on here IMHO is that a lot of lifetime, your guys and mine, is wasted. At the same time the code quality of the result is likely worse that what I'm using as the source to cut out those patches. As I can't outright proof this statement to you, let me recap the background for a moment: Around a decade ago I ported a rather thread-heavy thing (Askemos, which technically is something partially inspired by Erlang, bearing similarities to Termite - except that those processes are all made persistent and the states is replicated and synchronized in byzantine agreement over a part of the network; you might be able to imagine that this is really stressing the threading capabilities of the language in use) from rscheme to chicken. The code was at that time grown for ~7yrs; that's almost 100 modules, which took some months to port. ... ...Only to learn that the threading in chicken was not at all up for the job. Hence I spend a few more weeks fixing that one. Including adding an prio queue for timeout- and fd-list. What I could NOT produce where test cases for each of the bugs (1231, 1232, 1255, 1564 - like these are not all) I fixed in the process. Nor was is feasible to fix them one-by-one. (Yesterday evening I failed to properly backport the fix for 1564 into the ugly code implementing the timeout queue -- while asking myself why the hell it is useful; this queue should be replaced with a better version anyway.) The result I posted on chicken-users at that time. It was a complex fix. Sure. But those where sort of interrelated bugs. Then for about seven years I sadly maintained a chicken fork (which I'm still using in production) for these differences in order to be able to use chicken at all. Since 4.12 it is at least _possible_ to run this code on stock chicken. Partly because I changed my code to avoid triggering bugs remaining. So for me the question remains: wouldn't it be much, much more efficient to work sort-of hand-in-hand with one of the core developers, or maybe on the list to get the remaining things (bugs and improvements) fixed and reviewed? It would be so much more satisfying to me to actually produce code I could approve myself than backport yet another hotfix -- creating a result in the process I take issues with. (((Going into details, I'd probably do the prio-queue different today as I learned about chickens performance details. And I'm ready to do so. But at least I'd like to be allowed to use a prio queue using a proper interface than kludging inline handling of a linear list into well tested code -- likely creating fresh bugs in the process.))) Best /Jörg On Dec 2 2018, Kon Lovett wrote: see attached git (C4) & svn (C5) logs #(in C4 core local repo) git log --follow -p -- srfi-18.scm >srfi-18.log #(in C5 svn local repo) svn log --diff trunk >srfi-18_trunk-diff.log hth On Dec 2, 2018, at 1:19 PM, Jörg F. Wittenberger wrote: Thanks for the replies, chicken-install -r srfi-18 ; did the trick already I should have stated that that's what I have, what I've been looking for was the git history. I wonder for some statements why the hell they are there at all. Two possible reasons: a) I cleaned them up for being obsolete (due to former changes I made) b) removed since I touched the file, which begs the question "why where those added". Never mind. I can proceed at least. On Dec 2 2018, Kon Lovett wrote: well, that shows me. ;-) trying to track down why #497 $ chicken-install -r srfi-18 mapped (srfi-18) to () retrieving ... On Dec 2, 2018, at 10:42 AM, Kon Lovett wrote: C5 evicted srfi-18, along w/ srfi-1, 13, 14, & 69, to the egg store. chicken-install -retrieve. On Dec 2, 2018, at 10:39 AM, Jörg F. Wittenberger wrote: Hi all, when I tried to reply in a timely manner I apparently sent out a link to a broken file. Sorry for that. Just wanted to see if I could create a patch for the current master. For this I need srfi-18 egg source too. Just I can't find it. Jöry On Nov 30 2018, Jörg F. Wittenberger wrote: Hello Megane, On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". ... 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready This bears an uncanny resemblance to scheduler issues I've been fighting a long ago. Too long to ago. --- A fix Just allow the 'ready state for threads in mutex-unlock! ... Is this a correct fix? Too long ago. But it feels wrong. We'd rather make sure there is no ready thread in
Re: [Chicken-hackers] Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Thanks for the replies, chicken-install -r srfi-18 ; did the trick already I should have stated that that's what I have, what I've been looking for was the git history. I wonder for some statements why the hell they are there at all. Two possible reasons: a) I cleaned them up for being obsolete (due to former changes I made) b) removed since I touched the file, which begs the question "why where those added". Never mind. I can proceed at least. On Dec 2 2018, Kon Lovett wrote: well, that shows me. ;-) trying to track down why #497 $ chicken-install -r srfi-18 mapped (srfi-18) to () retrieving ... On Dec 2, 2018, at 10:42 AM, Kon Lovett wrote: C5 evicted srfi-18, along w/ srfi-1, 13, 14, & 69, to the egg store. chicken-install -retrieve. On Dec 2, 2018, at 10:39 AM, Jörg F. Wittenberger wrote: Hi all, when I tried to reply in a timely manner I apparently sent out a link to a broken file. Sorry for that. Just wanted to see if I could create a patch for the current master. For this I need srfi-18 egg source too. Just I can't find it. Jöry On Nov 30 2018, Jörg F. Wittenberger wrote: Hello Megane, On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". ... 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready This bears an uncanny resemblance to scheduler issues I've been fighting a long ago. Too long to ago. --- A fix Just allow the 'ready state for threads in mutex-unlock! ... Is this a correct fix? Too long ago. But it feels wrong. We'd rather make sure there is no ready thread in the queue waiting for a mutex in the first place. Diffing the changes I maintained quite a while back http://ball.askemos.org/Ad60e3fb123a79b2e5128915116b288f7/chicken-4.9.1-ball.tar.gz you will find that I added a ##sys#thread-clear-blocking-state! Towards the end of scheduler.scm and used it for consistency whereever I ran into not-so-clean unlocks. Now this is still an invasive change. But looking at the source of scheduler and srfi-18 in chicken 5 right now, I can't fight the feeling that it is working around the missing changes at several places. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Hi all, when I tried to reply in a timely manner I apparently sent out a link to a broken file. Sorry for that. Just wanted to see if I could create a patch for the current master. For this I need srfi-18 egg source too. Just I can't find it. Jöry On Nov 30 2018, Jörg F. Wittenberger wrote: Hello Megane, On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". ... 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready This bears an uncanny resemblance to scheduler issues I've been fighting a long ago. Too long to ago. --- A fix Just allow the 'ready state for threads in mutex-unlock! ... Is this a correct fix? Too long ago. But it feels wrong. We'd rather make sure there is no ready thread in the queue waiting for a mutex in the first place. Diffing the changes I maintained quite a while back http://ball.askemos.org/Ad60e3fb123a79b2e5128915116b288f7/chicken-4.9.1-ball.tar.gz you will find that I added a ##sys#thread-clear-blocking-state! Towards the end of scheduler.scm and used it for consistency whereever I ran into not-so-clean unlocks. Now this is still an invasive change. But looking at the source of scheduler and srfi-18 in chicken 5 right now, I can't fight the feeling that it is working around the missing changes at several places. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Regarding #1564: srfi-18: (mutex-unlock) Internal scheduler error
Hello Megane, On Nov 30 2018, megane wrote: Hi, Here's another version that crashes quickly with "very high probability". ... 24 Error: (mutex-unlock) Internal scheduler error: unknown thread state 25 # 26 ready This bears an uncanny resemblance to scheduler issues I've been fighting a long ago. Too long to ago. --- A fix Just allow the 'ready state for threads in mutex-unlock! ... Is this a correct fix? Too long ago. But it feels wrong. We'd rather make sure there is no ready thread in the queue waiting for a mutex in the first place. Diffing the changes I maintained quite a while back http://ball.askemos.org/Ad60e3fb123a79b2e5128915116b288f7/chicken-4.9.1-ball.tar.gz you will find that I added a ##sys#thread-clear-blocking-state! Towards the end of scheduler.scm and used it for consistency whereever I ran into not-so-clean unlocks. Now this is still an invasive change. But looking at the source of scheduler and srfi-18 in chicken 5 right now, I can't fight the feeling that it is working around the missing changes at several places. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH Re: slow polling
Hi Megane, thanks for looking at my submission. Let me try to answer your questions. On Nov 17 2018, megane wrote: Jörg F. Wittenberger writes: Am 19.02.2016 um 22:39 schrieb Jörg F. Wittenberger: ... I opened ticket 1259 for this. To make the kind reviewers job easier, I'll post diffs in piecemeal here. This patch goes after killing a single - but important - comment line in scheduler.scm: ;; This should really use a balanced tree: Now it does. This patch replaces the timeout queue with a balanced tree. It does not matter so much, which kind of tree we use. But a linear list is really a bad choice. Hi Jörg, What kind of performance benefits did you see with this patch? Sorry, it's 10 years since I wrote this code http://lists.nongnu.org/archive/html/chicken-users/2008-10/msg00103.html I'm not sure that my observations from those days still hold. Do you think the performance gains are worth the added complexity to the scheduler? At that time it definitely did. Did you consider binary heap as the priority queue implementation? It can be implemented with a vector and has nicer cache performance than search trees which your patch seems to use. Still I would not go with the patch today anymore: since I learned that the pure version of the llrb tree is considerable faster under CHICKEN (at least since argvectors days; I did not bother to try older versions). (The llrb-tree egg includes a - for that reason now disabled - mutating version of the llrb-tree, which you could try.) Given that experience I'd at least pitch the pure version against alternative algorithms to implement the priority queue. Yes I tried a heap for my initial implementation using poll(2) with chicken. That is at least for the fd-list. (I failed to locate the code right now; should be in this context: http://lists.nongnu.org/archive/html/chicken-hackers/2012-05/msg8.html ) I did play with some heaps for a toy coroutine scheduler and binary heap was faster than binomial heap and pairing heap in simple synthetic tests. Less garbage too. The latter two implementations use separate node structures which add garbage. That's interesting as it contradicts my observation where mutation appears to be worse than garbage. Though the latter was found measuring the very same algorithm (llrb tree) in pure and mutating versions. Best Regards /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Bug in make-parameter / parameterize
Sorry for the noise. May fault. This bug is not there. (Neither in C5 nor 4.13) I obviously did not read the chicken version number I cut&pasted. Let alone that I did not even expect any chicken prior to 4.12 installed on any of my machines. Regards On Oct 25 2018, Peter Bex wrote: On Thu, Oct 25, 2018 at 02:01:08PM +0200, Jörg F. Wittenberger wrote: Hi Chickeneers, learning from srfi-154 it seems there is a bug in make-parameter/parameterize. Here a simplified and shortend implementation of srfi-157 plus a test lifted from the srfi. With chicken (tested with version 4.11.1) it fails (returns #f). (chibi-scheme passes the test returning #t.) Might want to check with a recent version of CHICKEN. We fixed several issues related to parameter handling in 4.12.0. A quick check here with 4.13.0 seems to indicate the bug does not exist anymore. See also #1336 and #1227. Cheers, Peter ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Bug in make-parameter / parameterize
Hi Chickeneers, learning from srfi-154 it seems there is a bug in make-parameter/parameterize. Here a simplified and shortend implementation of srfi-157 plus a test lifted from the srfi. With chicken (tested with version 4.11.1) it fails (returns #f). (chibi-scheme passes the test returning #t.) Best /Jörg (let* ((current-dynamic-extent (lambda () (call-with-current-continuation (lambda (return) (call-with-values (lambda () (call-with-current-continuation (lambda (c) (return (lambda (thunk) (call-with-current-continuation (lambda (k) (c k thunk (lambda (k thunk) (call-with-values thunk k))) (with-dynamic-extent (lambda (dynamic-extent thunk) (dynamic-extent thunk (eq? (let* ((x (make-parameter 'a)) (de (parameterize ((x 'b)) (current-dynamic-extent (parameterize ((x 'c)) (with-dynamic-extent de (lambda () (x) 'b)) ;;; ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Would SRFI-157 (pre)support worth a consideration?
On Oct 24 2018, Peter Bex wrote: On Tue, Oct 23, 2018 at 09:33:02PM +0200, Jörg F. Wittenberger wrote: Hi all, with Chicken 5 being about to be released I know there is no adding features now. That's why I hesitate to post at all. At the other hand I'd guess it might be easier to now add half the feature I'd desire than later deal with a binary incompatible change. Maybe it's worth to consider adding this one more "thread-local mutable cell" in C5 threads now. To make room for srfi-157 support in C5.1 or so. I don't quite see why this should be required as part of 5.0; AFAIK thread local mutable cells would be a completely orthogonal new feature that could be added later. The idea was more like adding a 15th field in ##sys#make-thread in library.scm around line 5691. (Or maybe it would belong into slot #5 "state buffer"?) And it's nontrivial so not something to be adding at the RC stage. As I said, I'm not sure this should be done now. Because of the RC stage. Neither I'm sure this is really going to complicate things. As long as an srfi-157 egg could easily overwrite ##sys#make-thread. Just wanted to make sure it's been considered for a moment by someone more knowledgeable than me how badly it could hit home if I was missing something. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Would SRFI-157 (pre)support worth a consideration?
Hi all, with Chicken 5 being about to be released I know there is no adding features now. That's why I hesitate to post at all. At the other hand I'd guess it might be easier to now add half the feature I'd desire than later deal with a binary incompatible change. The problem: I've been looking into the idea today to port the "forcible" egg to chicken 5. Forcible was intented to do two things: a) fix force/delay wrt. srfi-18 threads (make sure there is only one invocation of the delayed expression) and b) implement "future"s (inherited from rscheme) which is syntactically like delay, but the expression evaluated in an asynchronous thread. ((There is a actually one more, but that's a programming experiment.)) Delayed, I had porting the egg since I knew it had an issue: "future" and "delay" are not equivalent wrt. access to parameters. Exactly what SRFI-155 discusses in the Rationale. The solution: Or at least the hint to the solution I found in https://srfi.schemers.org/srfi-157/srfi-157.html section "Implementation hints" second paragraph The idea: Maybe it's worth to consider adding this one more "thread-local mutable cell" in C5 threads now. To make room for srfi-157 support in C5.1 or so. Best Regards /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Regression in CHICKEN 5
Hi all, commit bb6bf66b4bb6c16fedaf0aea55e965f9497f2885 just undoes what commit 0ae333805f8fc782d4cdc36e1f3675028a26d9fa introduced (argument to csc-options etc. being passed through ->string). Commit dc07113cf79a1930c6a109c738138dbea15afbc0 throws this out completely. Hence egg files can not yet use unqoted arguments are now working. Instead they fail like this: Error: (string->list) bad argument type - not a string: -J Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] conditionals in .egg files
On Sep 28 2018, Peter Bex wrote: On Fri, Sep 28, 2018 at 08:28:21PM +0200, Jörg F. Wittenberger wrote: One more idea: How about an entry indicating the version of the egg file syntax? Having something to cond-expand is nice. But it might be hard to settle with a final version without unduly delaying CHICKEN 5. This leaves us with changing semantics of many eggs over time. With a an entry like (egg-version "1.X") future changes could easier apply backward compatible processing for old eggs. I'm not sure how useful that would be. Usually we'd keep backwards compatibility, which should be easy: only introduce new forms. Which creates bloat and the need to keep backward compatibility. Having a syntax version would make it easy to spot what is deprecated or no longer working. We could essentially grep for eggs needing attention. Without one ends up debugging. First and foremost the idea wanted to pave a way to experiment with syntax which we might eventually NOT want to support anymore. And remove support soon. So chicken-install could evolve faster than once every major chicken release. This scenario would exacerbate the need to locate a few eggs depending on syntax avail in a (short) time range. Coupled with a deprecation phase for forms that disappear, and our slow release process, I'm sure people will have no good excuse to not upgrade :) The problem with old versions of chicken-install would be they don't understand those new forms. But for those I guess you'd just pin the version of the egg to an older one. The old chicken-install is IMHO a no brainer: just complain about new syntax version. At the other hand: if the plan for cond-expand is to actually allow every component of the egg file to be conditioned on features, the idea is mood. But than I wonder, maybe we should enable let-syntax too. ;-) /runs and hides/ Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] conditionals in .egg files
Hi! I don't have to salute, do I? ;-) On Sep 27 2018, felix.winkelm...@bevuta.com wrote: Hi! I wonder: there is no way to conditionally set options, configure egg building, etc. I think it would be handy to have a "cond-expand" form that can be used inside ..egg files customize the egg build/installation depending on platform. This would also bring back the "-feature" option for chicken-install, that Jörg dearly missed, as CSC_OPTIONS can not be used to pass any options directly to the processing of an egg. Opinions? I'd love it. One more idea: How about an entry indicating the version of the egg file syntax? Having something to cond-expand is nice. But it might be hard to settle with a final version without unduly delaying CHICKEN 5. This leaves us with changing semantics of many eggs over time. With a an entry like (egg-version "1.X") future changes could easier apply backward compatible processing for old eggs. /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] functor support
Hi Megane, I understand that a functor in Chicken is more like a template. As you detailed it is possible to use macro expansion tricks instead of the include statement, essentially expanding the include manually. That's roughly what I'd feel chicken should do in the first place. To me as a user who wants to package a functor into an egg, it is surprise that the compiled and installed egg will include rather arbitrary files from the current directory. Together with the line number issue functors look a bit like a second thought and not well supported. In practice I'm going through some effort to avoid functors where I'd actually like to use them. (And still did not come up with a nice recipe how to do so.) Best /Jörg Am Sun, 09 Sep 2018 08:59:59 +0300 schrieb megane : > > Hi Jörg, > > Jörg F. Wittenberger writes: > > > Hi Chickeneers, > > > > attached is a test trying to package the functor example from the > > documentation as an egg. This is no good except to showcase some > > issues. > > > > 1. "include" should be expanded. (I reported this before, now here > > is the simple example.) > > I'm pretty sure this is not the way functors work. The functor body is > not walked at definition time at all. You can check this by putting > (begin-for-syntax (display "here")) in the functor definition, or just > put some invalid code like () or module definition (should cause inner > module warning.) Functors are more like templates. > > If you want to have the functor definition in a different file you can > expand the functor with a macro and splice in the other file's > contents there. > > > > > 2. Line numbers for errors refer to an unexpected location: all > > blame is assigned to the instatiation line. (This is rather > > annoying for me.) > > > > This would indeed be helpful. I have no idea how much work it would be > to implement this. > > Regards, > ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] functor support
Hi Chickeneers, attached is a test trying to package the functor example from the documentation as an egg. This is no good except to showcase some issues. 1. "include" should be expanded. (I reported this before, now here is the simple example.) 2. Line numbers for errors refer to an unexpected location: all blame is assigned to the instatiation line. (This is rather annoying for me.) Following the details Best /Jörg Do not install this egg, just do `chicken-install -n` reproduce the situation. After compilation try: 1.A) Run the test (The error in (*) is expected.) functor$ csi tests/run.scm CHICKEN (c) 2008-2018, The CHICKEN Team (c) 2000-2007, Felix L. Winkelmann Version 5.0.0rc1 (prerelease) (rev 9d480412) linux-unix-gnu-x86-64 [ 64bit dload ptables ] ; loading tests/run.scm ... ; loading ./squaring-functor.import.so ... ; loading ./squaring-functor.so ... ; including functor-code.scm ... Error: (*) bad argument type - not a number: #f 1.B) functor$ pushd tests functor/tests$ csi CHICKEN (c) 2008-2018, The CHICKEN Team (c) 2000-2007, Felix L. Winkelmann Version 5.0.0rc1 (prerelease) (rev 9d480412) linux-unix-gnu-x86-64 [ 64bit dload ptables ] #;1> (load "../squaring-functor.so") ; loading ../squaring-functor.so ... #;2> (load "../squaring-functor.import.scm") ; loading ../squaring-functor.import.scm ... #;3> (module nums (multiply) (import scheme) (define (multiply x y) (* x y))) #;4> (module number-squarer = (squaring-functor nums)) Error: (include) cannot open file: "functor-code.scm" 2.) Open squaring-functor.scm, comment out line 7 and remove the comment from line 8. Again do $ chicken-install -n functor$ csc tests/run.scm functor$ ./tests/run Error: (*) bad argument type - not a number: #f Call history: tests/run.scm:1: chicken.load#load-extension tests/run.scm:3: ##sys#register-compiled-module tests/run.scm:7: ##sys#register-compiled-module tests/run.scm:10: number-squarer#square tests/run.scm:7: nums#multiply tests/run.scm:13: number-squarer#square tests/run.scm:7: nums#multiply Compare to tests/run.scm line 7 is: (module number-squarer = (squaring-functor nums)) functor.tar.gz Description: functor.tar.gz ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] C lost -D on chicken-install
On Aug 21 2018, Peter Bex wrote: On Tue, Aug 21, 2018 at 09:22:47AM +0200, ko...@upyum.com wrote: felix.winkelm...@bevuta.com wrote: > There is no bad intent behind this - during the rewrite simply a few > options were dropped, since they didn't look crucial. If you want it > back, no problem. I've been missing it a little bit, but I agree with some of what has been said and would like to add my 2 cents. Should the features declared via chicken-install be passed to all dependencies being built? I think so. Vasilij's example is quite nice to support this opinion, building hypergiant will bring a lot of other eggs, and every one of them accept a "gles" feature to build for OpenGL ES instead of OpenGL. I ported (not published yet) some of these eggs and I've been missing a way to do different things depending on whether the gles feature is defined or not. I think we should add a cond-expand like mecanism in the egg file format as well, if we add `chicken-install -D` back. While I'm all for improving things, let's keep an eye on feature creep. We are only fixing critical bugs here, not adding new features: those can go into 5.1. Agreed. The question whether -D defined features should be propagated to dependencies easily becomes hairy. As chicken-install is supposed to be used interactively we would at least need to leave the ultimate decision to the user. Plus next thing about the default: should already installed dependencies be re-installed with the new feature installed? Perhaps for now the easiest way is to blindly pass the -D options onto csc. If a build script needs to do something special with this info, I'd suggest using environment variables for now. Alternatively, one could test-compile a program that uses cond-expand and re-exports it back to the build script. It's a bit hacky, but should be enough for these special cases. Using cond-expand in a build script sounds pretty difficult to do right with dependencies, because it may result in build and install scripts that no longer just depend on files for their freshness, but now also differ depending on what features were selected. This gets hairy very fast! If this is still considered important, I'd suggest making a ticket for CHICKEN 5.1 where we can discuss the best approach and weigh our options. My priorities would be: 1. For chicken 5.0 is vote for: a. Minimal: To ease the transition process make chicken-install -D emit a special message suggesting how to proceed. The generic message it costly in terms of users time figuring out why legacy code ceased to work. May take years to phase out old code. b. Better: Simply keep the legacy behavior. Blindly pass -D to csc. 2. Later a. Hash the options with the code into the key to the egg cache such that freshness depends on the features too. (If this is easy enough: propagate.) b. Figure out whether it's worth or not the code complexity to actually force dependencies to re-install depending on features given (and whether or not we should blindly re-install all or rather declare the features an egg is sensitive to under a new keyword in the .egg file and only re-install things when they are declared to recognize the feature -- which in turn would prompt the -D feature to better reject features not declared in the egg -- how was this about feature creep)? c. Mimic along the lines of the nix package manager. https://nixos.org/nix/ -- or maybe we find a something even better. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] C lost -D on chicken-install
I'm not sure this will do the trick. it the environment variable CSC_OPTIONS added to the options given in the .egg file or does it overwrite them? The former would count as a workaround, the latter not help in my case at all. The use case of the -D switch, which I need first and foremost to support is code instrumentation. There are lots of yocto build recipes which compile differently when -Ddebug is given. Peter is right: this makes determining the freshness of files harder. (In the case of yocto the bitbrake program can take care of that one. Chicken could mimic the trick. But that's certainly beyond chicken 5.0) BTW: C5 is easily troubled if there are *.import.scm files from C4 in the same directory. The resulting error messages are horror. They compile but upon import we get something like "eval not avail". Best /Jörg On Aug 21 2018, ko...@upyum.com wrote: Peter Bex wrote: While I'm all for improving things, let's keep an eye on feature creep. We are only fixing critical bugs here, not adding new features: those can go into 5.1. Then we can drop that issue altogether until 5.1. One can pass the desired features with CSC_OPTIONS already. CSC_OPTIONS="-D foo" chicken-install the-egg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] C lost -D on chicken-install
Oh, yes, please. The -D is crucial here. Nevertheless for whatever was dropped: it might be better to at least recognize and reject them with a specific message. Otherwise whatever build scripts will suddenly break with a rather generic message. This would be code which did work for quite a while. Might be hard to figure out why it broken. /Jörg On Aug 20 2018, felix.winkelm...@bevuta.com wrote: Hi all, apparently chicken-install lost the -D switch. To put it mildly: This worries me a lot. There is no bad intent behind this - during the rewrite simply a few options were dropped, since they didn't look crucial. If you want it back, no problem. felix ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] type declarations for define-record not working
Hi all, attached foo.scm. try (for C4 and C5): csc -s foo.scm C4 should do, C5 complains for me: Warning: at toplevel: assignment of value of type `(procedure foo#make-foo (*) (struct foo#foo))' to toplevel variable `foo#make-foo' does not match declared type `(procedure foo#make-foo (string) (struct foo))' Best /Jörg (module foo * (import scheme) (cond-expand (chicken-5 (import (chicken base) (chicken type) )) (else (import chicken))) (: make-foo (string --> (struct foo))) (define-record foo bar) ) ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] C lost -D on chicken-install
Hi all, apparently chicken-install lost the -D switch. To put it mildly: This worries me a lot. Now everyone will be looking for the best way how to bring this back. Or at least learn the answer the big question: Why the hell this is considered a bad idea and how to bring the functionality back anyway once you know what you are doing. To ease the transition maybe chicken-install should not to simply give the default help for the now unknown -D switch but a specific message telling the answer to that big question. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] csc-options in .eg file
It would be so practical if I could write (csc-options -O3) i.e., if the "arguments" to csc-options (and likewise link-options) would be implicit converted into strings. Any reasons this is not a good idea (TM). Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Feature request: make cond more resilient against misuse
Hi all, admitt this is a case of garbage-in-garbage out. A `cond` (or `case`) expression with an `else` branch behaves odd when an empty sequence of expression is given in the `else` branch: it results in an unbound variable "else" at runtime. Attached a test case. Try $ csc unboundvariableelse.scm $ ./unboundvariableelse Best /Jörg (module foo * (import scheme) (define (bar x) (cond ;; same for `case` here (else #;unbound))) ) (import foo) (display (bar #t)) ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Eggs for Chicken 5
Am Thu, 22 Mar 2018 12:22:03 -0400 schrieb John Cowan : > On Thu, Mar 22, 2018 at 12:11 PM, Sven Hartrumpf > wrote: > > I use a Chicken-compiled alexpander to get rid of syntax-rules when > > compiling > > some Scheme projects (with non-Chicken compilers). > > > > I'm interested in this. Which compilers do you use that don't have > syntax-rules? > I know that Stalin does not. I used the same approach to keep the code running on RScheme when I ported the Askemos/BALL code to Chicken. The interested can find it here: https://gitlab.com/ball-zero/ball /Jörg A shameless advertising plug: According to my understanding Askemos, being released in 2001, qualifies for the first implementation of a smart contract system. It is just slightly different from todays buzzword SC things: The virtual machine is heavily inspired by LISP and strives for a minimal "CoreAPI" of a handful of operators. Hence no room for a build-in virtual currency; those need to be done at application level. As application programming language we provide a pure subset of Scheme with embedded SQLite (one db per object). Inspired by Erlang we support communicating processes each having their own "blockchain", which sort of mitigates performance problems. We use byzantine fault tolerant consensus among parties and selected notaries instead of proof-of-X in random selection of third parties. ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Chicken 4.13: New bug in syntax-rules
On Feb 24 2018, Peter Bex wrote: On Mon, Jan 15, 2018 at 10:11:05PM +0100, Peter Bex wrote: On Sat, Jan 13, 2018 at 03:28:09PM +0100, Jörg F. Wittenberger wrote: > Hi all, > > I just boiled down an issue with syntax-rules processing new in chicken > 4.13. > > It is a bit involved. I needed three modules to demonstrate it. Good find! I've created ticket #1441 to track this. It also contains a highly simplified version of this situation which indeed also breaks in CHICKEN 4.13 but not 4.12. Hi again, I did some more research into this bug, and I'm less and less convinced it's really a bug. See my findings in the ticket. If it's not a bug, then I'd need another word to excuse my embarrassment. After all such code used to work for several years (since define-macros went away) until 4.13 came along, broke it and the bug reports I got where: "your code does not even compile!!1!". I'm not sure that this is "undefined behavior" in the spec. Even though some other Schemes have implementation issues too. Your findings in the ticket seem to indicate that at least Chibi does the right thing, as did 4.12. We may be able to make it work, but I don't see how just yet. Neither see I. However if not, we should at least document under known limitations that there are issues wrt. hygienic macro expansion. (After all it is not nice to send users spending a couple of days to hunt down situations, be them called a bug or a not-bug, where importing unrelated modules breaks things.) Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Chicken 4.13: New bug in syntax-rules
Sorry for the need to follow up: The file attached to my report is not a proper tar.gz: somehow I managed to run gzip twice on the tar. Expanding my comments I'd like to point out that the issue is rather confusing violation of hygiene: The symbol `current-place` is used (internally) in the macro defined and exported from the first module (fume.scm) and used in the third (barm.scm). It does not matter whether or not an unrelated (and unused) binding for `current-place` is established in the `barm` module. However if the unused binding is imported into `barm` from the second module, than things change. The syntax-rules macro from the fist module will use this foreign binding to shadow the internal binding. Now really hygienic anymore. Cheers /Jörg On Jan 13 2018, Jörg F. Wittenberger wrote: Hi all, I just boiled down an issue with syntax-rules processing new in chicken 4.13. It is a bit involved. I needed three modules to demonstrate it. Attached a tarball. $ make test should succeed with Chicken 4.12 but fail to compile with Chicken 4.13 due to $$ unbound. Now find barm.scm and change two things: Comment out the line "(import baz)" near the top and remove the comment to make the "(define current-place ..." effective. At this point "make test" should succeed with Chicken 4.13 too. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Chicken 4.13: New bug in syntax-rules
Hi all, I just boiled down an issue with syntax-rules processing new in chicken 4.13. It is a bit involved. I needed three modules to demonstrate it. Attached a tarball. $ make test should succeed with Chicken 4.12 but fail to compile with Chicken 4.13 due to $$ unbound. Now find barm.scm and change two things: Comment out the line "(import baz)" near the top and remove the comment to make the "(define current-place ..." effective. At this point "make test" should succeed with Chicken 4.13 too. Best /Jörg syn4-13.tar.gz Description: syn4-13.tar.gz ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] c5 install-cache
Hi all, my gut feeling tell me that the path beeing used in ~/.chicken-install.cache ought to have at least a binary compatibility segment. Otherwise I'd forcast dragons ahead the road. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] C5 usage problem: chanr-name is gone
last time I tried C5 it ate this: I'm using -extend to feed these lines globaly into my build: (char-name 'NUL (integer->char 0)) (char-name 'nul (integer->char 0)) (char-name 'cr (integer->char 13)) Now it complains: Error: unbound variable: char-name How should I best fix this? Thanks /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Bugs in C5 eggs with C5 pre4
Just tried. SRFI-13 chicken-install srfi-13 (after installing development snapshot c5 pre4) fails with Warning: reference to possibly unbound identifier `use' Fix was easy: find ~/.chicken-install.cache/srfi-13/srfi-13.scm and change the "(use srfi-14)" into "(import srfi-14)". SRFI-69 fails with Error: during expansion of ($flonum-hash ...) - unbound variable: fx- Fix is beyond me. /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] What to do with CHICKEN 4.13/4.12.1 release/snapshot?
On Nov 6 2017, Kon Lovett wrote: CHICKEN 4 LTS create a "Foundation for the Continuation" to support all NEKCIHCs ...for a to-be-defined length of "long term"; sign me up. Porting hundreds of modules and thousands LOC may or may not reveals issues. Once that's done only, we can see how we'd like to proceed. On Nov 5, 2017, at 3:09 PM, Evan Hanson wrote: Hi all, I think we should: 0.) Fix 1399 in master and chicken-5. 1.) Tag 4.13.0rc1. 1.1, ...) Iterate until we have a good RC, release as 4.13.0. 2.) Rename master to chicken-4. 3.) Rename chicken-5 to master. From that point on we develop on master while chicken-4 gets security patches only. We continue to support 4.13.X with security fixes until 5.0.0 is released + N days, however long we'd like to offer users time to upgrade. One year seems like a reasonable length of time to me. Cheers, Evan ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] [5] Avoid storing threads in trace-buffer
On Nov 4 2017, Christian Kellermann wrote: Given that I presumed to compile with -no-trace and focus on ressource conservation: No. I feel, I have been warned. No, I do not see a point in runtime cost for debug features I could previously disable completely. * J?rg F. Wittenberger [171104 20:07]: Just wondering. If memory usage was the concern, why not simple compile without tracing? This way we pay one more slot per thread. Plus some lookup. What do we earn? Maybe I'm missing something? You will miss helpful error messages? ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] [5] Avoid storing threads in trace-buffer
Just wondering. If memory usage was the concern, why not simple compile without tracing? This way we pay one more slot per thread. Plus some lookup. What do we earn? Maybe I'm missing something? Joerg On Sat, Oct 28, 2017 at 09:01:15PM +0200, felix.winkelm...@bevuta.com wrote: This is intended to fix #1356: threads may be retained too long in the trace buffer, because some sort of method is required to extract the call chain for a specific thread (multiple threads can create entries in the trace buffer and thus must be distinguished). Now a thread has an additional slot with a unique ID, which is stored in the trace buffer instead of the thread itself. Peter suggested a gensym, but this patch uses a pair, which is (slightly) cheaper and doesn't bump the gensym counter. Looks good to me, pushed! Cheers, Peter ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Make the compiler output deterministic
Am Sat, 21 Oct 2017 17:12:33 +0200 schrieb Kooda : > Here is the same set of patches, for master, in case we want that. Congratulations Kooda! I have not looked at any of the patches. But I feel a strong desire to want the result! %% lifting my hat /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Add unexport form for modules
I agree that an explicit export list is beneficial for polished code. However I do often use modules for *import* control, a.k.a. catching my typos, rather than export control. That's when the *-export is very useful. /Jörg On Jul 4 2017, megane wrote: John Cowan writes: On Tue, Jul 4, 2017 at 10:11 AM, megane wrote: Working with the export list is painful without some syntax support. You end up repeating every identifier name otherwise. I consider the repetition to be a virtue rather than a vice. The whole purpose of modules is to control what identifiers are exposed to the outside world and what identifiers are private. If you want to expose everything, why bother with a module at all? Just have a plain file that is included rather than imported. I, too, want to precisely control what a module exports and what it doesn't. I think keeping the * export would be nice for beginners coming from languages where export control is less manual work. I myself don't care anymore. ... ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Add unexport form for modules
Am Thu, 8 Jun 2017 13:29:48 -0400 schrieb John Cowan : > As we discussed on IRC, I see unexport as mitigating the problems with > (export *), and it's better to drop exporting * instead. > "Whitelisting is better > than blacklisting." Sure whitelisting better. But rather tedious during (early) development. Maybe this could be neat: (export (* (except (prefix NOTTHESE-) not-this-one nor-that-either))) ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] PS: chicken 5 static linking issue
Just notices this could be a hint: (import-syntax (except scheme inexact->exact) (except chicken add1 sub1 with-exception-handler) (onl... <-- This matches the imports in the module: (import (except scheme inexact->exact) (except chicken add1 sub1 with-exception-handler) (only srfi-34 raise)) Am Sat, 1 Apr 2017 21:38:13 +0200 schrieb "Jörg F. Wittenberger" : > Hi all, > > another half baked issue I encountered when giving chicken5 a try. > > My supposed to be statically linked executable does not work as > expected. After ~40 counts of "[debug] entering X..." I get: > > > [debug] entering v-clformat... > ; loading /home/u/test-5/lib/chicken/8/chicken.import.so ... > Error: (load) during expansion of (import-syntax ...) - unable to load > compiled module - cannot load compiled code dynamically - this is a > statically linked executable: > "/home/u/test-5/lib/chicken/8/chicken.import.so" > > > Obviously it should not even try. > > > > I wonder how I should track this down. > > Best > > /Jörg > > ___ > Chicken-hackers mailing list > Chicken-hackers@nongnu.org > https://lists.nongnu.org/mailman/listinfo/chicken-hackers ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] chicken 5 static linking issue
Hi all, another half baked issue I encountered when giving chicken5 a try. My supposed to be statically linked executable does not work as expected. After ~40 counts of "[debug] entering X..." I get: [debug] entering v-clformat... ; loading /home/u/test-5/lib/chicken/8/chicken.import.so ... Error: (load) during expansion of (import-syntax ...) - unable to load compiled module - cannot load compiled code dynamically - this is a statically linked executable: "/home/u/test-5/lib/chicken/8/chicken.import.so" Obviously it should not even try. I wonder how I should track this down. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Issue in -compile-syntax Was: Howto -extend chicken-5?
Hi all, turns out the culprit was the use of -compile-syntax $ csc -c -compile-syntax test-compile-syntax.scm Warning: reference to possibly unbound identifier `barize' Error: module unresolved: foo I'm still changing import clauses all over the place. It's going to take time until I will see how much harm removing the -compile-syntax did. Test attached. /Jörg Am Sun, 26 Mar 2017 21:04:25 +0200 schrieb Peter Bex : > On Sun, Mar 26, 2017 at 03:03:02PM +0200, Jörg F. Wittenberger wrote: > > (include "../mechanism/notation/Lalr/lalr-gen.scm");;[1] > > > > (define-syntax lalr-parser > >(er-macro-transformer > > (lambda (x r c) > > (apply gen-lalr-parser (cdr x) > > > [...] > > (module mystuff * > > (import lalrgen) (import-for-syntax lalrgen) > > ... > > > > Error: during expansion of (lalr-parser9196 ...) - unbound variable: > > gen-lalr-parser > > > > How should this work? > > Try > > (begin-for-syntax (include "../mechanism/notation/Lalr/lalr-gen.scm")) > (define-syntax lalr-parser ...) > > > [1]: This lalr-gen.scm is a big cond-expand on the scheme > > implementation defining among other things a syntax `logical-or` as > > alias to logical-ior and uses these definitions in the procedure > > gen-lalr-parser, which does the macro actual expansion. The > > implementation of the latter contains the reference to the renamed > > logical-ior. In other words: it's missing at expansion time. > > This was the crucial insight that was missing from your previous post. > > Cheers, > Peter ;; $ csc -c test-compile-syntax.scm ;; $ csc -c -compile-syntax test-compile-syntax.scm ;; ;; Warning: reference to possibly unbound identifier `barize' ;; ;; Error: module unresolved: foo (module foo (bar) (import chicken scheme) (begin-for-syntax (define (barize . x) 42)) (define-syntax bar (er-macro-transformer (lambda (x r c) (apply barize (cdr x) ) (module foobar * (import chicken scheme foo) ;(import hae?) (define barefoot (bar 23))) (import foobar) (use chicken.format) (format #t "Seen ~a\n" barefoot) ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Howto -extend chicken-5?
Almost! Am Sun, 26 Mar 2017 21:04:25 +0200 schrieb Peter Bex : > On Sun, Mar 26, 2017 at 03:03:02PM +0200, Jörg F. Wittenberger wrote: > > (include "../mechanism/notation/Lalr/lalr-gen.scm");;[1] > > > > (define-syntax lalr-parser > >(er-macro-transformer > > (lambda (x r c) > > (apply gen-lalr-parser (cdr x) > > > [...] > > (module mystuff * > > (import lalrgen) (import-for-syntax lalrgen) > > ... > > > > Error: during expansion of (lalr-parser9196 ...) - unbound variable: > > gen-lalr-parser > > > > How should this work? > > Try > > (begin-for-syntax (include "../mechanism/notation/Lalr/lalr-gen.scm")) > (define-syntax lalr-parser ...) This did not work. gen-lalr-parser is undefined. A slight variation did at least compile (tough it's quite unintuitive to write): (define-syntax lalr-parser (er-macro-transformer (begin-for-syntax (include"../mechanism/notation/Lalr/lalr-gen.scm") (lambda (x r c) (apply gen-lalr-parser (cdr x)) But fails with "call on non-procedure: #" in application. The variation to pull begin-for-syntax around the er-macro-transformer changes this (as expected) into an immediate complaint "expected syntax-transformer, but got: #". So it's probably the best thing to make begin-for-syntax to define the containing syntax at toplevel in such a way that your suggested way would work. Cheers /Jörg > > [1]: This lalr-gen.scm is a big cond-expand on the scheme > > implementation defining among other things a syntax `logical-or` as > > alias to logical-ior and uses these definitions in the procedure > > gen-lalr-parser, which does the macro actual expansion. The > > implementation of the latter contains the reference to the renamed > > logical-ior. In other words: it's missing at expansion time. > > This was the crucial insight that was missing from your previous post. > > Cheers, > Peter ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Howto -extend chicken-5?
Am Sat, 25 Mar 2017 21:13:10 +0100 schrieb Peter Bex : > On Sat, Mar 25, 2017 at 09:07:21PM +0100, Jörg F. Wittenberger wrote: > > Am I doing something wrong, which should not have been working in > > chicken4 either? How should I fix this? > > You probably need to add (import (chicken bitwise)). Nah, thanks for the suggestion, but this is not enough. Neither importing it into the module exporting the macro nor importing chicken.bitwise in the module before using the macro helps. Maybe it's chicken.bitwise, which may(ormaynot-dunno) be not effective for code loaded using -extend?[1] But maybe this is not worth to be considered. I did not like the use of -extend anyway. It was just the only way I found years ago. Actually I'd like to get rid of that and have a normal module. Removed the -extend option from the compile command and... somehow I'm to stupid to get that done: (module lalrgen ( (lalr-parser gen-lalr-parser) gen-lalr-parser ) (import scheme chicken) (import chicken.pretty-print chicken.bitwise) (include "../mechanism/notation/Lalr/lalr-gen.scm");;[1] (define-syntax lalr-parser (er-macro-transformer (lambda (x r c) (apply gen-lalr-parser (cdr x) ) (module mystuff * (import lalrgen) (import-for-syntax lalrgen) ... Error: during expansion of (lalr-parser9196 ...) - unbound variable: gen-lalr-parser How should this work? Thanks so much. /Jörg [1]: This lalr-gen.scm is a big cond-expand on the scheme implementation defining among other things a syntax `logical-or` as alias to logical-ior and uses these definitions in the procedure gen-lalr-parser, which does the macro actual expansion. The implementation of the latter contains the reference to the renamed logical-ior. In other words: it's missing at expansion time. ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Howto -extend chicken-5?
Hi, I'm trying to compile code which uses Dominique Boucher's lalr parser generator in chicken-5. This does not work the way it did in chicken-4. I compile like this $ chicken place-common.scm -output-file place-common.c -setup-mode -include-path . -verbose -extend ../mechanism/notation/Lalr/lalr-gen.scm i.e., I use -extend to inject the parser generator into the macro expander. (The file[1] lalr-gen.scm is just Dominique's parser generator code, not the runtime/driver part.) AND I do have a module[2], which is supposed to provide the same service at runtime. "place-common" imports the module: Error: during expansion of (lalr-parser9165 ...) - unbound variable: bitwise-ior Call history: [initialize-F] (vector-ref shift-table stateno) [initialize-F] ((##core#letrec* ((loop2 (##core#loop-lambda (j edges) (if (pair? j) (let ((symbol (vector-ref acces... [initialize-F] (shift-shifts sp) [shift-shifts] (vector-ref s 2) [initialize-F] (pair? j) [initialize-F] (vector-ref acces-symbol (car j)) [initialize-F] (car j) [initialize-F] (< symbol nvars) [initialize-F] (set-bit rowf (- symbol nvars)) [initialize-F] (- symbol nvars) [set-bit] (quotient b (BITS-PER-WORD)) [set-bit] (inexact->exact (expt 2 (remainder b (BITS-PER-WORD [set-bit] (expt 2 (remainder b (BITS-PER-WORD))) [set-bit] (remainder b (BITS-PER-WORD)) [set-bit] (vector-set! v x (logical-or (vector-ref v x) y)) [set-bit] (bitwise-ior597 (vector-ref v x) y) <-- Looks to me as if the define-syntax in [1] gets some renaming it should not: (define-syntax logical-or (syntax-rules () ((logical-or x ...) (bitwise-ior x ... Am I doing something wrong, which should not have been working in chicken4 either? How should I fix this? Thanks a lot /Jörg [1] https://gitlab.com/ball-zero/ball/blob/chicken_argvector/ball/mechanism/notation/Lalr/lalr-gen.scm [2] https://gitlab.com/ball-zero/ball/blob/chicken_argvector/ball/chicken/lalrgen.scm ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] strange warning
The latter; experimenting with chicken 5. Using import as in chicken 4. Am Sat, 25 Mar 2017 20:22:41 +0100 schrieb felix.winkelm...@bevuta.com: > > Hi, > > > > I noticed a warning confusing me: > > > > "Warning: the following extensions are not currently installed:" ... > > > > Actually I'm not installing any extensions. I compile and link them > > into a library of it's own. Currently 108 of them. Will I and up > > with a list of all 108 components when compiling the toplevel? > > > > That would be a bit miss-leading a warning then. > > I assume you mean the warning in batch-driver.scm:609 (or around that > line) ? > > Yes, the warning is somewhat misleading if you load dynamic libraries > from private locations (or the current directory) via > "require[-extension]". Or are you using "import" on CHICKEN 5? Then > it makes even less sense... > > I think we can remove it, unless others disagree. > > > felix > ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] strange warning
Hi, I noticed a warning confusing me: "Warning: the following extensions are not currently installed:" ... Actually I'm not installing any extensions. I compile and link them into a library of it's own. Currently 108 of them. Will I and up with a list of all 108 components when compiling the toplevel? That would be a bit miss-leading a warning then. /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] [PATCH] chicken-5-eggs: srfi-69 shoulld export hash-table-walk
attached a diff adding the missing export --- srfi-69.scm~ 2017-03-11 06:49:03.0 +0100 +++ srfi-69.scm 2017-03-25 18:48:02.038211597 +0100 @@ -84,6 +84,7 @@ hash-table-fold hash-table-for-each hash-table-map + hash-table-walk hash-by-identity) (import (scheme) ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] [PATCH] export "sort!" from data-strucutures
Hi all, just missed it. Applicable only to chicken-5. /Jörg >From ae9eec65f4ccc5d7161503405cb1c865da759452 Mon Sep 17 00:00:00 2001 From: jfw Date: Sat, 25 Mar 2017 14:26:04 +0100 Subject: [PATCH] export "sort!" from data-structures --- data-structures.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-structures.scm b/data-structures.scm index 73ad4ec..0a6ea24 100644 --- a/data-structures.scm +++ b/data-structures.scm @@ -31,7 +31,7 @@ (module chicken.data-structures (alist-ref alist-update alist-update! atom? butlast chop compress flatten intersperse join rassoc tail? - merge sort sorted? topological-sort + merge sort sort! sorted? topological-sort conc ->string string-chop string-chomp string-compare3 string-compare3-ci reverse-string-append -- 2.7.4 ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] arm64 problem?
Hi, compling on a fresh rpi3 devuan I ran into a message which may or may not explain why I'm seeing issues. /Jörg clang -Wall -Wno-unused -Wno-invalid-noreturn -fwrapv -fomit-frame-pointer -fno-strict-aliasing -Os -O3 -g -c comparators.c comparators.c:6504:4: warning: absolute value function 'abs' given an argument of type 'long' but has parameter of type 'int' which may cause truncation of value [-Wabsolute-value] t5=C_fixnum_abs(t4); /usr/include/chicken.h:1187:47: note: expanded from macro 'C_fixnum_abs' #define C_fixnum_abs(n) C_fix(abs(C_unfix(n))) ^ /usr/include/chicken.h:1039:56: note: expanded from macro 'C_fix' #define C_fix(n) ((C_word)((C_uword)(n) << C_FIXNUM_SH... ^ comparators.c:6504:4: note: use function 'labs' instead ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Handle possible EINTR in file-lock, file-lock/blocking and, file-unlock.
Am 05.02.2017 um 21:07 schrieb Peter Bex: > On Wed, Feb 01, 2017 at 07:50:30PM +0100, Jörg F. Wittenberger wrote: >> Hi all, >> >> I just found a couple of procedures in the posix unit, which did not >> handle EINTR well. > > The patch looks good to me. I must say it's not completely clear from > the manual what the intended semantics of the nonblocking file-lock > procedure are. I guess if another process has the file locked, we > simply error out with "cannot lock file"? This was at least the behavior the code did before. Actually I'd rather love to see this changed into returning #f instead of raising an error. Reading the documentation, which is not clear at this point, I did actually expect (as in "guess") this to be the case. But that would be an incompatible change wrt. the current behavior. Hence I did not mix this with the bugfix. For compatibility with the currently implemented behavior I'd suggest to change file-lock to accept one more optional argument: a procedure to be called when we currently error out. > If that behavious is inteneded, please apply the signed-off version > attached. > > Cheers, > Peter ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] [PATCH] Handle possible EINTR in file-lock, file-lock/blocking and, file-unlock.
Hi all, I just found a couple of procedures in the posix unit, which did not handle EINTR well. Best /Jörg >From 1d325e122f6adc2d02f170639ada657950c6038c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Wed, 1 Feb 2017 19:46:38 +0100 Subject: [PATCH] Handle possible EINTR in file-lock, file-lock/blocking and file-unlock. --- posixunix.scm | 26 +- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/posixunix.scm b/posixunix.scm index df9a89b..1e24910 100644 --- a/posixunix.scm +++ b/posixunix.scm @@ -1335,16 +1335,22 @@ EOF (posix-error #:file-error loc msg (##sys#slot lock 1) (##sys#slot lock 2) (##sys#slot lock 3)) ) (set! file-lock (lambda (port . args) - (let ([lock (setup port args 'file-lock)]) -(if (fx< (##core#inline "C_flock_lock" port) 0) -(err "cannot lock file" lock 'file-lock) -lock) ) ) ) + (let loop () + (let ((lock (setup port args 'file-lock))) + (if (fx< (##core#inline "C_flock_lock" port) 0) + (select _errno + ((_eintr) (##sys#dispatch-interrupt loop)) + (else (err "cannot lock file" lock 'file-lock))) + lock) )) ) ) (set! file-lock/blocking (lambda (port . args) - (let ([lock (setup port args 'file-lock/blocking)]) -(if (fx< (##core#inline "C_flock_lockw" port) 0) -(err "cannot lock file" lock 'file-lock/blocking) -lock) ) ) ) + (let loop () + (let ((lock (setup port args 'file-lock/blocking))) + (if (fx< (##core#inline "C_flock_lockw" port) 0) + (select _errno + ((_eintr) (##sys#dispatch-interrupt loop)) + (else (err "cannot lock file" lock 'file-lock/blocking))) + lock) )) ) ) (set! file-test-lock (lambda (port . args) (let ([lock (setup port args 'file-test-lock)]) @@ -1356,7 +1362,9 @@ EOF (##sys#check-structure lock 'lock 'file-unlock) (##core#inline "C_flock_setup" _f_unlck (##sys#slot lock 2) (##sys#slot lock 3)) (when (fx< (##core#inline "C_flock_lock" (##sys#slot lock 1)) 0) - (posix-error #:file-error 'file-unlock "cannot unlock file" lock) ) ) ) + (select _errno + ((_eintr) (##sys#dispatch-interrupt (lambda () (file-unlock lock + (else (posix-error #:file-error 'file-unlock "cannot unlock file" lock ) ) ;;; FIFOs: -- 2.6.2 ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] [CHICKEN 5] Change numerics representations
Am 04.10.2016 um 16:19 schrieb felix.winkelm...@bevuta.com: >> On Tue, Oct 4, 2016 at 4:00 AM, wrote: >> We could still get rid of the tagged pointer type. After some more >>> >> >> thought on the matter, I believe they're mostly worthless. >>> >>> "They" means tagged pointers? I agree. >>> >> >> I continue to disagree. Tagged pointers make it possible to have >> dynamically type-safe or type-dispatched treatment of C objects referred to >> by the pointer side. Logically it is equivalent to encapsulate the raw >> pointer in a record, but record dereference is much slower than pointer >> dereference for whatever reason, so it adds substantial overhead. > > "Substantial" may be a bit strong, though the overhead is indeed there. > AFAIK, tagged pointers are seldom used. If indeed they were used instead > of raw pointers everywhere in the FFI (say, by generally tagging each > pointer of a known type), then the situation would be different, of course. > > I don't feel strong enough to assess this. Any takers? At least some eggs would break. However, so far I found only iup among those I'm using. /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Reschedule when a thread blocked on thread-join! is forcibly resumed
Am 02.06.2016 um 13:28 schrieb Evan Hanson: > On 2016-06-02 11:32, Jörg F. Wittenberger wrote: >> No need to actually try. You patch does exactly what my first >> workaround was when I discovered the issue. > > Excellent, in that case we can re-hide `##sys#force-primordial` if this > is applied. Could we delay that hiding for a while please. I just started testing a release of our code; eventually no longer including a custom Chicken. It would be my nightmares coming true if I'd fall back into the situation that production code was incompatible to chicken master. >> However it looks to me as curing the symptom rather then cause. >> >> Convince me otherwise: IMHO forcing the primordial to execute is the >> actual bug. Once we don't do this anymore, we can go over all the code >> and remove the code dealing with the consequences. > > Yes, it fixes a bug without making other changes to the scheduler. The > current behaviour is obviously incorrect and the patch addresses the > obvious incorrectness, that's all. > > AFAICT from past discussions, whether forcing is good, bad, or just a > necessary evil is still unclear, and this needs some agreement before we > can simply rip out that code. I'm open to removing it in chicken-5, at > least, which is where something like a dedicated interrupt handling > thread could go should one materialize. If anyone has any thoughts about > this, please speak up. Aside: it makes it easier to experiment with alternative implementations if the hook is accessible. Cheery /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Reschedule when a thread blocked on thread-join! is forcibly resumed
On Jun 2 2016, felix.winkelm...@bevuta.com wrote: Am 02.06.2016 um 14:18 schrieb felix.winkelm...@bevuta.com: >> AFAICT from past discussions, whether forcing is good, bad, or just >> a necessary evil is still unclear, and this needs some agreement >> before we can simply rip out that code. I'm open to removing it in >> chicken-5, at least, which is where something like a dedicated >> interrupt handling thread could go should one materialize. If anyone >> has any thoughts about this, please speak up. > > Threads are and should be optional. > > > ferlix Sure, but how does this relate to the question of forcing the primordial? It would make a dedicated interrupt-handler thread impossible. Why? ##sys#force-primordial would become ##sys#force-interrupt-handler-thread or alike. Still local to the scheduler and thus optional. The only difference would be so far that the thread it unleashes would be the signal handler thread (which we could completely control from scheduler.scm) instead of the primordial (which is under user control). Without threads nothing would change. felix ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Reschedule when a thread blocked on thread-join! is forcibly resumed
Am 02.06.2016 um 14:18 schrieb felix.winkelm...@bevuta.com: >> AFAICT from past discussions, whether forcing is good, bad, or just a >> necessary evil is still unclear, and this needs some agreement before we >> can simply rip out that code. I'm open to removing it in chicken-5, at >> least, which is where something like a dedicated interrupt handling >> thread could go should one materialize. If anyone has any thoughts about >> this, please speak up. > > Threads are and should be optional. > > > ferlix Sure, but how does this relate to the question of forcing the primordial? /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] [PATCH] Reschedule when a thread blocked on thread-join! is forcibly resumed
Hi Evan, Am 02.06.2016 um 08:41 schrieb Evan Hanson: > Hi folks, > > After some digging I've found a small bug in our srfi-18 implementation > that can have quite sinister effects on programs unlucky enough to be > affected by it. Here's the commit message: > >> When a thread that's waiting on another with `thread-join!` is forced >> to execute -- usually on an interrupt, when the primordial thread is >> forcibly resumed via `##sys#force-primordial` -- it must return ... > Additionally, Jörg, I'd appreciate it if you could remove your > modifications to `##sys#force-primordial` and give this patch a try when > you have a chance. I think there's a good chance that the errors you > described in <56c0c8e0.8020...@softeyes.net> were caused by this issue. No need to actually try. You patch does exactly what my first workaround was when I discovered the issue. However it looks to me as curing the symptom rather then cause. Convince me otherwise: IMHO forcing the primordial to execute is the actual bug. Once we don't do this anymore, we can go over all the code and remove the code dealing with the consequences. The first alternative coming to my mind would be to have a dedicated interrupt handling thread, which we could unleash instead of the primordial. In a next step we could make this thread do all the interrupt dispatching, which in turn would allow us to remove all those ##sys#dispatch-interrupt code abundant in the i/o code. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH Re: slow polling
I can only repeat: the all-in-one change did not fly neither will the piecemeal approach. We need to come up with a small API atop of which users can switch scheduler implementations. Am 20.05.2016 um 21:26 schrieb Peter Bex: > On Fri, Feb 19, 2016 at 10:39:23PM +0100, Jörg F. Wittenberger wrote: >> A "Betthupferl" is Bavarian (a German dialect spoken in some remote, >> hilly areas) for the last, small mean given to the kids upon bedtime. >> Contradictory to all teachings often a sweet. >> >> This patch is not supposed to do any harm. >> >> It refactors parts of the code to minimize the upcoming diffs. >> Furthermore it basically takes srfi-18 out of the equation. Only >> whitespace/comment diffs left there. > > I had a look at this patch, but it does too much and too little at the > same time: > > - The moving of unblock-threads-for-timeout is a good change, but it >should be a separate commit. > - You define fdset-clear as a no-op, so unless I'm missing something, it >will never ever remove the underlying file descriptors from the sets, >even if the fd is closed. This is a recipe for disaster. Can't say much about that anymore. This code was written about 8 years ago. At that time it did work. And it did so under a load which does apparently stress chicken's i/o more than any other. But hey, this was tested on Linux only, ages ago. Sure it's no proof. > - Even if fdset-clear wasn't a no-op, this commit would be incompatible >with select(), which is the reason we tear down and recreate the fdset >from fd-list every time. I am aware it's not very efficient (and I'd >even admit to it being "brain dead" as you call it), but the way >select() works is that it will mutate the input set in-place, so you >can't rely on it maintaining a consistent state. POSIX poll() is >better designed in almost every way imagineable, but we do have to >deal with the shit show that is Windows. To keep things sane we do >it in one consistent way for both select() and poll(), instead of >having two code paths to maintain in code that is already very hairy. > > I'll have to think about this, maybe we can come up with a way to avoid > paying the price on UNIX for the shit Windows implementation. Better > would be to have a *proper* implementation on Windows, but nobody has > stepped up to actually overhaul the entire code to make that possible, > so far. > > Cheers, > Peter > ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] force-primordial - dunno what to do
Am 18.05.2016 um 20:36 schrieb Peter Bex: > On Mon, Feb 15, 2016 at 12:36:59PM +0100, Jörg F. Wittenberger wrote: >> in case nobody finds why we ##sys#force-primordial find a patch attached >> which simply removes it. > > Hello Joerg, > > I've finally had some time and tested with your patch and without, and > found that with the patch, the REPL starts behaving a bit strange. > > For example: > > $ csi > #;1> (use srfi-18) > #;2> (thread-start! (lambda () (let lp () (thread-sleep! 1) (print "hai") > (lp > > This will start printing "hai". When you press ^C, in the current > implementation, you'll immediately see *** user interrupt *** and get > a new prompt. In the version without force-primordial, it will > sometimes do that, and other times it will just sit there, printing, > until you press "enter". On Windows I guess? This looks as if I read that problem before somehow related to i/o. Nevertheless it would need fixing. > So I would say removing force-primordial altogether is not desirable. If there was any kind of explanation/documentation what the purpose of this is, than I'd read that up and shut up. Short of this, I'd rather desire removal for the sake of simplicity itself. More importantly we appear to have a conflict in the code here. You presented a case where the call is apparently required. I found one where it is definitely wrong. Let's turn to the concept. Looks like it is impossible to fix the implementation. > I'm not sure about removing it just in the ##sys#unblock-threads-for-i/o. > So unless someone can *unequivocally* show that this is not needed (or > even causes problems), I vote that we keep it the way it is. > > Cheers, > Peter > ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH: more efficient scheduler
Am 18.05.2016 um 21:17 schrieb Peter Bex: > On Mon, Feb 15, 2016 at 01:57:52PM +0100, Jörg F. Wittenberger wrote: >> Hi, >> >> the attached patch does not fix any bugs. It brings some performance >> improvements under certain load conditions. The cost: It adds the >> overhead of another counter and a dispatch on a a fixnum modulo >> operation - IMHO negotiable. >> >> The change is helpful when sits watching on several file descriptors for >> requests _and_ uses several threads, message passing etc. to produce >> responses. > > Hi Joerg, > > I've tested with some varying types of load with Spiffy (using wrk, ab > and siege). The results are inconclusive either way. The patch doesn't > seem to make much of a difference, at least with a reasonably simple > Spiffy setup. > > This is quite an invasive patch, which moves whole swaths of code and > also complicates the scheduler further by adding more lists and tweaks, I can understand. What I'd like most would be a way to pull the scheduler out of the core system completely. The dual use and interference with force-primordial adds a lot of complexity to the current implementation as your related finding in the other thread shows. Use cases have different load or even just users want to play with scheduling policies for the fun of it. I can't proof the implementation I ran for the past years to work under every kind of load not yet tried. But it is IMHO simpler. Cuts out force-primordial, does not need to sys#dispatch-interupts which in turn could be removed everywhere. Nobody is going to swallow it as a whole, but in piecemeal supporting those it -sure- increases code complexity. > and even removes some things like the check for C_signal_interrupted_p, > which makes me very hesitant to just apply it for an alleged performance > boost. On the whole, I think it's too dangerous to fiddle with the > scheduler unless it brings either considerable simplification of the > scheduler's code, or measurable improvements for common workloads. For varying values of "common workload" ;-) In my case the standby load is about 50 child processes and 100-200 open file handles. Most of the latter are non-critical as they connect to network but the one signaling callbacks from external pthreads is critical. One there is load there can be several hundred short lived chicken threads passing messages around. > Also, you did mention getting segfaults with your version, so who's > to say this change doesn't *introduce* any segfault crashes? The code was running that way for about eight years or so. Segfaults came and went in this time. At the moment I don't recall when precisely, but I did not see those segfaults maybe mid-March. > So, unless you can show us how to reproduce this measured improvement, > I think it's best not to apply this patch. I see, the piecemeal approach will not work. We should work towards an API for the scheduler to hook into and remove it from core. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] shouldn't file-close dispatch an interrupt on EINTR?
Hi all, after literally years of wondering why I'm seeing lost file descriptors I _guess_ I spotted the reason. file-close raises an error when close(2) returns a negative value. IMHO it should look like this (not yet tested): (define file-close (lambda (fd) (##sys#check-exact fd 'file-close) (if (fx< (##core#inline "C_close" fd) 0) (select _errno ((_eintr) (##sys#dispatch-interrupt (lambda () (file-close fd (else (posix-error #:file-error 'file-close "cannot close file" fd))) ) ) ) Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] csc -profile has problems
Hi, csc -profile bails out on some code it otherwise compiles. So far the smallest example I have is the reference implementation of srfi-35. It complains that really-make-condition-type is called with the wrong number of arguments right here: (define-condition-type &message &condition message-condition? (message condition-message)) Another example is in the comparators egg: (define (binary? comparator a b) (binaryhttps://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
PS: regarding the pre-allocated argvector I forgot to mention one additional change you need to make: some procedures in runtime.c (e.g., values_continuation) either still need to C_alloc or be changed to work in case av==av2 holds. Look in the diff for C_memcpy replaced by C_memmove. Am 28.02.2016 um 15:19 schrieb Jörg F. Wittenberger: > There is an number of arguments limit documented in "Deviations from the > standard". (BTW: I guess the 1000 claimed there is outdated.) > > But is this limit checked anywhere? My reading of C_apply makes be > wonder. Am I missing a check elsewhere? > > Otherwise I'd add it with the patch. > > BTW: currently a make check completed here. This one re-uses a single, > stack allocated argvector of TEMPORARY_STACK_SIZE whenever the current > code would do C_alloc or av2[N]. Notable exception C_context_switch. > Interestingly: while this is the simplest code one could use, the length > tagged argvector is consistently about 2% faster here on all tests so far. > > Cheers > > /Jörg > > Am 27.02.2016 um 20:58 schrieb Jörg F. Wittenberger: >> Am 27.02.2016 um 14:46 schrieb Jörg F. Wittenberger: >>> Am 27.02.2016 um 13:09 schrieb Jörg F. Wittenberger: >>>> Am 27.02.2016 um 12:25 schrieb Jörg F. Wittenberger: >>>>> Hi folks, >>>>> >>>>> if you really consider anything to be done to the argvector handling >>>>> before the next release >>>> >>>> ... >>>> >>>> I wonder: why not malloc exactly one argvector of TEMPORARY_STACK_SIZE >>>> word and drop all the checking? >>>> >>>> Then either the current av vector is the one passed in, then we can >>>> safely reuse it. If it's not we re-use the global anyway. >>> >>> Looking at those options I wonder if there is not an even better option. >>> >>> This is the relevant change in my patch: >>> >>> #if USE_OLD_AV >>> #define C_allocate_fresh_argvector(n) C_alloc(n) >>> #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : >>> C_force_allocate_fresh_argvector(avl)) >>> #define C_argvector_flush() /* does nothing */ >>> >>> Those two are the macros doing the argvector handling. Here mapping >>> back to equivalent code of what master would do. Only those are called >>> from runtime.c and the C backend. Well, not exactly, there is a >>> C_kontinue_av, which is a av C_kontinue. Trivial. However runtime.c >>> already uses it (this is where the performance gain came in). But it's >>> #defined back to C_kontinue depending on USE_OLD_AV anyways. >>> >>> #else >>> >>> This is the implementation of the length tagged argvector. >>> >>> #define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) >>> && (C_default_argvector_value[0] >= (n))) >>> #define C_argvector_flush()(C_default_argvector_value = NULL) >>> >>> This would have to be changed to possibly reset the >>> C_default_argvector_value and return the temporary_stack. >>> >>> #define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value >>> = C_alloc((n)+1)), *C_default_argvector_value=(n), >>> C_default_argvector_value+1) >>> >>> #define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? >>> C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl)) >>> #define C_argvector_size(av) (av[-1]) >>> #define C_allocate_argvector(c, av, avl) c) >= (avl)) || >>> (C_argvector_size(av) >= (avl))) ? av : >>> C_force_allocate_fresh_argvector(avl)) >>> #endif >>> >>> If, instead, we would only ever put a pointer to a stack allocated >>> argument vector large enough for the apply count into the >>> C_default_argvector_value. Then we could forgo the whole length tagging >>> and checking. Even against "c" we may or may not want to check >>> (profiling will show). >>> >>> Something like this should work: >>> >>> #define C_allocate_fresh_argvector(avl) ( C_default_argvector_value != >>> NULL ? C_default_argvector_value : C_demand(avl) ? >>> C_default_argvector_value = C_alloc(TEMPORARY_STACK_SIZE) : >>> C_default_argvector_value = NULL, temporary_stack) >>> >>> Except for the TEMPORARY_STACK_SIZE, which is not available to macros >>> and that's a good thing. However as this not in no way critical code, >>> we might want to call into run
Re: [Chicken-hackers] Saver a patch: Re: Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Hi, sorry for the disturbance. I did not expect those to be unwelcome. I assumed that there was a problem and a fix too, it might be a good thing. Am 28.02.2016 um 17:49 schrieb Peter Bex: > On Sun, Feb 28, 2016 at 03:58:11PM +0100, Jörg F. Wittenberger wrote: >> Hi, >> >> the patch I sent is harmful. I forgot to reset chicken.h to "backward >> compatible". >> >> Attached a better version. > > I'd really appreciate it if you could stop bombarding the mailing list > with comments and patches. There's already an accumulated backlog of > patches that I need to look at, and the issue at hand (stack overflows) > is actually orthogonal to the idea of reusing argvectors more. I understood your original message as saying "when allocating an argvector we might overflow the stack". So if I miss-understood your statement, then yes, those things are orthogonal and we should not discuss the argvector changes now. But if the overflow happens because of an argvector allocation on the stack - then knowing that we do *not* allocate fresh memory for the argvector would IMHO solve this. Doesn't it? > Because these two things are orthogonal I would strongly prefer to fix > the stack overflow issue with the smallest possible patch and then make > a release. That's been the idea of mine too. > Besides, you're wasting time until this has been fixed because the > argvector code might get modified a lot, so that means your patches Originally I did not propose this to go into the current release. And the main point of the patch was to make upcoming changes easier by collecting argvector management into macros. Plus: using a argvector with a special structure (I first asserted that those would carry a fixed tag guard instead of the length) made it much easier for me to debug. > will probably conflict after the actual bug has been fixed. I had preferred to discuss in the sort term how to fix the bug and in the long term how to change the argvector code. At this point I don't mind those changes to conflict. That's what happens during the search for the best solution. Now the point wrt. the bug is: which bug? With the diff I sent yesterday there is no stack space required for the argvector ever. Sure the diff is nowhere near minimal. I could not have made the change without first sorting out what C_alloc is argvector related and what is not. But it contains IMHO the solution to the problem at hand. The latter could be ported back to master with a much smaller diff. That's what I'd be happy to do and had planned to look into today. But I would not bother if you folks don't like the change for some reason. Now I wonder what I should do. The diff I'd suggest would introduce one new global C_default_argvector and C_alloc tempstack space to this address in CHICKEN_run after setting C_restart. All cases where the code now uses C_alloc to create an argvector would have to be changed to return C_default_argvector. Watch out for the exception in C_context_switch that is. Cheers /Jörg PS: no diff attached ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] Saver a patch: Re: Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Hi,
the patch I sent is harmful. I forgot to reset chicken.h to "backward
compatible".
Attached a better version.
After patching, recompile to get a csc using the new macros. Then
change the first line in chicken.h to
#define USE_OLD_AV 0
recompile now.
Modify the second line to
#define USE_FIXED_DFLT ( !USE_OLD_AV && 0)
that is - change to 1 to 0 - if you want to try length-prefixed
argvector (I hope this still works).
Cheers
/Jörg
Am 28.02.2016 um 15:39 schrieb Jörg F. Wittenberger:
> Am 28.02.2016 um 15:19 schrieb Jörg F. Wittenberger:
>> BTW: currently a make check completed here. This one re-uses a single,
>> stack allocated argvector of TEMPORARY_STACK_SIZE whenever the current
>> code would do C_alloc or av2[N]. Notable exception C_context_switch.
>> Interestingly: while this is the simplest code one could use, the length
>> tagged argvector is consistently about 2% faster here on all tests so far.
>
> Attached the diff from my build tree to master.
>
> This obviously needs cleanup.
>
> But there needs to be a decision. Do we want simpler code at the
> expense of 2-5% runtime? Or do we want the length tagged version
> eventually?
diff --git a/c-backend.scm b/c-backend.scm
index 3f9846f..c96ae2c 100644
--- a/c-backend.scm
+++ b/c-backend.scm
@@ -483,13 +483,8 @@
;; CPS context, so callee never returns to current function.
;; And even so, av[] is already copied into temporaries.
(cond (caller-has-av?
- (gen #t "C_word *av2;")
- (gen #t "if(c >= " avl ") {")
- (gen #t " av2=av; /* Re-use our own argvector */")
- (gen #t "} else {")
- (gen #t " av2=C_alloc(" avl ");")
- (gen #t "}"))
- (else (gen #t "C_word av2[" avl "];")))
+ (gen #t "C_word *av2 = C_allocate_argvector(c, av, " avl ");"))
+ (else (gen #t "C_word *av2 = C_allocate_fresh_argvector(" avl ");")))
(when selfarg (gen #t "av2[0]=" selfarg ";"))
(do ((j (if selfarg 1 0) (add1 j))
(args args (cdr args)))
diff --git a/chicken.h b/chicken.h
index 3694cd6..92d1296 100644
--- a/chicken.h
+++ b/chicken.h
@@ -1,3 +1,5 @@
+#define USE_OLD_AV 1 // set to "true" for backward compatible version to boot modified chicken
+#define USE_FIXED_DFLT ( !USE_OLD_AV && 1)
/* chicken.h - General headerfile for compiler generated executables
;
; Copyright (c) 2008-2016, The CHICKEN Team
@@ -1010,6 +1012,43 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_heaptop ((C_word **)(&C_fromspace_top))
#define C_drop(n) (C_temporary_stack += (n))
#define C_alloc(n) ((C_word *)C_alloca((n) * sizeof(C_word)))
+
+#if USE_OLD_AV
+#define C_allocate_fresh_argvector(n) C_alloc(n)
+#define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+// TBD: runtime internal
+#define C_argvector_flush() 0
+#define C_argvector_size(av) 0
+#define C_force_allocate_fresh_argvector(x) C_allocate_fresh_argvector(n)
+#elsif USE_FIXED_DFLT
+// TBD: runtime internal
+#define C_argvector_flush()0 /* no longer used (C_default_argvector_value = NULL) */
+// remove this if this version is choosen, we don't need it, used only twice in runtime.c
+#define C_argvector_size(av) (C_default_argvector_value == av ? /*FIXME TEMPORARY_STACK_SIZE should be in runtime.c where it is known */ 4096 : 0)
+
+//#define C_force_allocate_fresh_argvector(avl) ( C_demand(avl) ? (C_default_argvector_value = C_alloc(avl)) : (C_argvector_flush(), C_temporary_stack))
+#define C_force_allocate_fresh_argvector(avl) (C_default_argvector_value = C_alloc(avl))
+
+// TDB: leave only these exported #defines
+//#define C_allocate_fresh_argvector(avl) ( /*FIXME should we assert(avl<=limit)*/ C_default_argvector_value != NULL ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl))
+#define C_allocate_fresh_argvector(avl) C_default_argvector_value
+
+// TBD: try this, may be faster: #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+// TBD: try assert(C_default_argvector_value != NULL) here and never look back if this works. (deferred)
+//#define C_allocate_argvector(c, av, avl) ( (C_default_argvector_value != NULL) ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl))
+#define C_allocate_argvector(c, av, avl) C_default_argvector_value
+
+#else
+#define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) && (C_default_argvector_value[0] >= (n)))
+#define C_argvector_flush()(C_default_argvector_value = NULL)
+#define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value = C_alloc((n)+1)), *C_default_argvector_value=(n), C_default_argvector_value+1)
+#define C_al
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Am 28.02.2016 um 15:19 schrieb Jörg F. Wittenberger:
> BTW: currently a make check completed here. This one re-uses a single,
> stack allocated argvector of TEMPORARY_STACK_SIZE whenever the current
> code would do C_alloc or av2[N]. Notable exception C_context_switch.
> Interestingly: while this is the simplest code one could use, the length
> tagged argvector is consistently about 2% faster here on all tests so far.
Attached the diff from my build tree to master.
This obviously needs cleanup.
But there needs to be a decision. Do we want simpler code at the
expense of 2-5% runtime? Or do we want the length tagged version
eventually?
Cheers
/Jörg
diff --git a/c-backend.scm b/c-backend.scm
index 3f9846f..c96ae2c 100644
--- a/c-backend.scm
+++ b/c-backend.scm
@@ -483,13 +483,8 @@
;; CPS context, so callee never returns to current function.
;; And even so, av[] is already copied into temporaries.
(cond (caller-has-av?
- (gen #t "C_word *av2;")
- (gen #t "if(c >= " avl ") {")
- (gen #t " av2=av; /* Re-use our own argvector */")
- (gen #t "} else {")
- (gen #t " av2=C_alloc(" avl ");")
- (gen #t "}"))
- (else (gen #t "C_word av2[" avl "];")))
+ (gen #t "C_word *av2 = C_allocate_argvector(c, av, " avl ");"))
+ (else (gen #t "C_word *av2 = C_allocate_fresh_argvector(" avl ");")))
(when selfarg (gen #t "av2[0]=" selfarg ";"))
(do ((j (if selfarg 1 0) (add1 j))
(args args (cdr args)))
diff --git a/chicken.h b/chicken.h
index 3694cd6..92d1296 100644
--- a/chicken.h
+++ b/chicken.h
@@ -1,3 +1,5 @@
+#define USE_OLD_AV 0 // set to "true" for backward compatible version to boot modified chicken
+#define USE_FIXED_DFLT ( !USE_OLD_AV && 1)
/* chicken.h - General headerfile for compiler generated executables
;
; Copyright (c) 2008-2016, The CHICKEN Team
@@ -1010,6 +1012,43 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_heaptop ((C_word **)(&C_fromspace_top))
#define C_drop(n) (C_temporary_stack += (n))
#define C_alloc(n) ((C_word *)C_alloca((n) * sizeof(C_word)))
+
+#if USE_OLD_AV
+#define C_allocate_fresh_argvector(n) C_alloc(n)
+#define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+// TBD: runtime internal
+#define C_argvector_flush() 0
+#define C_argvector_size(av) 0
+#define C_force_allocate_fresh_argvector(x) C_allocate_fresh_argvector(n)
+#elsif USE_FIXED_DFLT
+// TBD: runtime internal
+#define C_argvector_flush()0 /* no longer used (C_default_argvector_value = NULL) */
+// remove this if this version is choosen, we don't need it, used only twice in runtime.c
+#define C_argvector_size(av) (C_default_argvector_value == av ? /*FIXME TEMPORARY_STACK_SIZE should be in runtime.c where it is known */ 4096 : 0)
+
+//#define C_force_allocate_fresh_argvector(avl) ( C_demand(avl) ? (C_default_argvector_value = C_alloc(avl)) : (C_argvector_flush(), C_temporary_stack))
+#define C_force_allocate_fresh_argvector(avl) (C_default_argvector_value = C_alloc(avl))
+
+// TDB: leave only these exported #defines
+//#define C_allocate_fresh_argvector(avl) ( /*FIXME should we assert(avl<=limit)*/ C_default_argvector_value != NULL ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl))
+#define C_allocate_fresh_argvector(avl) C_default_argvector_value
+
+// TBD: try this, may be faster: #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+// TBD: try assert(C_default_argvector_value != NULL) here and never look back if this works. (deferred)
+//#define C_allocate_argvector(c, av, avl) ( (C_default_argvector_value != NULL) ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl))
+#define C_allocate_argvector(c, av, avl) C_default_argvector_value
+
+#else
+#define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) && (C_default_argvector_value[0] >= (n)))
+#define C_argvector_flush()(C_default_argvector_value = NULL)
+#define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value = C_alloc((n)+1)), *C_default_argvector_value=(n), C_default_argvector_value+1)
+#define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl))
+#define C_argvector_size(av) (av[-1])
+//#define C_allocate_argvector(c, av, avl) (C_argvector_size(av) >= (avl) ? av : C_force_allocate_fresh_argvector(avl))
+// should try this too (turned out to be faster here):
+#define C_allocate_argvector(c, av, avl) c) >= (avl)) || (C_argvector_size(av) >= (avl))) ? av : C_force_allocate_fresh_argvector(avl))
+#endif
+
#if defined (__llvm__) && defined (__G
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
There is an number of arguments limit documented in "Deviations from the standard". (BTW: I guess the 1000 claimed there is outdated.) But is this limit checked anywhere? My reading of C_apply makes be wonder. Am I missing a check elsewhere? Otherwise I'd add it with the patch. BTW: currently a make check completed here. This one re-uses a single, stack allocated argvector of TEMPORARY_STACK_SIZE whenever the current code would do C_alloc or av2[N]. Notable exception C_context_switch. Interestingly: while this is the simplest code one could use, the length tagged argvector is consistently about 2% faster here on all tests so far. Cheers /Jörg Am 27.02.2016 um 20:58 schrieb Jörg F. Wittenberger: > Am 27.02.2016 um 14:46 schrieb Jörg F. Wittenberger: >> Am 27.02.2016 um 13:09 schrieb Jörg F. Wittenberger: >>> Am 27.02.2016 um 12:25 schrieb Jörg F. Wittenberger: >>>> Hi folks, >>>> >>>> if you really consider anything to be done to the argvector handling >>>> before the next release >>> >>> ... >>> >>> I wonder: why not malloc exactly one argvector of TEMPORARY_STACK_SIZE >>> word and drop all the checking? >>> >>> Then either the current av vector is the one passed in, then we can >>> safely reuse it. If it's not we re-use the global anyway. >> >> Looking at those options I wonder if there is not an even better option. >> >> This is the relevant change in my patch: >> >> #if USE_OLD_AV >> #define C_allocate_fresh_argvector(n) C_alloc(n) >> #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : >> C_force_allocate_fresh_argvector(avl)) >> #define C_argvector_flush() /* does nothing */ >> >> Those two are the macros doing the argvector handling. Here mapping >> back to equivalent code of what master would do. Only those are called >> from runtime.c and the C backend. Well, not exactly, there is a >> C_kontinue_av, which is a av C_kontinue. Trivial. However runtime.c >> already uses it (this is where the performance gain came in). But it's >> #defined back to C_kontinue depending on USE_OLD_AV anyways. >> >> #else >> >> This is the implementation of the length tagged argvector. >> >> #define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) >> && (C_default_argvector_value[0] >= (n))) >> #define C_argvector_flush()(C_default_argvector_value = NULL) >> >> This would have to be changed to possibly reset the >> C_default_argvector_value and return the temporary_stack. >> >> #define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value >> = C_alloc((n)+1)), *C_default_argvector_value=(n), >> C_default_argvector_value+1) >> >> #define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? >> C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl)) >> #define C_argvector_size(av) (av[-1]) >> #define C_allocate_argvector(c, av, avl) c) >= (avl)) || >> (C_argvector_size(av) >= (avl))) ? av : >> C_force_allocate_fresh_argvector(avl)) >> #endif >> >> If, instead, we would only ever put a pointer to a stack allocated >> argument vector large enough for the apply count into the >> C_default_argvector_value. Then we could forgo the whole length tagging >> and checking. Even against "c" we may or may not want to check >> (profiling will show). >> >> Something like this should work: >> >> #define C_allocate_fresh_argvector(avl) ( C_default_argvector_value != >> NULL ? C_default_argvector_value : C_demand(avl) ? >> C_default_argvector_value = C_alloc(TEMPORARY_STACK_SIZE) : >> C_default_argvector_value = NULL, temporary_stack) >> >> Except for the TEMPORARY_STACK_SIZE, which is not available to macros >> and that's a good thing. However as this not in no way critical code, >> we might want to call into runtime.c like >> C_a_i_allocate_fresh_argvector(naming convections?) which would respect >> runtime options etc. > > > "make check" just passed for modifications like this > > #if USE_OLD_AV // NOT APPLICABLE > ... > #elsif USE_FIXED_DFLT // THIS IS WHAT IS ACTUALLY EXPANDED > // TBD: runtime internal > #define C_argvector_flush()(C_default_argvector_value = NULL) > #define C_argvector_size(av) (C_default_argvector_value == av ? /*FIXME > TEMPORARY_STACK_SIZE should be in runtime.c where it is known */ 4096 : 0) > > #define C_force_allocate_fresh_argvector(avl) ( C_demand(avl) ? > (C_default_argvector_value = C_alloc(avl)) : (C_argve
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Am 27.02.2016 um 14:46 schrieb Jörg F. Wittenberger: > Am 27.02.2016 um 13:09 schrieb Jörg F. Wittenberger: >> Am 27.02.2016 um 12:25 schrieb Jörg F. Wittenberger: >>> Hi folks, >>> >>> if you really consider anything to be done to the argvector handling >>> before the next release >> >> ... >> >> I wonder: why not malloc exactly one argvector of TEMPORARY_STACK_SIZE >> word and drop all the checking? >> >> Then either the current av vector is the one passed in, then we can >> safely reuse it. If it's not we re-use the global anyway. > > Looking at those options I wonder if there is not an even better option. > > This is the relevant change in my patch: > > #if USE_OLD_AV > #define C_allocate_fresh_argvector(n) C_alloc(n) > #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : > C_force_allocate_fresh_argvector(avl)) > #define C_argvector_flush() /* does nothing */ > > Those two are the macros doing the argvector handling. Here mapping > back to equivalent code of what master would do. Only those are called > from runtime.c and the C backend. Well, not exactly, there is a > C_kontinue_av, which is a av C_kontinue. Trivial. However runtime.c > already uses it (this is where the performance gain came in). But it's > #defined back to C_kontinue depending on USE_OLD_AV anyways. > > #else > > This is the implementation of the length tagged argvector. > > #define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) > && (C_default_argvector_value[0] >= (n))) > #define C_argvector_flush()(C_default_argvector_value = NULL) > > This would have to be changed to possibly reset the > C_default_argvector_value and return the temporary_stack. > > #define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value > = C_alloc((n)+1)), *C_default_argvector_value=(n), > C_default_argvector_value+1) > > #define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? > C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl)) > #define C_argvector_size(av) (av[-1]) > #define C_allocate_argvector(c, av, avl) c) >= (avl)) || > (C_argvector_size(av) >= (avl))) ? av : > C_force_allocate_fresh_argvector(avl)) > #endif > > If, instead, we would only ever put a pointer to a stack allocated > argument vector large enough for the apply count into the > C_default_argvector_value. Then we could forgo the whole length tagging > and checking. Even against "c" we may or may not want to check > (profiling will show). > > Something like this should work: > > #define C_allocate_fresh_argvector(avl) ( C_default_argvector_value != > NULL ? C_default_argvector_value : C_demand(avl) ? > C_default_argvector_value = C_alloc(TEMPORARY_STACK_SIZE) : > C_default_argvector_value = NULL, temporary_stack) > > Except for the TEMPORARY_STACK_SIZE, which is not available to macros > and that's a good thing. However as this not in no way critical code, > we might want to call into runtime.c like > C_a_i_allocate_fresh_argvector(naming convections?) which would respect > runtime options etc. "make check" just passed for modifications like this #if USE_OLD_AV // NOT APPLICABLE ... #elsif USE_FIXED_DFLT // THIS IS WHAT IS ACTUALLY EXPANDED // TBD: runtime internal #define C_argvector_flush()(C_default_argvector_value = NULL) #define C_argvector_size(av) (C_default_argvector_value == av ? /*FIXME TEMPORARY_STACK_SIZE should be in runtime.c where it is known */ 4096 : 0) #define C_force_allocate_fresh_argvector(avl) ( C_demand(avl) ? (C_default_argvector_value = C_alloc(avl)) : (C_argvector_flush(), C_temporary_stack)) // TDB: leave only these exported #defines #define C_allocate_fresh_argvector(avl) ( /*FIXME should we assert(avl<=limit)*/ C_default_argvector_value != NULL ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl)) // TBD: try this, may be faster: #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl)) // TBD: try assert(C_default_argvector_value != NULL) here and never look back if this works. (deferred) #define C_allocate_argvector(c, av, avl) ( (C_default_argvector_value != NULL) ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl)) #else ... the length tagged argvector version here #endif Let me call it a day now. Tomorrow is the dog's day. More testing, benchmark and a patch not before Monday. Cheers /Jörg > > As long as no code flushes the C_default_argvector value, that one will > be reused anyway. Flushing it does the C_reclaim. Last resort is the > temporary stack. Passing as argument vector we still can anything. >
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Am 27.02.2016 um 13:09 schrieb Jörg F. Wittenberger: > Am 27.02.2016 um 12:25 schrieb Jörg F. Wittenberger: >> Hi folks, >> >> if you really consider anything to be done to the argvector handling >> before the next release > > ... > > I wonder: why not malloc exactly one argvector of TEMPORARY_STACK_SIZE > word and drop all the checking? > > Then either the current av vector is the one passed in, then we can > safely reuse it. If it's not we re-use the global anyway. Looking at those options I wonder if there is not an even better option. This is the relevant change in my patch: #if USE_OLD_AV #define C_allocate_fresh_argvector(n) C_alloc(n) #define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl)) #define C_argvector_flush() /* does nothing */ Those two are the macros doing the argvector handling. Here mapping back to equivalent code of what master would do. Only those are called from runtime.c and the C backend. Well, not exactly, there is a C_kontinue_av, which is a av C_kontinue. Trivial. However runtime.c already uses it (this is where the performance gain came in). But it's #defined back to C_kontinue depending on USE_OLD_AV anyways. #else This is the implementation of the length tagged argvector. #define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) && (C_default_argvector_value[0] >= (n))) #define C_argvector_flush()(C_default_argvector_value = NULL) This would have to be changed to possibly reset the C_default_argvector_value and return the temporary_stack. #define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value = C_alloc((n)+1)), *C_default_argvector_value=(n), C_default_argvector_value+1) #define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl)) #define C_argvector_size(av) (av[-1]) #define C_allocate_argvector(c, av, avl) c) >= (avl)) || (C_argvector_size(av) >= (avl))) ? av : C_force_allocate_fresh_argvector(avl)) #endif If, instead, we would only ever put a pointer to a stack allocated argument vector large enough for the apply count into the C_default_argvector_value. Then we could forgo the whole length tagging and checking. Even against "c" we may or may not want to check (profiling will show). Something like this should work: #define C_allocate_fresh_argvector(avl) ( C_default_argvector_value != NULL ? C_default_argvector_value : C_demand(avl) ? C_default_argvector_value = C_alloc(TEMPORARY_STACK_SIZE) : C_default_argvector_value = NULL, temporary_stack) Except for the TEMPORARY_STACK_SIZE, which is not available to macros and that's a good thing. However as this not in no way critical code, we might want to call into runtime.c like C_a_i_allocate_fresh_argvector(naming convections?) which would respect runtime options etc. As long as no code flushes the C_default_argvector value, that one will be reused anyway. Flushing it does the C_reclaim. Last resort is the temporary stack. Passing as argument vector we still can anything. > The downside: now we have one more area to scan in the garbage > collector. (That's why I preferred to stack allocated one so far). Upside: not additional code in the gc. > However reading the related code I get the feeling that we even could > simply use the temporary_stack as the argvector. However I'm not sure > about that one. > > Just thoughts. Comments? Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Am 27.02.2016 um 12:25 schrieb Jörg F. Wittenberger: > Hi folks, > > if you really consider anything to be done to the argvector handling > before the next release ... I wonder: why not malloc exactly one argvector of TEMPORARY_STACK_SIZE word and drop all the checking? Then either the current av vector is the one passed in, then we can safely reuse it. If it's not we re-use the global anyway. The downside: now we have one more area to scan in the garbage collector. (That's why I preferred to stack allocated one so far). However reading the related code I get the feeling that we even could simply use the temporary_stack as the argvector. However I'm not sure about that one. Just thoughts. Best /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Release blocker? Stack checks completely FUBAR? Or am I just paranoid?
Hi folks, if you really consider anything to be done to the argvector handling before the next release, please have a look at the patch I posted recently here http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00082.html not for the minor performance gain, read it for the re-structuring it does. Do not yet apply. It's outdated already due to conflicting changes in cthulu (actually a minor change easy to resolve manually). I'm happy to produce a new one incorporating the lessons learned discussing this issue here. The main advantage of the patch in this situation is that it already pulls all the argvector related allocations into two macros. Easy to be change to try what works best. Since the first patch #defines them back to C_alloc as before it leave you with binary compatible chicken. If activated, the argvector is allocated afresh after garbage collection and basically always re-used once it grew to the maximum size (the only exception are where the compiler actually produces code using C_kontinue) plus ...see below or look for comments containing "no av-reuse" in the patch. However I have not yet seen a case where the c-backend could not be changed to call the C_kontinue_av passing the old argvector too. This reduces the odds of an argvector allocation to overflow the stack to those cases where a large vector is allocated rarely by the application (otherwise it would have had a high chance to have allocate a large argvector shortly after gc, which did fit the stack and is still reused). So this is saying: I'd guess it mitigates the problem. Am 26.02.2016 um 16:25 schrieb Peter Bex: > On Fri, Feb 26, 2016 at 04:03:12PM +0100, felix.winkelm...@bevuta.com wrote: >>> These two probably need to be fixed before we can do a release. >>> The second was always broken but it might not have been such a big >>> deal because we never allocated as much on the stack(?). The first >>> is new with argvectors. >> >> Unless one operates with native threads, the stack can be considered >> "large enough" (IIRC, it even grows on demand on modern systems, >> up to certain limits, but that may be wrong) > > Yeah, but the C_stack_check will raise an exception if it grows beyond > our arbitrarily defined soft limit. See the Salmonella link in my mail. > >>> Like I said, I'm a bit ill so this may all be just a fever dream, but >>> the proper solution eludes me. A fix for 2) might involve collecting >>> all the possible function calls a generated function will perform and >>> then pre-allocating the av2 (but only when needed), and then checking >>> whether that would cause the stack to overflow. >> >> Do you mean 1) here? Since a CPS-procedure will call only a limited >> amount of CPS functions (usually 1), this should be doable. In the long run, I'd prefer this approach. However I'd guess adding another stack check would not do too much harm. Especially not when usually re-using the argvector. It occurs to me that it would only be a minor change to C_force_allocate_fresh_argvector which is only called once the re-use tests already failed. Instead of allocating with C_alloc and caching the value as default, it would have to check the stack and if it does not fit flush the default vector and return room from the temporary stack. > Yeah, I probably meant 1). Alternatively, we may make the allocation of > the vector return a pointer into the C stack *or* into the temporary stack > depending on whether there's still room in the C stack. Something like this is already happening. Tagged "no av-reuse" in the diff. > Then, fill the argvector with arguments and either call the function or > call into the GC with the CPS closure just like we would do at the start > of the CPS function. The disadvantage is that we'd be doing one more > stack check, the advantage is that when we call more CPS functions, we > don't need to be pessimistic by using the maximum of the argument counts. I don't understand this. >>> A fix for 1), if I'm not mistaken, would be to swap the way the stack >>> checks are done: the C_demand and C_stack_probe should observe the >>> reserve, while C_stack_check1() should only look at the hard limit. >> >> (2?) I'm not sure whether I'm following completely, but the reserve should >> probably respected in any case. > > Yeah, but it should be consistently applied in both cases, I think. > Currently the stack probe/demand at the start of a CPS function doesn't > respect the reserve while the C_stack_check does respect it, causing > these spurious "stack overflow" errors. In the (very) long run, I'd wonder if it would be possible to define that stack checks are done either on entry or at the call site, but not both. However this should not delay a release. >>> Besides, I think C_stack_check1 looks too complicated; why does it >>> create the temporary *_sp variable? Is that to avoid a pointer >>> overflow/underflow? Couldn't it just do a check like this: >>> >>> if ((C_byte *)C_st
Re: [Chicken-hackers] slow polling
Am 24.02.2016 um 10:29 schrieb Peter Bex: > I really hope you understand this and can agree with it. As I wrote in this other mail before I received this one: how about removing the identifier from the hide list. This should not be able to break anything and allow me to get rid of the overhead. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] force-primordial: proposal of a compromise solution - was Re: slow polling
Am 24.02.2016 um 10:21 schrieb Jörg F. Wittenberger: > Am 24.02.2016 um 10:15 schrieb Jörg F. Wittenberger: >> Am 24.02.2016 um 09:29 schrieb felix.winkelm...@bevuta.com: >>>> I've done some tests and I was able to get strange behaviour from the >>>> scheduler, see >>>> http://paste.call-cc.org/paste?id=d9e4c5b8f8473fd1114dcec56c9c8a079b252362 >>>> However, none of these behaved any different under CHICKEN 4.9.0.1 >>>> or 4.10.0, so it's not strictly speaking a regression that MUST be fixed >>>> before 4.11. > > However: it appears to me that there is nobody who can explain why we > need this function to be there. Probably it had a reason in some very > early version of chicken. But Felix: do you recall? > > At this time it is IMHO just one more bug. A bug which bites. Why not > fix it? Thinking twice: how about a compromise solution: Remove ##sys#force-primordial from the 'hide' list in scheduler.scm. This would allow me to use mutate-procedure! in my code to overwrite it to get rid of the problem. Much better than asking people to install a modified chicken. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] slow polling
Am 24.02.2016 um 10:15 schrieb Jörg F. Wittenberger: > Am 24.02.2016 um 09:29 schrieb felix.winkelm...@bevuta.com: >>> I've done some tests and I was able to get strange behaviour from the >>> scheduler, see >>> http://paste.call-cc.org/paste?id=d9e4c5b8f8473fd1114dcec56c9c8a079b252362 >>> However, none of these behaved any different under CHICKEN 4.9.0.1 >>> or 4.10.0, so it's not strictly speaking a regression that MUST be fixed >>> before 4.11. However: it appears to me that there is nobody who can explain why we need this function to be there. Probably it had a reason in some very early version of chicken. But Felix: do you recall? At this time it is IMHO just one more bug. A bug which bites. Why not fix it? ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] slow polling
Am 24.02.2016 um 09:29 schrieb felix.winkelm...@bevuta.com: >> I've done some tests and I was able to get strange behaviour from the >> scheduler, see >> http://paste.call-cc.org/paste?id=d9e4c5b8f8473fd1114dcec56c9c8a079b252362 >> However, none of these behaved any different under CHICKEN 4.9.0.1 >> or 4.10.0, so it's not strictly speaking a regression that MUST be fixed >> before 4.11. > > Does it have to be fixed in Chicken 4 at all? I think every effort should put > into getting > Chicken 5 ready. The current pace is insane and will not be sustainable > forever. Let's > get 4.11 out and then turn to new things. Not necessarily. But it may make it much, much easier for me to participate in the process. Those scheduler bugs are the reason why I'm forced to distribute a modified version of chicken (with a modified scheduler) for several years along with the software, which those bugs break. I'd love to get rid of that deviation. It burns my time for no good reason. At this point I'm not able to make complete transition from 4.9 to 4.11 because I'm loosing too much to make practical use of it. I have no idea so far what changes are coming in from chicken 5 and I won't be able to find out if I can't run my code. Please don't leave me hanging. Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Argvector handling - maybe we could do better at that
Am 19.02.2016 um 13:59 schrieb Peter Bex:
> I'd be happy to hear if it gives good results, of course.
You did not expect earth shattering results, did you?
Here some numbers. At this time only unit-benchmarks using the tests
from llrb-tree egg.
Numbers are execution time of (master / modified - master) in %:
inserts reference
hash fixnum key 3.84 5.44
hash symbol key 1.83 -0.91
llrb fixnum key 9.84 5.06
llrb fixnum mutating 8.42 1.22
llrb (using comparators) 6.63 1.91
llrb symbol key 1.07 9.72
llrb string key 7.81 3.73
In words: with the exception of symbol-indexed hash tables I've got
between 1-10% performance increase on average for 10 runs of 10^6
operations using the same 10^6 random keys.
Some preliminary test using mixed load appear to confirm these sub-10%
gains.
Two patches attached.
1.) The first is _supposed_ to do no harm by itself at all. It contains
all changes but ifdefs them back to backward compatible code. You need
to apply this and compile a new chicken, which will produce code using
the modified argvector related macros.
At this time it might be a good idea to backup your chicken-boot and
make a new one.
Now changing the first line in chicken.h should make you chicken
actually use the modified argvector handling.
I ran successfully "make check" (and the benchmarks and my code) with
the modified version.
2.) The second patch removes the transitional code.
Let me know if this causes any issue.
Cheers
/Jörg
From 7b11f8b7c022589d46754e091368b16c02980596 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?=
Date: Tue, 23 Feb 2016 19:58:07 +0100
Subject: [PATCH] increase reuse of argvector
---
c-backend.scm | 9 +--
chicken.h | 26 ++-
runtime.c | 218 +++---
3 files changed, 142 insertions(+), 111 deletions(-)
diff --git a/c-backend.scm b/c-backend.scm
index 3f9846f..c96ae2c 100644
--- a/c-backend.scm
+++ b/c-backend.scm
@@ -483,13 +483,8 @@
;; CPS context, so callee never returns to current function.
;; And even so, av[] is already copied into temporaries.
(cond (caller-has-av?
- (gen #t "C_word *av2;")
- (gen #t "if(c >= " avl ") {")
- (gen #t " av2=av; /* Re-use our own argvector */")
- (gen #t "} else {")
- (gen #t " av2=C_alloc(" avl ");")
- (gen #t "}"))
- (else (gen #t "C_word av2[" avl "];")))
+ (gen #t "C_word *av2 = C_allocate_argvector(c, av, " avl ");"))
+ (else (gen #t "C_word *av2 = C_allocate_fresh_argvector(" avl ");")))
(when selfarg (gen #t "av2[0]=" selfarg ";"))
(do ((j (if selfarg 1 0) (add1 j))
(args args (cdr args)))
diff --git a/chicken.h b/chicken.h
index 3694cd6..845aac4 100644
--- a/chicken.h
+++ b/chicken.h
@@ -1,3 +1,4 @@
+#define USE_OLD_AV 1 // set to 1 for backward compatible version to boot modified chicken
/* chicken.h - General headerfile for compiler generated executables
;
; Copyright (c) 2008-2016, The CHICKEN Team
@@ -1010,6 +1011,21 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_heaptop ((C_word **)(&C_fromspace_top))
#define C_drop(n) (C_temporary_stack += (n))
#define C_alloc(n) ((C_word *)C_alloca((n) * sizeof(C_word)))
+
+#if USE_OLD_AV
+#define C_allocate_fresh_argvector(n) C_alloc(n)
+#define C_allocate_argvector(c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+#else
+#define C_argvector_reuse_dflt(n) ((C_default_argvector_value != NULL) && (C_default_argvector_value[0] >= (n)))
+#define C_argvector_flush()(C_default_argvector_value = NULL)
+#define C_force_allocate_fresh_argvector(n) ((C_default_argvector_value = C_alloc((n)+1)), *C_default_argvector_value=(n), C_default_argvector_value+1)
+#define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? C_default_argvector_value+1 : C_force_allocate_fresh_argvector(avl))
+#define C_argvector_size(av) (av[-1])
+//#define C_allocate_argvector(c, av, avl) (C_argvector_size(av) >= (avl) ? av : C_force_allocate_fresh_argvector(avl))
+// should try this too:
+#define C_allocate_argvector(c, av, avl) c) >= (avl)) || (C_argvector_size(av) >= (avl))) ? av : C_force_allocate_fresh_argvector(avl))
+#endif
+
#if defined (__llvm__) && defined (__GNUC__)
# if defined (__i386__)
# define C_stack_pointer ({C_word *sp; __asm__ __volatile__("movl %%esp,%0":"=r"(sp):);sp;})
@@ -1225,7 +1241,13 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_block_address(ptr, n, x) C_a_unsigned_int_to_num(ptr, n, x)
#define C_offset_pointer(x, y) (C_pointer_address(x) + (y))
#define C_do_apply(c, av) ((C_proc)(void *)C_block_item((av)[0], 0))((c), (av))
-#define C_kontinue(k, r)do { C_word avk[ 2 ]; avk[ 0 ] = (k); avk[ 1 ] = (r); ((C_proc)(void *)C_block_ite
Re: [Chicken-hackers] slow polling
Am 21.02.2016 um 19:52 schrieb Peter Bex: > On Sun, Feb 21, 2016 at 07:43:06PM +0100, Jörg F. Wittenberger wrote: >> Hi folks, >> >> IMHO the patch in >> http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00032.html >> SHOULD be applied - it fixes a serious bug. > > This is a very invasive change. Do you have a test case that shows > how this breaks? In practice: yes I do. Right here.[1] Now this is not a proof. However I don't know yet how to produce a simple test case. The problem is that we'd need to force the signal to be delivered during poll/select. How would I write a test which does? Alternatively we may try to _understand_ what the code is going to do and what it is supposed to do. What we need to do is: restart the poll/select/sleep after handling the signal. The latter is done in the forced gc. The former by the loop in ##sys#schedule. What forcing the primordial is going to to is easy to see: it will continue with the procedure stored in slot 1 of the primordial thread. Whatever that is.[1] Doing so is obviously often wrong. Now there is IMHO no point in going after those cases. The important question is: what is this primordial thread supposed to do at this point? I simply don't see _any_ reason to unleash it. But I understand that unleashing it is doing damage. Just try disabling (try the simple path - make ##sys#force-primordial to do nothing) and test. I could not find any damage. Cheers /Jörg [1] My app runs for a while and then segfaults. When I enable "dbg" in scheduler.scm (or - as I did to verify that this is the problem - have a "dbg2" doing the same but use it only for the case where polling returns -1) I see this being the last thing before segfault. [2] Blocked on mutex or condition variable -> blow through (always wrong), sitting in thread-join! -> depends on thread state, may blow though, block again without doing anything else (the only good case) or in raise exceptions for 'ready' and 'running'. (In my case the primordial sits in thread-join! and the joined thread _may_ be active, though I did not try to follow how it actually came to the segfault.) For blocking in i/o I did not check. ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] slow polling
Hi folks, IMHO the patch in http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00032.html SHOULD be applied - it fixes a serious bug. The modified scheduling policy may be considered once 4.11 is out. Switching the timeout queue to a priority queue still looks as if it is worth it. But...code complexity. The rest may be a read herring. Almost that is. There is definitely something measurable to gain, but it is little - not worth the additional build-time complexity. At least not yet at this time[1]. This throws me back to the drawing board. How do I figure out where chicken spends those 30% additional runtime 4.9 did not? Any clue appreciated. Cheers /Jörg [1] I have a working version of those modifications here. Now I understand that the repeated re-creation of the fdset is because it needs to be compatible with the select(2) based version. Alternatively (as I submitted the code) we need to conditionally compile scheme code on NO_POSIX_POLL - apparently I'm not the only one who does not know how to do that. Am 19.02.2016 um 18:46 schrieb Jörg F. Wittenberger: > Am 19.02.2016 um 14:02 schrieb Jörg F. Wittenberger: >> Now I see the low load case to be almost 50% faster and the old code to >> be 30% faster and more stable on delivering responses. I have no idea >> what the combined effect would be. >> >> How do we deal with this? > > I opened ticket 1259 for this. > > To make the kind reviewers job easier, I'll post diffs in piecemeal here. > > Details to follow below. > > Cheers > > /Jörg > > To follow the related changes: > > 1. Apply > > http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00032.html > > this should just fix a bug > > 2. Apply > > http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00033.html > > 3. Apply the attached patch. > > This only adds comments: Section headers large enough for "meld" at > least to pick up the difference. (To both the original and master.) > You want this to make head or tail of the upcoming changes. Eventually > there will have to be a cleanup diff to get rid of them. ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH Re: slow polling
Fixes for last patch. Sorry, this had to happen sooner or later. This patch just fixes use of outdates predicates. Cheers /Jörg BTW: Interesting that this did work to some extend. (#1 reason to begin with the timeout queue was that the original code still uses fixnum timeouts for backward compatibility). Am 20.02.2016 um 18:54 schrieb Jörg F. Wittenberger: > Am 19.02.2016 um 22:39 schrieb Jörg F. Wittenberger: >> ... >>> I opened ticket 1259 for this. >>> >>> To make the kind reviewers job easier, I'll post diffs in piecemeal here. > > This patch goes after killing a single - but important - comment line in > scheduler.scm: > > ;; This should really use a balanced tree: > > Now it does. > > This patch replaces the timeout queue with a balanced tree. > > >It does not matter so much, which kind of tree we use. But a >linear list is really a bad choice. > >Assume you have a tcp-server alike: 100 client connections and >you read the next input line. It's probably (but not sure) already >in the OS's buffer. But chicken core will nevertheless establish a >timeout. The latter will so far traverse the (linear) timeout >queue. Chances are all those other timeouts are prior established >reads using the same timeout too. Thus you find the insert point >right at the end. > > > If you have an application which makes heavy use of chicken's core tcp > unit (I don't use chicken's timeouts at all for this, I do use them, but > only via thread-sleep!) I'd be interested to hear how much good or bad > this patch does for you. > > Cheers > > /Jörg >From 697b8780cb57a9720affc8216d194c211fdef49a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Sat, 20 Feb 2016 19:15:26 +0100 Subject: [PATCH] Fix missuse of predicate --- scheduler.scm | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/scheduler.scm b/scheduler.scm index 997668f..7081acf 100644 --- a/scheduler.scm +++ b/scheduler.scm @@ -334,8 +334,7 @@ EOF (let ((entry (make-timeout-list-entry tm (list t (##sys#setslot t 4 entry) (set! ##sys#timeout-list-head entry))) - ((fx> tm - (prio-queue-node-index ##sys#timeout-list-head)) + ((timeout< (prio-queue-node-index ##sys#timeout-list-head) tm) (let ((entry (timeout-queue-node-lookup ##sys#timeout-list tm))) (if entry (begin @@ -346,8 +345,7 @@ EOF (let ((entry (make-timeout-list-entry tm (list t (##sys#setslot t 4 entry) (timeout-queue-node-insert! ##sys#timeout-list entry) - ((fx< tm - (prio-queue-node-index ##sys#timeout-list-head)) + ((timeout< tm (prio-queue-node-index ##sys#timeout-list-head)) (timeout-queue-node-insert! ##sys#timeout-list ##sys#timeout-list-head) (let ((entry (make-timeout-list-entry tm (list t @@ -535,7 +533,7 @@ dunno what to do ;; Sleep for the number of milliseconds of next thread ;; to wake up. (let ((tmo (prio-queue-node-index (timeout-queue-next - (##core#inline "C_msleep" (fxmax 0 (##core#inline "C_quickflonumtruncate" (fp- tmo now ) + (##core#inline "C_msleep" (fpmax 0 (##core#inline "C_quickflonumtruncate" (fp- tmo now ) (define (##sys#thread-block-for-timeout! t tm) (dbg t " blocks for timeout " tm) -- 2.6.2 ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] PATCH Re: slow polling
Am 19.02.2016 um 22:39 schrieb Jörg F. Wittenberger: > ... >> I opened ticket 1259 for this. >> >> To make the kind reviewers job easier, I'll post diffs in piecemeal here. This patch goes after killing a single - but important - comment line in scheduler.scm: ;; This should really use a balanced tree: Now it does. This patch replaces the timeout queue with a balanced tree. It does not matter so much, which kind of tree we use. But a linear list is really a bad choice. Assume you have a tcp-server alike: 100 client connections and you read the next input line. It's probably (but not sure) already in the OS's buffer. But chicken core will nevertheless establish a timeout. The latter will so far traverse the (linear) timeout queue. Chances are all those other timeouts are prior established reads using the same timeout too. Thus you find the insert point right at the end. If you have an application which makes heavy use of chicken's core tcp unit (I don't use chicken's timeouts at all for this, I do use them, but only via thread-sleep!) I'd be interested to hear how much good or bad this patch does for you. Cheers /Jörg From 38c2b03e6a96097fa4fd4eeb4971305d084ae90f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Sat, 20 Feb 2016 18:13:49 +0100 Subject: [PATCH] Use balanced a tree for the timeout queue --- distribution/manifest | 1 + llrbtree.scm | 505 ++ scheduler.scm | 333 ++--- 3 files changed, 777 insertions(+), 62 deletions(-) create mode 100644 llrbtree.scm diff --git a/distribution/manifest b/distribution/manifest index 1dd037f..3fc253a 100644 --- a/distribution/manifest +++ b/distribution/manifest @@ -28,6 +28,7 @@ posixunix.c posixwin.c profiler.c scheduler.c +llrbtree.scm srfi-69.c srfi-1.c srfi-13.c diff --git a/llrbtree.scm b/llrbtree.scm new file mode 100644 index 000..4b19e4d --- /dev/null +++ b/llrbtree.scm @@ -0,0 +1,505 @@ +;; #!/usr/bin/csi +;; (C) 2008, 2010, 2013 Jörg F. Wittenberger. + +;; Redistribution permitted under either GPL, LGPL or BSD style +;; license. + +;; (use extras) + +;; Changes +;; Rewritten from the 2008 version; now in syntax-rules. + +;;* Left Leaning Red Black Tree + +;;** Code Generator + +;; Generate LLRB trees within arbitrary datastructures. + +(define-syntax define-llrbtree/positional + (syntax-rules () +((_ + ;; The "features" is a list of symbols to control code + ;; expansion. "pure" expands to an implementation, which never + ;; updates nodes. "ordered" will enforce total order among the + ;; element. "leftmost" will include code to maintain a leftmost + ;; value of the tree (not recommended, may be removed). + features + ;; The "update*" syntax must accept a node structure and + ;; key-value pairs. Keys are color:, left: and right: + + ;; "update" : If feature "pure" is set, "update" must expand + ;; to a newly allocated node, otherwise is MUST expand to a + ;; side effect full update of the original node. + update + ;; The following identifiers are bound in the expanded code. + ;; Pass #f for procedures not to be expanded. + init-root-node! ;; defined + t-lookup ;; defined + t-min ;; defined + t-fold ;; defined + t-for-each ;; defined + t-insert ;; defined + t-delete ;; defined + t-delete-min ;; defined + t-empty? ;; defined + + ;; These syntax is used expand to code for comparision + ;; expressions. + t-k-eq? ;; key<>node-key "equal" + t-eq? ;; node-key<>node-key "equal" + t-k-node-key "less then" + t-node "less then" + + ;; set-left!, set-right! and set-color! are unused, obsolete and + ;; will be removed in future version. + left set-left! + right set-right! + color set-color! + ) + (begin + (define-syntax if/pure + (syntax-rules (pure) + ((_ kt kf) (if/pure features kt kf)) + ((_ () kt kf) kf) + ((_ (pure . more) kt kf) kt) + ((_ (kw . more) kt kf) (if/pure more kt kf + + (define-syntax if/ordered + (syntax-rules (ordered) + ((_ kt kf) (if/ordered features kt kf)) + ((_ () kt kf) kf) + ((_ (ordered . more) kt kf) kt) + ((_ (kw . more) kt kf) (if/ordered more kt kf + + (define-syntax if/leftmost + (syntax-rules (leftmost) + ((_ kt kf) (if/leftmost features kt kf)) + ((_ () kt kf) kf) + ((_ (leftmost . more) kt kf) kt) + ((_ (kw . more) kt kf) (if/leftmost more kt kf + + (define-syntax cond-define + (syntax-rules () + ((_ (#f . params) . body) #f) + ((_ (id . params) . body) + (define (id
[Chicken-hackers] PATCH Re: slow polling
The "Betthupferl"
Am 19.02.2016 um 18:46 schrieb Jörg F. Wittenberger:
> Am 19.02.2016 um 14:02 schrieb Jörg F. Wittenberger:
...
> I opened ticket 1259 for this.
>
> To make the kind reviewers job easier, I'll post diffs in piecemeal here.
A "Betthupferl" is Bavarian (a German dialect spoken in some remote,
hilly areas) for the last, small mean given to the kids upon bedtime.
Contradictory to all teachings often a sweet.
This patch is not supposed to do any harm.
It refactors parts of the code to minimize the upcoming diffs.
Furthermore it basically takes srfi-18 out of the equation. Only
whitespace/comment diffs left there.
PS: passes all relevant tests so far.
Cheers
/Jörg
> Details to follow below.
>
> Cheers
>
> /Jörg
>
> To follow the related changes:
>
> 1. Apply
>
> http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00032.html
>
> this should just fix a bug
>
> 2. Apply
>
> http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00033.html
>
> 3. Apply the attached patch.
>
> This only adds comments: Section headers large enough for "meld" at
> least to pick up the difference. (To both the original and master.)
> You want this to make head or tail of the upcoming changes. Eventually
> there will have to be a cleanup diff to get rid of them.
>
>
From 3f2306aaec36a6c7b7dca49e15bca03baf73a7fb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?=
Date: Fri, 19 Feb 2016 22:26:49 +0100
Subject: [PATCH] Some refactoring.
* some predicates whose scetchy implementation may soon change given decriptive names
* `unblock-threads-for-timeout!` moved out of `schedule` - the former handles the timeout queue, the latter the scheduling policy
* `thread-unblock!` to become the "clean unblock" procedure
* `##sys#thread-clear-blocking-state!` to handle all blocking reasons at a central spot
---
scheduler.scm | 180 +-
srfi-18.scm | 20 ++-
2 files changed, 110 insertions(+), 90 deletions(-)
diff --git a/scheduler.scm b/scheduler.scm
index 245f400..feaac28 100644
--- a/scheduler.scm
+++ b/scheduler.scm
@@ -32,7 +32,7 @@
##sys#update-thread-state-buffer ##sys#restore-thread-state-buffer
pending-queue ##sys#unblock-threads-for-i/o
fdset-set fdset-test create-fdset stderr
- ##sys#clear-i/o-state-for-thread! ##sys#abandon-mutexes)
+ ##sys#thread-clear-blocking-state! ##sys#abandon-mutexes)
(not inline ##sys#interrupt-hook)
(unsafe)
(foreign-declare #< " tmo2 " (now: " now ")")
- (if (not (equal? tmo1 tmo2)) (dbg "chicken would loose timeout " tmo1 " since " tto " has " tmo2))
- (if (equal? tmo1 tmo2) ;XXX why do we check this?
- (if (fp>= now tmo1) ; timeout reached?
- (begin
- (##sys#setislot tto 13 #t) ; mark as being unblocked by timeout
- (##sys#clear-i/o-state-for-thread! tto)
- (##sys#thread-basic-unblock! tto)
- (loop (cdr lst)) )
- (set! ##sys#timeout-list lst) )
- (loop (cdr lst)) ) ) ) )
- (if (and (null? ##sys#fd-list) (ready-queue-empty?))
- (if (null? ##sys#timeout-list)
- (##sys#signal-hook #:runtime-error "deadlock")
- ;; Sleep for the number of milliseconds of next thread
- ;; to wake up.
- (let ((tmo1 (caar ##sys#timeout-list)))
- (##core#inline
- "C_msleep"
- (fxmax
- 0
- (##core#inline "C_quickflonumtruncate" (fp- tmo1 now )
+ (unless (##sys#timeout-list-empty?)
+ (##sys#unblock-threads-for-timeout!))
(update-pending-queue!)
@@ -314,6 +304,42 @@ dunno what to do
(set! ##sys#timeout-list r))
(loop r l))
+(define-inline (##sys#thread-basic-unblock! t)
+ (dbg "unblocking: " t)
+ (##sys#add-to-ready-queue t) )
+
+(define (##sys#unblock-threads-for-timeout!)
+ (dbg "timeout queue " ##sys#timeout-list)
+ (let ((now (##core#inline_allocate ("C_a_i_current_milliseconds" 4) #f)))
+(let loop ((lst ##sys#timeout-list))
+ (if (null? lst)
+ (set! ##sys#timeout-list '())
+ (let* ([tmo1 (caar lst)] ; timeout of thread on list
+ [tto (cdar lst)] ; thread on list
+ [tmo2 (##sys#slot tto 4)] ) ; timeout value stored in thread
+ (dbg "timeout: " tto " -> " tmo2 " (now: " now ")")
+ (if (not (equal? tmo1 tmo2)) (dbg "chicken would loose timeout " tmo1 " since " tto " has " tmo2))
+ (if (equal? tmo1 tmo2) ;XXX why do we check this?
+ (if (fp>= now tmo1) ; timeout reached?
+ (begin
+ (##sys#setislot tto 13 #t) ; mark as being unblocked by timeout
+ (##sys#thread-clear-blocking-state! tto)
+ (##sys#thread-basic-unblock! tto)
+
Re: [Chicken-hackers] slow polling
Am 19.02.2016 um 14:02 schrieb Jörg F. Wittenberger: > Now I see the low load case to be almost 50% faster and the old code to > be 30% faster and more stable on delivering responses. I have no idea > what the combined effect would be. > > How do we deal with this? I opened ticket 1259 for this. To make the kind reviewers job easier, I'll post diffs in piecemeal here. Details to follow below. Cheers /Jörg To follow the related changes: 1. Apply http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00032.html this should just fix a bug 2. Apply http://lists.nongnu.org/archive/html/chicken-hackers/2016-02/msg00033.html 3. Apply the attached patch. This only adds comments: Section headers large enough for "meld" at least to pick up the difference. (To both the original and master.) You want this to make head or tail of the upcoming changes. Eventually there will have to be a cleanup diff to get rid of them. From 124d4cb348b84da560389d357a363c706af5f5c6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?= Date: Fri, 19 Feb 2016 18:33:56 +0100 Subject: [PATCH] add section markers large enough for "meld" to pick them up --- scheduler.scm | 17 + srfi-18.scm | 13 +++-- 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/scheduler.scm b/scheduler.scm index fa1c2ab..245f400 100644 --- a/scheduler.scm +++ b/scheduler.scm @@ -144,6 +144,23 @@ EOF (syntax-rules () ((_ . _) #f))) +;;; BEGIN NEW SECTION (Integrating old scheduler) +#| +we +need "diff" to find this worth to be kept as is + +dunno what to do + +|# +;;; END NEW SECTION (Integrating old scheduler) + +#| +we +need "diff" to find this worth to be kept as is + +dunno what to do + +|# (define-syntax panic (syntax-rules () ((_ msg) (##core#inline "C_halt" msg diff --git a/srfi-18.scm b/srfi-18.scm index 09888ff..5111531 100644 --- a/srfi-18.scm +++ b/srfi-18.scm @@ -279,7 +279,7 @@ (when (##sys#slot mutex 4) ; abandoned (return (##sys#signal (##sys#make-structure 'condition '(abandoned-mutex-exception) (list (##sys#slot mutex 1) ) ) (define (assign) - (##sys#setislot ct 11 #f) + (##sys#setislot ct 11 #f) ;;; FIXME: still required? (check) (if (and threadsup (not thread)) (begin @@ -307,6 +307,7 @@ ct 1 (lambda () (if (##sys#slot ct 13) ; unblocked by timeout + ;;; FIXME: still required? (begin (##sys#setslot mutex 3 (##sys#delq ct (##sys#slot mutex 3))) (##sys#setislot ct 11 #f) @@ -339,7 +340,7 @@ (let ((t (##sys#slot mutex 2))) (when t (##sys#setislot mutex 2 #f) - (##sys#setslot t 8 (##sys#delq mutex (##sys#slot t 8) ; unown from owner + (##sys#setslot t 8 (##sys#delq mutex (##sys#slot t 8) (when cvar (##sys#setslot cvar 2 (##sys#append (##sys#slot cvar 2) (##sys#list ct))) (##sys#setslot ct 11 cvar) ; block object @@ -369,7 +370,7 @@ (##sys#setslot wt 11 #f) (##sys#add-to-ready-queue wt)) (else - (##sys#error 'mutex-unlock "Internal scheduler error: unknown thread state: " + (##sys#error 'mutex-unlock! "Internal scheduler error: unknown thread state: " wt wts))) ) ) (if (eq? (##sys#slot ct 3) 'running) (return #t) @@ -411,7 +412,7 @@ [t0s (##sys#slot t0 3)] ) (##sys#setslot cvar 2 (##sys#slot ts 1)) (when (or (eq? t0s 'blocked) (eq? t0s 'sleeping)) - (##sys#thread-basic-unblock! t0) ) ) ) ) ) + (##sys#thread-basic-unblock! t0) ) ) ) ) ) ;; TBD (define (condition-variable-broadcast! cvar) (##sys#check-structure cvar 'condition-variable 'condition-variable-broadcast!) @@ -420,14 +421,14 @@ (lambda (ti) (let ([tis (##sys#slot ti 3)]) (when (or (eq? tis 'blocked) (eq? tis 'sleeping)) - (##sys#thread-basic-unblock! ti) ) ) ) + (##sys#thread-basic-unblock! ti) ) ) ) ;; TBD (##sys#slot cvar 2) ) (##sys#setislot cvar 2 '()) ) ;;; Change continuation of thread to signal an exception: -(define (thread-signal! thread exn) +(define (thread-signal! thread exn) ;; TBD (##sys#check-structure thread 'thread 'thread-signal!) (dbg "signal " thread exn) (if (eq? thread ##sys#current-thread) -- 2.6.2 ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
[Chicken-hackers] slow polling
Hi folks, I guess I found the reason, why my 4.9.1 based chicken is about 30% faster in practice while master performs better on all little load and unit benchmarking. Observing that my version has a rather small deviation from the median, while master has extremely fast and extremely low values, this has either to come from some "rare", but extraordinary expensive operation. Turns out to be a regression. When I submitted the polling code, it was a little complex, that's true. But it was carefully crafted and well tested. It went to the trouble to keep the fd-list in synch with the poll/select set. For the only purpose to be sure the poll(2) ends up in practice to be just that, call poll. When it was integrated into core, it was rewritten to used this - sorry, but IMHO brain dead - approach to re-create the fd set from the fd list each time each time we check. Now I see the low load case to be almost 50% faster and the old code to be 30% faster and more stable on delivering responses. I have no idea what the combined effect would be. How do we deal with this? I don't want to keep maintaining my own "fast chicken" branch. But I can't convince myself to switch to master. Waiting almost 3 seconds for my browsers start page was already bordering on pain. Switching 4-5 is no option. There are other users too. I'd like to get this into core. It's working for years. But it's going to be a huge change. Basically most of scheduler and some code in srfi-18 is affected. I suppose: not before 4.11 - but who's going to review the patch? (I did not yet prepare it. It's just the question what I'm whether I'm working on "my branch" and ignore whatever or attempt to produce the smallest diff to ease the review. The latter being more work and only helpful within a small enough time window.) Cheers /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Argvector handling - maybe we could do better at that
Am 19.02.2016 um 13:43 schrieb Peter Bex: > On Fri, Feb 19, 2016 at 11:35:01AM +0100, Jörg F. Wittenberger wrote: >> So better the other way around: get rid of the globals. >> >> One thing I did not yet try: maybe it's beneficial to have a version of >> C_kontinue which receives the current av, its size and the c argument to >> enable av-reuse without accessing those globals. > > That won't work because the continuation will lose the vector's size. Depends on what the alternave C_kontinue would do... > >> What tried was a length-prefixed av. Just C_alloc one more word, stick >> the length in and return av+1. Thus getting rid of the global size. > > That was my idea too. I went for the low hanging fruit first. >> Works till gc. :-/ > > I think that's because it's copied onto the temp stack, and then copied > back onto the stack. In the restart you'll need to prepend the av with > the size too. YES. But that's the question: where the hell is the restart? I can't find it in the source. If you could point me there, I'd be glad. > >> For reference I attach my current modifications. (Including Peters >> fixes to location-ref to avoid a patch conflict.) > > I won't look into this until we've released 4.11, it's too much > distraction, and generates too much noise on the mailinglist. > There are other fixes we need to apply first before considering > such large changes. Sure, this was just for reference in case someone had a comment. I'm *not* suggesting this to go anywhere! 5-10% are no huge gain and I'm still positive that we still can do a little better at least. Cheery /Jörg ___ Chicken-hackers mailing list Chicken-hackers@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-hackers
Re: [Chicken-hackers] Argvector handling - maybe we could do better at that
Am 17.02.2016 um 17:36 schrieb Jörg F. Wittenberger:
> Am 17.02.2016 um 13:22 schrieb Jörg F. Wittenberger:
>> Am 16.02.2016 um 22:54 schrieb Peter Bex:
>>> How about the stack-allocated vector I proposed? It'd also be the
>>> smallest change from what we currently have, I think.
The past days I've tried a few modifications towards this end.
Short story: there is something to gain. But little it is. In the 5%
range when modifying hash tables and almost 10% more when modifying a
binary tree. Lookups gain nothing so far.
Long story: I failed on the next logical step and need help.
So far I maintain _two_ more globals, the currently re-usable argvector
and it's allocated length.
Initially I only checked the required argvector length (the "avl" value
in c-backend.scm) against the allocated length (in the hope that the
compiler might catch more cases where the "c" parameter is unused except
for the av-reuse test and optimize that away) and returned the reusable
av upon success. However it turned out that checking against the actual
arg count "c" first and the global only if that fails was consistently
faster. Easy to understand in a way, cache locality I'd guess.
So better the other way around: get rid of the globals.
One thing I did not yet try: maybe it's beneficial to have a version of
C_kontinue which receives the current av, its size and the c argument to
enable av-reuse without accessing those globals.
What tried was a length-prefixed av. Just C_alloc one more word, stick
the length in and return av+1. Thus getting rid of the global size.
Works till gc. :-/
Looks like I'm loosing the length tag. Sure I do. I wanted to
re-install it after gc (which invalidates the reusable last av anyway).
Help I need to find the spot where the stuff saved in
C_save_and_reclaim is being restored.
For reference I attach my current modifications. (Including Peters
fixes to location-ref to avoid a patch conflict.)
Hints much appreciated.
Cheers
/Jörg
diff --git a/c-backend.scm b/c-backend.scm
index d61b59a..aed69e6 100644
--- a/c-backend.scm
+++ b/c-backend.scm
@@ -483,13 +483,8 @@
;; CPS context, so callee never returns to current function.
;; And even so, av[] is already copied into temporaries.
(cond (caller-has-av?
- (gen #t "C_word *av2;")
- (gen #t "if(c >= " avl ") {")
- (gen #t " av2=av; /* Re-use our own argvector */")
- (gen #t "} else {")
- (gen #t " av2=C_alloc(" avl ");")
- (gen #t "}"))
- (else (gen #t "C_word av2[" avl "];")))
+ (gen #t "C_word *av2 = C_allocate_argvector(C_current_argvector_size, c, av, " avl ");"))
+ (else (gen #t "C_word *av2 = C_allocate_fresh_argvector(" avl ");")))
(when selfarg (gen #t "av2[0]=" selfarg ";"))
(do ((j (if selfarg 1 0) (add1 j))
(args args (cdr args)))
diff --git a/chicken.h b/chicken.h
index 5a41f73..6b22eaf 100644
--- a/chicken.h
+++ b/chicken.h
@@ -1004,6 +1004,24 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_heaptop ((C_word **)(&C_fromspace_top))
#define C_drop(n) (C_temporary_stack += (n))
#define C_alloc(n) ((C_word *)C_alloca((n) * sizeof(C_word)))
+
+#if 0 // set to "true" for backward compatible version to boot modified scheme
+#define C_argvector_reuse_dflt(n) 0 // never, it does not exist
+#define C_allocate_fresh_argvector(n) C_alloc(n)
+#define C_allocate_argvector(as, c, av, avl) ( (c >= avl) ? av : C_force_allocate_fresh_argvector(avl))
+#else
+#define C_MIN_AV_SIZE 4
+#define C_argvector_reuse_dflt(n) (C_current_argvector_size >= (n))
+#define C_argvector_flush()((C_current_argvector_size = 0), (C_default_argvector_value = NULL))
+//gone #define C_allocate_fresh_argvector(n) (((n) <= C_MIN_AV_SIZE) ? ((C_current_argvector_size = (n)), C_default_argvector_value = C_alloc(n)) : (C_argvector_flush(), C_alloc(n)))
+//gone #define C_force_allocate_fresh_argvector(n) ((C_current_argvector_size = ((n) > C_MIN_AV_SIZE ? (n) : C_MIN_AV_SIZE)), C_default_argvector_value = C_alloc(C_current_argvector_size))
+#define C_force_allocate_fresh_argvector(n) ((C_current_argvector_size = (n)), C_default_argvector_value = C_alloc(C_current_argvector_size))
+#define C_allocate_fresh_argvector(avl) (C_argvector_reuse_dflt(avl) ? C_default_argvector_value : C_force_allocate_fresh_argvector(avl))
+//gone #define C_allocate_argvector(as, c, av, avl) ( ((avl <= C_MIN_AV_SIZE) && (av == C_default_argvector_value)) ? av : (c >= avl) ? av : C_allocate_fresh_argvector(avl))
+// good #define C_allocate_argvector(as, c, av, avl) ((as) >= (avl) ? av : C_force_allocate_fresh_argvector(avl))
+#define C_allocate_argvector(as, c
Re: [Chicken-hackers] Argvector handling - maybe we could do better at that
Am 17.02.2016 um 13:22 schrieb Jörg F. Wittenberger:
> Am 16.02.2016 um 22:54 schrieb Peter Bex:
>> How about the stack-allocated vector I proposed? It'd also be the
>> smallest change from what we currently have, I think.
>
> How about trying the following. This should make it easy to experiment.
The attached patch appears to not do any harm at all.
All it does is reserve a macro
C_allocate_argvector(as, c, av, avl)
wherein "as" is a reserved parameter
and use it back in c-backend.scm.
I hesitate to propose this to be included into 4.11. At the other hand
it looks innocent enough and may provide an easier reference point once
we need to tell somebody which is the last known version to boot chicken
from source.
Best
/Jörg
From 3e040b681d5d918854f20ae3c0f09074e7e1d59c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=B6rg=20F=2E=20Wittenberger?=
Date: Wed, 17 Feb 2016 17:24:11 +0100
Subject: [PATCH] Provide C_allocate_argvector
---
c-backend.scm | 8 ++--
chicken.h | 3 +++
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/c-backend.scm b/c-backend.scm
index d61b59a..26f7876 100644
--- a/c-backend.scm
+++ b/c-backend.scm
@@ -483,12 +483,8 @@
;; CPS context, so callee never returns to current function.
;; And even so, av[] is already copied into temporaries.
(cond (caller-has-av?
- (gen #t "C_word *av2;")
- (gen #t "if(c >= " avl ") {")
- (gen #t " av2=av; /* Re-use our own argvector */")
- (gen #t "} else {")
- (gen #t " av2=C_alloc(" avl ");")
- (gen #t "}"))
+ (gen #t "C_word *av2 = C_allocate_argvector(C_current_argvector_size, c, av, " avl ");"))
+ ;; TBD: Check if we could do better in this case too:
(else (gen #t "C_word av2[" avl "];")))
(when selfarg (gen #t "av2[0]=" selfarg ";"))
(do ((j (if selfarg 1 0) (add1 j))
diff --git a/chicken.h b/chicken.h
index 5a41f73..db8d8b6 100644
--- a/chicken.h
+++ b/chicken.h
@@ -1004,6 +1004,9 @@ typedef void (C_ccall *C_proc)(C_word, C_word *) C_noret;
#define C_heaptop ((C_word **)(&C_fromspace_top))
#define C_drop(n) (C_temporary_stack += (n))
#define C_alloc(n) ((C_word *)C_alloca((n) * sizeof(C_word)))
+#define C_current_argvector_size 0 /* reserved */
+#define C_default_argvector_value 0 /* reserved */
+#define C_allocate_argvector(as, c, av, avl) ( (c) >= (avl) ? av : C_alloc(avl))
#if defined (__llvm__) && defined (__GNUC__)
# if defined (__i386__)
# define C_stack_pointer ({C_word *sp; __asm__ __volatile__("movl %%esp,%0":"=r"(sp):);sp;})
--
2.6.2
___
Chicken-hackers mailing list
Chicken-hackers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-hackers
