Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On 12 Oct 2017, at 12:09, 73budden . wrote: This tracing tool should help a lot. I believe this tool should be supplied by asdf team. Even I begin to be more positive towards efforts of ASDF team to clean up all the mess that was in ASDF initially, but obviously society is not quite happy with breaking changes, so some small tool with a good manual would make life easier. This will probably not happen for a while -- my work on other aspects, and just staying on top of bugs and testing takes most of the available time. But I think I can provide pieces of a recipe for this kind of debugging and if someone out there could give feedback, I will see to it that the recipe gets into the manual. I think if you want to see the plan that ASDF produces to perform a requested operation, you should use something like: ``` (setf *plan* (asdf/plan:make-plan nil (make-operation 'load-op) (find-system "sysname"))) ``` Then, to figure out what's happening, I would suggest ``` (trace asdf:operation-done-p) ``` ...to see if ASDF is wrong about whether or not it needs to do some operations. Then I would try something like tracing `PERFORM`. I'd have to think a little about what to do if `MAKE-PLAN` gives you a plan you don't expect. cheers, r Printing readtable before loading, I think, requires just a line or two. Dumping log of operations might be one (trace) call, so that's trivial for the person who knows how ASDF is organized. Writing a small two-paragraph addition to manual would relief a lot of pain and stress for all.
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On Thu, Oct 12, 2017 at 6:53 PM, Stelian Ionescuwrote: > On Thu, 2017-10-12 at 13:03 -0500, Robert Goldman wrote: >> That may be, but it was unfair to get angry at the ASDF maintainers >> about this. This is just a pre-existing error that was *manifested* >> because of a change in ASDF. It's not our fault that this error >> appeared, it's not our fault that the puri library is no longer >> maintained, and we can't be expected to avoid releasing improved >> versions of ASDF because there exists buggy, unmaintained code in >> Quicklisp. > > No it's not your fault, but I think it would be a very sensitive idea > to avoid annoying ASDF users, simply for practical reasons, because > you'll find yourself with people who fork or refuse to update ASDF. > Well, in this case, it was a combination of things that are not our fault, and things that are. So my apologies for the things that were my fault. > Something more useful would be to: > * introduce the notion of ASDF compilation options, as a set of key- > value pairs which control different compilation modes or effects ASDF already kind of has that, as keyword arguments to operate. They was a bit of a cleanup in ASDF 3.3.0 already: the arguments are not passed blindly into non-sensical arguments to the operation class anymore, and are now well-defined. Thus they can now be used the way you propose. > * make the new strictness modes, like preserving the readtable, depend > on those toggles, but upon introduction the default should be perfect > backwards-compatibility, even if that is something you consider broken Agreed. Any new strictness should be added cautiously and slowly, disabled by default or adding only a (style-)warning. In this case, the strictness was added by accident (because the forms that prevented the strictness were victim of a refactoring I botched while adding a different feature, and the strictness was only added in a corner case for which I had no test, but just added one now). > * blog about the fancy new way toggle and explain why it's better to > use it than not Will do, if I keep hacking on it. > * let the libraries' users nag the developers to change the code to be > compatible with the new strictness checks Maybe. Historically, the main problem has been unmaintained libraries. > * wait a couple of years (at least) until you see that most of > Quicklisp libraries have been ported to the new way of doing things and > if that happens maybe consider turning it on by default. In that case, > announce it publicly Yes, we've typically waited for a year or two before we added any new strictness (e.g. everything is now UTF-8 by default). > * if adoption didn't happen, keep it disabled happily knowing that you > can always turn it on in your company's internal projects. > This has happened in the past, e.g. for properly catching deferred warnings. Most users never bothered to do it, maintainers never bothered to fix their libraries, etc. Maybe with more outreach they would, but I'm kind of burned out with CL right now. > This is more or less how we do things at work. It has a certain amount > of overhead but it gains you good will from the community; on the other > hand enabling things by default, and on a short notice, only makes you > seem like you're imposing your preference on everybody else just for > the sake of it. I think it's better to let things sink in and allow the > users of ASDF to come to a consensus, although that's a slow process. > I've tried very hard not to do that this time. It was a bug. (Unlike four to five years ago, when I learned the hard way not to push too hard the deferred warning support or the syntax-control branch; the latter just came back and bit me.) —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org Amateur bureaucrats are often even worse than professional bureaucrats. — John McCarthy
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On Thu, 2017-10-12 at 13:03 -0500, Robert Goldman wrote: > That may be, but it was unfair to get angry at the ASDF maintainers > about this. This is just a pre-existing error that was *manifested* > because of a change in ASDF. It's not our fault that this error > appeared, it's not our fault that the puri library is no longer > maintained, and we can't be expected to avoid releasing improved > versions of ASDF because there exists buggy, unmaintained code in > Quicklisp. No it's not your fault, but I think it would be a very sensitive idea to avoid annoying ASDF users, simply for practical reasons, because you'll find yourself with people who fork or refuse to update ASDF. Something more useful would be to: * introduce the notion of ASDF compilation options, as a set of key- value pairs which control different compilation modes or effects * make the new strictness modes, like preserving the readtable, depend on those toggles, but upon introduction the default should be perfect backwards-compatibility, even if that is something you consider broken * blog about the fancy new way toggle and explain why it's better to use it than not * let the libraries' users nag the developers to change the code to be compatible with the new strictness checks * wait a couple of years (at least) until you see that most of Quicklisp libraries have been ported to the new way of doing things and if that happens maybe consider turning it on by default. In that case, announce it publicly * if adoption didn't happen, keep it disabled happily knowing that you can always turn it on in your company's internal projects. This is more or less how we do things at work. It has a certain amount of overhead but it gains you good will from the community; on the other hand enabling things by default, and on a short notice, only makes you seem like you're imposing your preference on everybody else just for the sake of it. I think it's better to let things sink in and allow the users of ASDF to come to a consensus, although that's a slow process. -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. signature.asc Description: This is a digitally signed message part
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On Thu, Oct 12, 2017 at 6:11 PM, Robert Goldmanwrote: >> I will merge this into master, but do not have time to make a new release, >> or even test much this week or next. >> >> So this will simply remain in master for now. > > This patch is now available in ASDF 3.3.0.1 > I also pushed a regression test directly to master. I will also try to salvage the most important stuff to salvage from the syntax-control branch, namely some *shared-readtable* and *shared-print-pprint-dispatch* tables in asdf (or uiop?) to restore around the build (e.g. in asdf:operate) so it is well-defined what readtable is used when building with asdf (i.e. the readtable active when asdf was initially loaded, usually your Lisp's initial readtable). —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org [T]he “evolution stops at the neck” reflex. They're willing to accept that everything else about humans has evolved through evolution but the thing that is most important in explaining your personhood, namely your mind, somehow evolution doesn’t apply to it. — Gad Saad
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On 12 Oct 2017, at 13:08, Robert Goldman wrote: On 12 Oct 2017, at 13:04, Faré wrote: For those in a hurry for a fix, here is the merge request: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85 I will merge this into master, but do not have time to make a new release, or even test much this week or next. So this will simply remain in master for now. This patch is now available in ASDF 3.3.0.1
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
Wow! Good news. 2017-10-12 21:08 GMT+03:00, Robert Goldman: > On 12 Oct 2017, at 13:04, Faré wrote: > >> For those in a hurry for a fix, here is the merge request: >> https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85 > > I will merge this into master, but do not have time to make a new > release, or even test much this week or next. > > So this will simply remain in master for now. > >
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On 12 Oct 2017, at 13:04, Faré wrote: For those in a hurry for a fix, here is the merge request: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85 I will merge this into master, but do not have time to make a new release, or even test much this week or next. So this will simply remain in master for now.
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On Thu, Oct 12, 2017 at 5:09 PM, 73budden .wrote: > Warning about modifying standard readtable is issued by SBCL (at least > in my SBCL 1.3.18), please grep for the message in SBCL sources, and > it seem to be introduced in 1.0.24. I took a look at puri's definition > (maybe old, in my local copy of quicklisp) and it looks like it > actually tries to modify standard readtable. > More precisely, puri modifies the _current_ *readtable*, which is very rude. If anyone maintains puri (fat luck), he should remove this bug and instead offer an interface using named-readtables. The error happens when the current readtable happens to be the standard readtable, which will happen if someone uses with-standard-io-syntax. Looking at ASDF sources, there is one place that uses it, and indeed, a bug was introduced during the refactoring of ASDF 3.3.0: whereas earlier variants of ASDF saved the *readtable* from outside the with-standard-io-syntax to restore it inside, the new ASDF fails to do it. Reduced test case: try to (asdf:load-system "ddop") with the following ddop.asd. (defsystem "ddop" :defsystem-depends-on ("puri")) I believe this justifies a 3.3.1 bugfix release indeed. It also justifies spending more time on the syntax-control branch supposed to cleanup the readtable issues, and/or getting all software fixed so it never modifies the current readtable unless it explicitly makes and sets a new readtable. > It'd be very nice to have instructions on how to trace what happens > while system is being built:that is, which files are compiled, which > are loaded and what is a readtable in the beginning of each load > operation. > You can pass a :verbose t flag to load-system and its friends. As for tracing what readtable are used, there does not exist infrastructure for even identifying readtables. You're welcome to write one and seek assistance in writing a hook in ASDF (e.g. as a :before method on perform compile-op or load-op, which would not have helped you much in this case, or as a local modification to uiop:compile-file*, if you have a local ASDF source checkout). > Running builds in old and new asdf versions and comparing logs it > would be relatively easy to figure out what is wrong. > Only if given enough information to reproduce, which the complainer failed to provide. Luckily, I investigated nevertheless. For those in a hurry for a fix, here is the merge request: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85 —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org The two most common things in the Universe are hydrogen and stupidity. — Harlan Ellison
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
On 12 Oct 2017, at 12:53, Konstanski, Carlos wrote: All signs point at puri being the culprit. I am absolutely not disagreeing with that. I provided the offending code in the bug report myself. The question is: what to do about it? The project is dead. The home page is gone. Here is what I will do: I'll see if I can get drakma to steer clear of it. They will undoubtedly be interested in not having their fine code dragged down by a dead library. The only other option is to fix puri. I don't even know how to begin doing that with a dead project. Where did Kevin Rosenberg disappear off to? 2017-10-12 11:45 GMT-06:00 Robert Goldman <rpgold...@sift.info>: Forwarded message: From: 73budden . <budde...@gmail.com> To: Robert Goldman <rpgold...@sift.info> Cc: Faré <fah...@gmail.com>, ASDF-devel <asdf-devel@common-lisp.net> Subject: Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER Date: Thu, 12 Oct 2017 20:09:44 +0300 Warning about modifying standard readtable is issued by SBCL (at least in my SBCL 1.3.18), please grep for the message in SBCL sources, and it seem to be introduced in 1.0.24. I took a look at puri's definition (maybe old, in my local copy of quicklisp) and it looks like it actually tries to modify standard readtable. But here it does not happen when it is being loaded via my patched asdf 3.1.6 and my own IDE with forked SLIME with many systems pre-loaded. I don't know why and have no time to investigate further. Obviously, my insights would be of limited use because my environment is very different from that of OP. So we might be able to help you debug this It'd be very nice to have instructions on how to trace what happens while system is being built:that is, which files are compiled, which are loaded and what is a readtable in the beginning of each load operation. Running builds in old and new asdf versions and comparing logs it would be relatively easy to figure out what is wrong. I guess that in OP's setup something had changed readtable before, but this does not happen anymore. And obviously many people would face similar problems. This tracing tool should help a lot. I believe this tool should be supplied by asdf team. Even I begin to be more positive towards efforts of ASDF team to clean up all the mess that was in ASDF initially, but obviously society is not quite happy with breaking changes, so some small tool with a good manual would make life easier. Printing readtable before loading, I think, requires just a line or two. Dumping log of operations might be one (trace) call, so that's trivial for the person who knows how ASDF is organized. Writing a small two-paragraph addition to manual would relief a lot of pain and stress for all. WBR, Budden 2017-10-12 18:46 GMT+03:00, Robert Goldman <rpgold...@sift.info>: I'm with Faré on this one. I don't see evidence that this change is because ASDF is doing something bad. I believe it's consistent with the hypothesis that there was some imperfectly-controlled aspect of building that is done differently now (e.g., files loaded in a different order where both orders are compatible with the constraints in the system definitions). This doesn't even look like an ASDF error to me -- it looks like an error that occurred while loading a system that ASDF has re-packaged. So we might be able to help you debug this, but without more evidence, there's no reason to believe that ASDF has done anything to you: it looks like the change in ASDF just reveals a pre-existing bug. -- Carlos Konstanski MTS IV Cslt Verizon Cloud Platform carlos.konstan...@verizonwireless.com Cell: 15126218301 Slack: vzw-vsi.slack.com username: @ckonstanski That may be, but it was unfair to get angry at the ASDF maintainers about this. This is just a pre-existing error that was *manifested* because of a change in ASDF. It's not our fault that this error appeared, it's not our fault that the puri library is no longer maintained, and we can't be expected to avoid releasing improved versions of ASDF because there exists buggy, unmaintained code in Quicklisp. An apology from you would be appropriate at this point.
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
Warning about modifying standard readtable is issued by SBCL (at least in my SBCL 1.3.18), please grep for the message in SBCL sources, and it seem to be introduced in 1.0.24. I took a look at puri's definition (maybe old, in my local copy of quicklisp) and it looks like it actually tries to modify standard readtable. But here it does not happen when it is being loaded via my patched asdf 3.1.6 and my own IDE with forked SLIME with many systems pre-loaded. I don't know why and have no time to investigate further. Obviously, my insights would be of limited use because my environment is very different from that of OP. > So we might be able to help you debug this It'd be very nice to have instructions on how to trace what happens while system is being built:that is, which files are compiled, which are loaded and what is a readtable in the beginning of each load operation. Running builds in old and new asdf versions and comparing logs it would be relatively easy to figure out what is wrong. I guess that in OP's setup something had changed readtable before, but this does not happen anymore. And obviously many people would face similar problems. This tracing tool should help a lot. I believe this tool should be supplied by asdf team. Even I begin to be more positive towards efforts of ASDF team to clean up all the mess that was in ASDF initially, but obviously society is not quite happy with breaking changes, so some small tool with a good manual would make life easier. Printing readtable before loading, I think, requires just a line or two. Dumping log of operations might be one (trace) call, so that's trivial for the person who knows how ASDF is organized. Writing a small two-paragraph addition to manual would relief a lot of pain and stress for all. WBR, Budden 2017-10-12 18:46 GMT+03:00, Robert Goldman: > I'm with Faré on this one. I don't see evidence that this change is > because ASDF is doing something bad. I believe it's consistent with the > hypothesis that there was some imperfectly-controlled aspect of building > that is done differently now (e.g., files loaded in a different order > where both orders are compatible with the constraints in the system > definitions). > > This doesn't even look like an ASDF error to me -- it looks like an > error that occurred while loading a system that ASDF has re-packaged. > > So we might be able to help you debug this, but without more evidence, > there's no reason to believe that ASDF has done anything to you: it > looks like the change in ASDF just reveals a pre-existing bug. > > >
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
Warning about modifying standard readtable is issued by SBCL (at least in my SBCL 1.3.18), please grep for the message in SBCL sources, and it seem to be introduced in 1.0.24. I took a look at puri's definition (maybe old, in my local copy of quicklisp) and it looks like it actually tries to modify standard readtable. But here it does not happen when it is being loaded via my patched asdf 3.1.6 and my own IDE with forked SLIME with many systems pre-loaded. I don't know why and have no time to investigate further. Obviously, my insights would be of limited use because my environment is very different from that of OP. > So we might be able to help you debug this It'd be very nice to have instructions on how to trace what happens while system is being built:that is, which files are compiled, which are loaded and what is a readtable in the beginning of each load operation. Running builds in old and new asdf versions and comparing logs it would be relatively easy to figure out what is wrong. I guess that in OP's setup something had changed readtable before, but this does not happen anymore. And obviously many people would face similar problems. This tracing tool should help a lot. I believe this tool should be supplied by asdf team. Even I begin to be more positive towards efforts of ASDF team to clean up all the mess that was in ASDF initially, but obviously society is not quite happy with breaking changes, so some small tool with a good manual would make life easier. Printing readtable before loading, I think, requires just a line or two. Dumping log of operations might be one (trace) call, so that's trivial for the person who knows how ASDF is organized. Writing a small two-paragraph addition to manual would relief a lot of pain and stress for all. WBR, Budden 2017-10-12 18:46 GMT+03:00, Robert Goldman: > I'm with Faré on this one. I don't see evidence that this change is > because ASDF is doing something bad. I believe it's consistent with the > hypothesis that there was some imperfectly-controlled aspect of building > that is done differently now (e.g., files loaded in a different order > where both orders are compatible with the constraints in the system > definitions). > > This doesn't even look like an ASDF error to me -- it looks like an > error that occurred while loading a system that ASDF has re-packaged. > > So we might be able to help you debug this, but without more evidence, > there's no reason to believe that ASDF has done anything to you: it > looks like the change in ASDF just reveals a pre-existing bug. > > >
Re: [E] new compiler error re: SET-DISPATCH-MACRO-CHARACTER
I'm with Faré on this one. I don't see evidence that this change is because ASDF is doing something bad. I believe it's consistent with the hypothesis that there was some imperfectly-controlled aspect of building that is done differently now (e.g., files loaded in a different order where both orders are compatible with the constraints in the system definitions). This doesn't even look like an ASDF error to me -- it looks like an error that occurred while loading a system that ASDF has re-packaged. So we might be able to help you debug this, but without more evidence, there's no reason to believe that ASDF has done anything to you: it looks like the change in ASDF just reveals a pre-existing bug.