Re: best practices query: non-emacs packages based on tangled source
Immanuel, my take > - Not possible to tangle all code going to a specified file i'm not sure what this means. > - Not possible to add line directives without major surgery yes, that would be a problem (... if i were programming in, e.g., C, etc., so *is* for other people). > - Not all language modes do the correct thing my *sense* here is that the level of standardization across different babel languages in terms of which header arguments are supported, as well as their meanings, is moderate. one solution to this (which seems to have happened, at least to an extent) is good per-language documentation of which header arguments are supported, and what they mean within the context of the particular language. harder would be agreement and implementation of some "core set" of header arguments/semantics on which one could count, followed by a list of per-langugage extensions, or followed by places where "we agree to disagree", or some such? anyway, a lot of work. > - No way to prevent overwriting an unchanged file yes, i too have a hack i use for this (and, it would be nice if there were something 'out of the box' that did this). > - Special casing e.g. Don=E2=80=99t tangle to a file called =E2=80=98no=E2= > =80=99. somewhere, some how, fonts were lost. :) cheers, Greg
Re: best practices query: non-emacs packages based on tangled source
I think the org way of tangling just doesn't work well for compiled languages.
Here are some reasons:
- Not possible to tangle all code going to a specified file
- Not possible to add line directives without major surgery
- Not all language modes do the correct thing
- No way to prevent overwriting an unchanged file
- Special casing e.g. Don’t tangle to a file called ‘no’.
Specific gripe:
- :prologue and :epilogue are not honoured by all modes
>>>
#+ BEGIN_SRC emacs-lisp :prologue "hello" :epilogue "goodbye" :tangle ex1.el
(+ 1 2)
#+ END_SRC
#+ BEGIN_SRC haskell :tangle ex1.hs :prologue "hello" :epilogue "goodbye"
hey
#+ END_SRC
hey
#+HEADER: :tangle ex1.cpp
#+ BEGIN_SRC cpp :prologue "hello" :epilogue "goodbye"
main()
{}
#+ END_SRC
#+ BEGIN_SRC bash :tangle ex1.bash :prologue "hello" :epilogue "goodbye"
aha
#+ END_SRC
>>>
Based on the org machinery I've written a simple tangler which addresses some
of these shortcomings. It can add line directives to recursively expanded noweb
references -- and get that right. It can also take any action at the
end of tangling
like comparing your tangled file to the one on disk and not overwrite
if it's the same,
or just sending your tangled buffer to an interpreter not writing it
to file (remember that
error references point back to the .org file). Also it can tangle all
blocks going to the
same file. This has been tested for Haskell, C++ and elisp.
It's in a private repo on github because it's not ready for release
yet, but if someone
is interested I can give them access.
Immanuel
On Tue, Oct 27, 2020 at 11:30 PM Dr. Arne Babenhauserheide
wrote:
>
>
> TRS-80 writes:
>
> > Therefore, any stuff I plan on releasing publicly, I do not do in
> > literate style (JMHO). However if you are dead set on doing literate
> > paradigm, then maybe my experience is invalid for your use-case.
>
> My experience is that literate style works very well for tutorials, but
> whenever I developed a larger program within org-mode, I tangled it once
> I had to start debugging and then kept it as separate source.
>
> Once the code becomes complex, I want to stay in one programming
> language without outside features that also add extra complexity.
>
> That said, for simple code the additional freedom of multi-language
> programming in org-mode is awesome. I’m using that to create random
> roleplaying characters directly from random tables in org-mode, with
> those tables being the primary source (because the book is the most
> important product, not the code).
>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken
--
-- Researching the dual problem of finding the function that has a
given point as fixpoint.
Re: best practices query: non-emacs packages based on tangled source
TRS-80 writes: > Therefore, any stuff I plan on releasing publicly, I do not do in > literate style (JMHO). However if you are dead set on doing literate > paradigm, then maybe my experience is invalid for your use-case. My experience is that literate style works very well for tutorials, but whenever I developed a larger program within org-mode, I tangled it once I had to start debugging and then kept it as separate source. Once the code becomes complex, I want to stay in one programming language without outside features that also add extra complexity. That said, for simple code the additional freedom of multi-language programming in org-mode is awesome. I’m using that to create random roleplaying characters directly from random tables in org-mode, with those tables being the primary source (because the book is the most important product, not the code). Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: best practices query: non-emacs packages based on tangled source
On 2020-10-15 14:11, Greg Minshall wrote: if anyone has any techniques they've used, liked (or hated), i'd love to hear. I am someone who keeps my Emacs config in a literate style in blocks within an Orgmode outline. And I enjoy it! However I somewhat recently came across a project that was developed with a literate paradigm. It was an immediate turn off for me. Because now in addition to learning someone else's code, I have to learn how their brain works, how they organized the file, etc. It just added another layer of complexity on top of something that is already potentially complex (the code itself). Maybe I was just immediately soured because I didn't like their formatting (not leaving enough spaces where I would have, etc.) or how they organized the project. And I probably didn't give it a fair shake if I'm being honest. But I don't have a lot of time, and things like that just create too much friction (IMO) when I was only slightly motivated to make a contribution in the first place. And remember what I said at first, I am someone who appreciates the literate paradigm (at least for my own config). Imagine how someone who does not even know who Knuth is would feel about it. Therefore, any stuff I plan on releasing publicly, I do not do in literate style (JMHO). However if you are dead set on doing literate paradigm, then maybe my experience is invalid for your use-case. Cheers, TRS-80
Re: best practices query: non-emacs packages based on tangled source
Tom, thanks very much for your very detailed analysis and explanation of the issues you've had. it's taken me this long to read through it carefully. the issues with python source blocks [C-c C-c] versus [C-c C-v C-t] are in fact unfortunate, as it does make it hard to develop without lots of tangling. (i've run into that before with Python/org, but without really understanding what was going on.) i mostly write R code these days, and it's pretty happy either way. however, i also write javascript code, and while i do it for front end (so, don't execute it inside of org mode), i suspect it has the same issues python does of conflating file names and modules, and might benefit from some enhancement you suggest about treating source blocks as "things" to be imported into the running babel session (maybe an enhancement of <>?). for me, my ideal will be when the org "language" is documented as an RFC, and people have produced stand alone tanglers, so i can release code, and others, without emacs, without org mode, can edit the "source of truth that is the org file", and run make to tangle and build whatever random software package i'm releasing. again, thank you for the reply. cheers, Greg
Re: best practices query: non-emacs packages based on tangled source
Hi Greg, Great question. This came out a bit longer than I anticipated since I wrote up a couple of relevant workflows. Sync between org source blocks and tangled code is something that I think needs improvement. I have covered the difference in semantics between tangled code and babel evaluated code (along with some others factors) since it can have an impact on what workflows you might choose. Following on Timothy's suggestion, one key thing that I think is needed is the ability to detangle nested and arbitrary code. Detangle of code tangled with :comments noweb is not fully implemented. A full detangling implementation would make it easier and safer to automatically detangle back to the org source blocks using a pre-commit hook or similar. Detangling from the org sources is also something that needs to be implemented for this to work. I think that a more complete detangling implementation could go a long way toward making it easier for those who are not used to org to commit to a project. In the absence of this, I have found that most of my existing workflows actively avoid keeping tangled code and org sources tracked in git at the same time unless absolutely necessary, and even with git there to back me up I have shot myself in the foot tangling over files that I forgot to detangle. Below are a few examples. In all cases I have had to consciously work around the issue of having tangled code that is outside the source of truth that is the org file. As an additional note before the examples, I have found that the trade offs when tangling code also depend on the language you are using. For example, I consider some languages, such as Python, to be obligate tanglers since their semantics conflate modules and files. Org babel might be able to work around this in some cases, but it would mean, for example, that ob-python would have to explicitly compensate for this deficiency by implementing the ability to treat source blocks as modules to be loaded into a session or somehow pulled in during the prologue by pre-parsing blocks to look for import statements, etc. The deficiencies of a language mean that if you want certain functionality for that language then org babel can't just treat the code as text, and might have to go to great lengths to try to keep the semantics of babel evaluation and of tangled code aligned. The workflow that I have found to be the most reasonable I developed while working on an elisp project (it is public but I'm not quite ready to link it on this list). In this workflow I add a hook via ~(add-hook 'before-save-hook #'org-babel-tangle nil t)~ for any org file that should be tangled, that way I don't have to worry about whether I remember to tangle etc. However, there are a couple of issues. It does not work in reverse, you still always have to edit the org blocks. It will become annoyingly slow if you have many blocks to tangle. You really want it to run only when the source blocks change, not whenever the org files changes. Finally, The exact semantics of tangling multiple blocks to the same file can have a major impact on performance. So even this best case is not the greatest and doesn't enable your specific use case (detangle issues in particular are a show stopper). Since this is an elisp project I have to run tests on the tangled file in a separate instance of Emacs to ensure that it works as expected. In this project I also have a completely unreadable file that is valid and executable in 3 languages. Tangled blocks that are easily readable in the org sources are commented out intentionally in the tangled file. This is a worst case for detangling. I suspect that it can be done, but it will push the detangling implementation to the limit. At the moment, there is no way to detangle this file back to readable form at all, and it is not clear that anyone should try to edit the tangled file in the first place. All this to say, if we reason from this extreme example, maybe the best thing is to tangle at the last possible moment, never keep the tangled form under version control etc. Unfortunately the use case for this file is to bootstrap Emacs, which means that in order for it to be useful it _must_ be tangled and put under version control since the systems it needs to run on don't have Emacs. The worst experience I have had was when I was developing python code where I needed to capture the output of the block in order to populate tables. Over time the code grew to the point where it needed to become a library. This is where Python being an obligate tangler reared its head, and the differences in semantics between tangled and evaluated code became a major pain. Combine this with the fact that my testing workflows in Python essentially require me to edit the tangled code for me not to lose my mind, but I would also forget to detangle, and sometimes overwrite on retangle, and I was quite unhappy. As a result I eventually gave up and moved all python development out of the org
Re: best practices query: non-emacs packages based on tangled source
Hi Greg, Just one little thing that occurs to me, for accepting PRs if you add the header arg (globally would probably be best) :comments link That with M-x org-babel-detangle should help with accepting PRs. Hope that helps, Timothy. Greg Minshall writes: hi. i apologize if this has been asked before (especially if by me). but, since i had a question recently about Org Src... buffers, this came up. i'm wondering what people do who want to release a non-emacs'y package (an R package, say, or ...), and who did their development "from within" a .org file. i can "build" whatever files are needed to release the package. but, it's nice to be able to let people look at the sources, maybe submit 'pull requests', etc. if anyone has any techniques they've used, liked (or hated), i'd love to hear. thank you very much, Greg
Re: best practices query: non-emacs packages based on tangled source
Hi Greg, What I do with my Elvish modules (https://github.com/zzamboni/elvish-modules , https://github.com/zzamboni/elvish-completions) is to just include the Org files together with the tangled .elv files. --Diego On Thu, Oct 15, 2020 at 8:28 PM Greg Minshall wrote: > hi. i apologize if this has been asked before (especially if by me). > but, since i had a question recently about Org Src... buffers, this came > up. > > i'm wondering what people do who want to release a non-emacs'y package > (an R package, say, or ...), and who did their development "from within" > a .org file. > > i can "build" whatever files are needed to release the package. but, > it's nice to be able to let people look at the sources, maybe submit > 'pull requests', etc. > > if anyone has any techniques they've used, liked (or hated), i'd love to > hear. > > thank you very much, Greg > >
Re: best practices query: non-emacs packages based on tangled source
On Thursday, 15 Oct 2020 at 21:11, Greg Minshall wrote: > i can "build" whatever files are needed to release the package. but, > it's nice to be able to let people look at the sources, maybe submit > 'pull requests', etc. I have recently done this with a Julia project which I make available at github. All code and documentation is developed in a single org and files tangled etc. to create the github repository for the project. I include the org file in the repository although the HTML that I generate from it (as documentation) is hosted on my work web site for visibility reasons. If you're interested, you can see all the files including the complete org file at https://github.com/ericsfraga/Fresa.jl -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-57-g8402c4
Re: best practices query: non-emacs packages based on tangled source
There is no great answer I am aware of. However, I will sometimes generate a markdown version of the source so that at least non-emacs users have a slightly better chance of being able to view the source in a more format friendly manner than a 'raw' org file. However, pull requests and the like are more likely going to be diffs on the generated sources as most people will want to use their preferred editor and that will likely need the generated source file is in order to get their editor 'IDE' features etc. This is one reason I tend not to use org's 'literate programming' model for anything other than documentation, simple examples, configuration files, sql and basic scripting. I find for more complex development, especially when it requires multiple files, namespaces/modules, long running repl sessions, extensive test suits etc, using org adds another layer of complexity which soon outstrips the benefits of having documentation and source in one file. Of course, this will also depend on the development language/platform. I tend to use languages which involve a fair bit of 'REPL' based development rather than a more traditional write, generate, compile, debug loop. On the other hand, when it comes to documentation, tutorials, configuration files and workflow automation, org is definitely my preferred tool. Tim Greg Minshall writes: > hi. i apologize if this has been asked before (especially if by me). > but, since i had a question recently about Org Src... buffers, this came > up. > > i'm wondering what people do who want to release a non-emacs'y package > (an R package, say, or ...), and who did their development "from within" > a .org file. > > i can "build" whatever files are needed to release the package. but, > it's nice to be able to let people look at the sources, maybe submit > 'pull requests', etc. > > if anyone has any techniques they've used, liked (or hated), i'd love to > hear. > > thank you very much, Greg -- Tim Cross
best practices query: non-emacs packages based on tangled source
hi. i apologize if this has been asked before (especially if by me). but, since i had a question recently about Org Src... buffers, this came up. i'm wondering what people do who want to release a non-emacs'y package (an R package, say, or ...), and who did their development "from within" a .org file. i can "build" whatever files are needed to release the package. but, it's nice to be able to let people look at the sources, maybe submit 'pull requests', etc. if anyone has any techniques they've used, liked (or hated), i'd love to hear. thank you very much, Greg
