> Only philosophy I know is that it is plain text. Is there any official > philosophy? I have no idea, at least manual does not give me > references. I cannot find "philosophy", send me references.
You are right. There is no official "philosophy" in org. In my reply I tried to follow the topic starter's view: Texas Cyberthal <texas.cybert...@gmail.com>: > By philosophy, I mean the dev consensus on the correct way to do > things, and coded configuration and usability biases. According to my experience with org-mode development (I am not talking about third-party packages here), it is discouraged to change org-mode towards hiding metadata in org files or store *unique* data (that cannot be derived from the contents of the org files) related to org-mode externally (not in org files). It is not official statement, but rather my impression so far. > Question is what is meant by database, it can be anything. One can > save LISP data. Recent files, desktop, eww bookmarks, init.el or > .emacs file are also all similar databases, there is the underused > EIEIO with persistent stuff that represent built-in database > functionality. Org-mode does not limit user customisations. It does not limit third-party packages either. Users are free to use any other tools, store their data anywhere other than org files (why would org force users to do otherwise?). What I referred to earlier is just the core org-mode development. > And people try to do exactly that, people are developing Org in the > manner to relate objects in Org file to anything else. And they do > hard work as they do it manually. Relational database speeds up and > does not tell user to manually hyperlink various relations. Could you provide some more examples? I do not see how relational database is different from creating hyperlinks in org. Either way, the user needs to file an object/headline somewhere into org file, which is inherently manual. > I see hard work by many people who try to enhance Org as hierarchical > knowledge of data because people want to have feature X, but group of > those enhancements in reality belong to relational databases and not > to text files. It is an interesting point. I would be happy if some existing tools could be reused instead of re-inventing the wheel for org. Do you have concrete examples where it can be useful? If you have, I encourage you to bring up a feature request to discuss this possibility with org-mode devs. Best, Ihor Jean Louis <bugs@gnu.support> writes: > * Ihor Radchenko <yanta...@gmail.com> [2020-11-23 17:18]: > :PROPERTIES: > :CREATED: [2020-11-23 Mon 18:42] > :ID: edebb3e7-e755-4ecc-a5e8-a3353a3f5fd0 > :END: >> Dear Jean Louis, >> >> Your description of the database reminds me how org-roam handles the >> files - it also uses an external database for linking and allows quick >> incremental search that does not really depend on where the >> files/headings are stored. > > Sounds good, I can see there is graph database used. > >> However, what you are talking about is against org-mode philosophy, >> as I know it. > > Only philosophy I know is that it is plain text. Is there any official > philosophy? I have no idea, at least manual does not give me > references. I cannot find "philosophy", send me references. > > It says "to keep simple things simple". But Org is far far from being > simple any more. It offers good principles, paradigms and people built > many enhancements upon those. Speedily it becomes way much more than > simple. > > Headings do not look any more as headings, it looks like pieces of > code to a person that is new. Properties, tags, clocks, schedule, > deadline, all that tries to store itself under specific heading. There > is easily too much of it and things are not simple any more. > >> Currently, the dev's stance is that org files must be >> self-sufficient. > > There is no compact principle there practically. Anything is > possible. That Org files are not practically self-sufficient shows the > fact that there are 129 Emacs packages in one Org distribution. > >> Org-mode should not depend on external database to manage the org >> files and operations with them. Everything must be plain text! > > Question is what is meant by database, it can be anything. One can > save LISP data. Recent files, desktop, eww bookmarks, init.el or > .emacs file are also all similar databases, there is the underused > EIEIO with persistent stuff that represent built-in database > functionality. > > That Org files are not self-sufficient shows the demand that there is > almost no Org user who does not have add-ons, packages, modifications, > configurations. > > Would it be really self-sufficient there would be no development going > on, right? > > Babel executions clearly show that Org is not self sufficient and > depends on number of external software. > > I don't mind of philosophy, in fact I would like that philosophy is > really that what it wanted to be, but that time is over. > > I am just pointing out that it is by many means not self sufficient. > > Is by default LaTeX export enabled? I think it is. How big is the > LaTeX package? It is huge, and Org depends on it for export. > > Thus Org is far far from being self-sufficient. > > Almost every system has GDBM database, if Emacs would have bindings to > GDBM, there would be so much less of development in general for many > various packages and Emacs would be expanding faster and easier. > > In fact I think that author and initial developers could not predict > at the time where the Org goes and that speaking of self-sufficient > and "plain text" only is history. > > Before I found out about Org I was using back in time `hnb' in console > to track various planning tasks. It works nice and simple. That is > really self sufficient. Org definitely not. > > HNB - Hierarchical Notebook > http://hnb.sourceforge.net/ > > In the mean time I have created database where I can store TODO, > Notes, Calls, SMS, People, Invoices, Groups, Mailing Lists and so on, > and made my own shell and Perl interfaces to it. And I used to manage > it through: GeDaFe - PostgreSQL Generic Database Interface > http://gedafe.github.io/doc/gedafe-sql.en.html and this was and is > hierarchical or better graph knowledge management and relational > database. > > Creating simple table in the database automatically helped me to > manage that table. It is trivial to create NOTES table with schedule > date, clock in, TODO or other conditions and tags. The interface is > just there and automatic to whatever table I create the interface is > there to add/modify/delete/search/refine records. That is what I would > say "simple" and keeping things simple and indefinitely extensible > without modification of software. The fundamental GeDaFe principle I > would like to try to implement in Emacs. And same database I use for > web interface I am using also within Emacs. > > GeDaFe principle is following: > > - define your data (like handling notes, TODO, or executing scripts within > notes) > > - work and forget about any underlying functions, > add/create/delete/modify/index or search, make reports with simple exports > > - expand with more definitions of data when necessary (like add > various properties, or other data tables, like contacts, invoices, > etc.) and repeat the process. > > Org also shows that it does not hold "Notes" only, it holds more than > that, I have written average book size technical documents with > it. Only just one part of the document is printed from one single node > that belongs to single project among many. People use such documents > on the ground. My use case is not for simple notes. > > A node in a tree can be anything, and Org enhancements develop by that > principle. For example there is org-contacts where nodes are meant to > be "People". Such development is so much hard coded. > > Simpler would be to define the type of nodes and work by their > types. One could need just one type table and one node table and that > is about all. > > The type table could say: > > - this node is heading > - or, this node is text under heading > - or, this node is paragraph among many others > - or, this node is hyperlink to WWW URI > - or, this node is hyperlink to file > - this node is movie to play > - this type is PDF to be opened on page 3 > - this one is really note > - this one is note but with action assigned like TODO, etc. > > Nodes could have its properties defined like for anything. Nodes can > reference its parent nodes. Node can be parent to any heading. > > Once such 2 tables are defined magic starts to take place, it becomes > its subtree with all kinds of node types including Babel execution and > similar. Meta data is meta, it can be updated but need not be > visible. Computer is handling meta data. > > Node can be a page in a subtree that becomes a website. > > Node can be object for person or company, or just anything. > > I am currently using my nodes to quickly research various subjects by > using that type of dynamic knowledge repository. > > Org file is dynamic knowledge repository. > > About Dynamic Knowledge Repositories (DKR) > https://www.dougengelbart.org/content/view/190/163/ > > Then around 2015 I have discovered Cherrytree and have made bunch of > notes with it, and that is self-sufficient one program that keeps > simple things simple as it is much more simpler than Org. It offers a > visual interface to all features related to the knowledge management > > Cherrytree - hierarchical note taking application with rich text and syntax > highlighting > https://www.giuspen.com/cherrytree/ > > TAGS in Cherrytree are automatic if I remember well, TAG is simply > name of the node. If I call node TODO, then nodes under are simply > TODO items. > > Later I discovered there is something similar in Emacs so I started > with Org and use it mostly for instruction writing and project > management. And I find many options over kill for me. On the other > hand my usage of Org would be overkill for somebody, so it depends of > viewpoints. > > In general I was always using headings and subheadings, trees, lists, notes > by using text. > > Somewhere 2004 I started using Markdown one among first as I found it > simpler than ASCIIDOC and M4, but not as perfect. > > 2020 and 2021 I like to raise the level of dynamic knowledge > journaling to the meta level where I think less about underlying > functions in software. > > That experience also tells that Org is definitely not simple. > > Among hnb, GeDaFe database approach, Cherrytree and Org mode, for > "keeping things simple" like note taking and TODO items, project > management, Cherrytree was the best for me. > > Among all those for keeping complex things simple the database > approach is the best. Of course that I use database within Emacs and > it is not visible to user what it is. Database allows simultaneous > operation by several people on distance and that is the groupware > feature as envisioned by Doug Engelbart. > > For document writing and nice formatting with LaTeX export, Org mode > is the best personally. > > For notes, database based notes are best as only so I have relations > between the note and other objects. With 200,000 contacts in the > database which I can quickly access and assign notes to them, how > would that work with Org? Hardly. Notes are related to people, to > projects, plans, opportunities, research subjects, companies and so > on. There is no simple way in Org mode to relate one note to multiple > other related subjects. > > And people try to do exactly that, people are developing Org in the > manner to relate objects in Org file to anything else. And they do > hard work as they do it manually. Relational database speeds up and > does not tell user to manually hyperlink various relations. > > I have sent thousands of tasks to people by using this function > below. And I had to define for each Org file who are "assignees" for > that Org file. And I have like 50 assignees, so I need to copy and > paste their nick names or identifiers into the Org file. There it > comes the attribute of being "self-sufficient", it becomes > self-sufficient because I had to define all assignees into that > specific file, but I find it tedious and not useful. > > (defun rcd/org-send-assigned-task () > "Sends assigned task to designated individual as Org" > (interactive) > (let* ((member-data (rcd-org-extract-assigned-member-email-data)) > (id (if member-data (first member-data) nil)) > (signature (if (equal (type-of (symbol-value (fifth member-data))) > 'cons) > (third (symbol-value (fifth member-data))) "")) > (file (rcd-org-subtree-to-file signature)) > (subject (rcd/org-find-headline)) > (esubject (escape-% subject)) > (ask (unless id (y-or-n-p "No assignment found. Do you want to send it > by email?"))) > (name (if id (third member-data))) > ;; (name (if ask (read-from-minibuffer "Name:") name)) > (voice (format "The task '%s' is being sent to '%s'" subject name)) > (email (if id (if (equal (type-of (fourth member-data)) 'cons) > (car (fourth member-data)) > (fourth member-data)))) > (email (if ask (cf-search-email (read-from-minibuffer "Search for > email: ")) email)) > (really (y-or-n-p (format "Do you really want to send it to: %s?" (if > ask email name))))) > (if (and really (or id ask)) > (if (string-match "@" email) > (progn > ;; (message (escape-% subject)) > (speak voice) > (mutt-send-file name email esubject file)) > (message "No email specified")) > (message "Aborted sending.")))) > >> Moreover, some devs are even reluctant to hide metadata (like unique >> ID implemented in org-id module) from users (which is possible and >> also partially implemented). > > I hope that I have demonstrated that one who develops could review > several various paradigms and learn more. I am fine with any decision > of design for Org mode as I use it as it is and I have for me other > ways of managing data. I am not sure how much those features have been > discussed to say that hiding meta data is good or not good. It is > better to discuss and find what is more useful. > > What I can see is that people complain for speed and they build > extensions that help with it. Extensions are external while built-in > Org does not keep up with the dynamic how people expect it to be. > > For example I would expect Org to be very simple, very very simple, I > would rewind it back to its roots. Somebody else would like > complexities. Currently I can see that Org is not made for > complexities. It is better to unwind the development and make Org in > core very basic and speedy and let people enhance it with external > packages. In general it is better to remain simple. Even that may not > be possible any more. > > I see hard work by many people who try to enhance Org as hierarchical > knowledge of data because people want to have feature X, but group of > those enhancements in reality belong to relational databases and not > to text files. Developers wish to accommodate various people and yet > by doing so they develop it into complex software. (129 .el packages > for one Emacs mode!?) > > Among those one shall read the Doug Engelbart's paradigms, then > especially if one is developer of system of data management that many > people use, one shall explore other paradigms, including various > approaches to data handling. And overall one shall keep it simple as > it is main fundament of Org to be simple, while practical fact remains > that it is not anymore simple, not at all. > > I remember back in time with BASIC programming language that it had > also something like a built-in database that one could put on the > bottom of the file. Then there is also Perl's __DATA__ that is placed > straight into the file and one could also place images and other > stuff. Maybe the meta data could be kept this way in just one heading > named Meta data, and this heading would not be exported, it could be > hidden from the user and it could contain the database necessary for > single Org file. By pressing key to show properties one could see > properties or simply make them hidden. By making a copy of subtree the > metadata would also copy as usual. By exporting, one could make Org > without meta data, and I like using this information as I like sending > Org headings without personal meta data to third parties.