Re: Graph database
Thank you beneroth and Joh-Tob for this impressive and insightful explanation, very informative for me as well, thank you, I will put this on my PicoLisp notes. On Thu, Mar 26, 2020 at 8:29 AM Joh-Tob Schäg wrote: > Have you already looked at the family example? > > Here is my brief overview of ascending order of abstractedness: > PicoLisp had no graph database. What is has is this: > - The ability to serialize/de-serialize all structures in the heap > (Lists, numbers, functions, symbols etc) > - It has the ability to fetch and deserialize structures when they > are accessed by their reference (also called external symbol) and also > GC the loaded content when it is no longer accessable > - the ability to make atomic changes to such structures (by write > locking the database). > - it has relations which allow the database to place pointers in one > or multiple directions, make space for indexes etc... (I have not > fully grasped that myself) > - The are indexes which automatically maintained and those are > accessible to the database allowing you to discover objects placed in > the database. > - It has an mechanism which allows you to maintain database > consistent of different machines (replication) > > In PicoLisp you can solve many graph database problems with those > structures, however PicoLisp does not care or know what a vertex is. > It does not come support for weighted vertexes, you could build your > own class but is usually more efficient to just implement what you > actually need, not implement some paradigm because you heard of > it/like it. > > Tailor PicoLisp to your problem, not your paradigm. This is a thing i > had to learn over a long time. > > On Wed, 25 Mar 2020 at 21:46, Lawrence Bottorff wrote: > > > > I'm afraid at my level of CS theory I don't really know what is meant by > a picolisp atom being persistent, much less across distributed picolisp > instances. Could someone give me a concrete example of what you describe > as: "Any named bag of items automatically represents a (directed, > undirected) graph. The name then is the node, the items in the bag then > there represent the edges." I do understand the tree structure of a lisp > program. But that doesn't make it a graph database. When I tried to fathom > the Picolisp "graph database" example, I was quickly confused. The GUI > actually added confusion, AFAIC. I'm guessing from what I could ausknobeln > from example that the Picolisp version of a CLOS object is a vertex, and > the inheritance of that object from other (higher, more general?) objects > is a sort of edge. Correct me if I'm wrong. But then there was talk of > "records." Is creating a record the same as creating an object instance -- > and this record/object is a vertex? Where, what are the edges? > > > > Don't get me wrong, I have long felt that Lisp -- with its parsing > actually visible in the code you write -- is or could be very > graphDB-friendly; however, Lisp is functional, i.e., you write functions. > And even though they are set up as a graph-like tree in nested lists form, > they are not in themselves data in the traditional sense, rather, code > meant to take you from a domain/input to a range/result. This is not a > "record" (or graph vertex?) creation/query/deletion paradigm. > > > > But this relates to a long-standing question I've had about software > libraries. As it stands, they may be auto-indexed for our viewing pleasure, > but they aren't in any real database form so that you might simply have > your program "query-and-plug-in" a library. (Although I've heard Haskell's > hlint almost writes your code for you!) No, you have to find the module, > plug it in yourself. The whole "code is data", therefore, doesn't seem to > get past the higher-order function trick of passing in a function as an > argument. What more is there to "code is data?" In Fortran the data was in > fact parked just below the code. > > > > At some point I'm just scared and rambling on > > > > On Wed, Mar 25, 2020 at 7:12 AM Guido Stepken > wrote: > >> > >> Lawrence, you haven't yet understood, that any Lisp, by default, is > it's own Graph Database. Especially Picolisp, where Alex has made any > Picolisp Atom persistent and even distributed across other Picolisp > instances. 'Data is code, code is data'. > >> > >> Any named bag of items automatically represents a (directed, > undirected) graph. The name then is the node, the items in the bag then > there represent the edges. Even Picolisp sources you can consider a > (di
Re: Graph database
Have you already looked at the family example? Here is my brief overview of ascending order of abstractedness: PicoLisp had no graph database. What is has is this: - The ability to serialize/de-serialize all structures in the heap (Lists, numbers, functions, symbols etc) - It has the ability to fetch and deserialize structures when they are accessed by their reference (also called external symbol) and also GC the loaded content when it is no longer accessable - the ability to make atomic changes to such structures (by write locking the database). - it has relations which allow the database to place pointers in one or multiple directions, make space for indexes etc... (I have not fully grasped that myself) - The are indexes which automatically maintained and those are accessible to the database allowing you to discover objects placed in the database. - It has an mechanism which allows you to maintain database consistent of different machines (replication) In PicoLisp you can solve many graph database problems with those structures, however PicoLisp does not care or know what a vertex is. It does not come support for weighted vertexes, you could build your own class but is usually more efficient to just implement what you actually need, not implement some paradigm because you heard of it/like it. Tailor PicoLisp to your problem, not your paradigm. This is a thing i had to learn over a long time. On Wed, 25 Mar 2020 at 21:46, Lawrence Bottorff wrote: > > I'm afraid at my level of CS theory I don't really know what is meant by a > picolisp atom being persistent, much less across distributed picolisp > instances. Could someone give me a concrete example of what you describe as: > "Any named bag of items automatically represents a (directed, undirected) > graph. The name then is the node, the items in the bag then there represent > the edges." I do understand the tree structure of a lisp program. But that > doesn't make it a graph database. When I tried to fathom the Picolisp "graph > database" example, I was quickly confused. The GUI actually added confusion, > AFAIC. I'm guessing from what I could ausknobeln from example that the > Picolisp version of a CLOS object is a vertex, and the inheritance of that > object from other (higher, more general?) objects is a sort of edge. Correct > me if I'm wrong. But then there was talk of "records." Is creating a record > the same as creating an object instance -- and this record/object is a > vertex? Where, what are the edges? > > Don't get me wrong, I have long felt that Lisp -- with its parsing actually > visible in the code you write -- is or could be very graphDB-friendly; > however, Lisp is functional, i.e., you write functions. And even though they > are set up as a graph-like tree in nested lists form, they are not in > themselves data in the traditional sense, rather, code meant to take you from > a domain/input to a range/result. This is not a "record" (or graph vertex?) > creation/query/deletion paradigm. > > But this relates to a long-standing question I've had about software > libraries. As it stands, they may be auto-indexed for our viewing pleasure, > but they aren't in any real database form so that you might simply have your > program "query-and-plug-in" a library. (Although I've heard Haskell's hlint > almost writes your code for you!) No, you have to find the module, plug it in > yourself. The whole "code is data", therefore, doesn't seem to get past the > higher-order function trick of passing in a function as an argument. What > more is there to "code is data?" In Fortran the data was in fact parked just > below the code. > > At some point I'm just scared and rambling on > > On Wed, Mar 25, 2020 at 7:12 AM Guido Stepken wrote: >> >> Lawrence, you haven't yet understood, that any Lisp, by default, is it's own >> Graph Database. Especially Picolisp, where Alex has made any Picolisp Atom >> persistent and even distributed across other Picolisp instances. 'Data is >> code, code is data'. >> >> Any named bag of items automatically represents a (directed, undirected) >> graph. The name then is the node, the items in the bag then there represent >> the edges. Even Picolisp sources you can consider a (directed) graph, often >> also called 'syntax tree'. >> >> If you like, you can put, group all "edges" with same properties into a new, >> searchable bag of edges for fast lookup. Since it's all lazy evaluated (even >> the persistent nodes), as Alex already pointed out, it's still ultra fast. >> And since in Picolisp everything can be
Re: Graph database
Dear Lawrence Sounds to me that your head got stuffed a bit too well with over-complicated concepts. No offense! That is the nature of most software education, and its even worse in the business world. And we programmers have a high tendency to believe we are more clever when we are working on more complex systems:"Weeks of coding can save you hours of planning." PicoLisp is radical in its focus on simplicity, although it is not an easy language. Less so if one is trained by education and other languages to think in more complicated patterns than really necessary. I went through this struggles too, and it made me into an overall much better software developer, also in other languages & stacks. Some fundamentals: * PicoLisp has not much in common with CommonLisp, though they share common ancestors * As with lisp languages in general: the source code is not a list of instructions, but an abstract syntax tree (AST) o compilers (in the mainstream sense) for other programming languages construct internally an AST from source code, and then they optimize that AST and translate it to machine code o for PicoLisp, the reader (R of REPL) translates source code (which can also be repl text input) into an AST, basically trees of pointers o this AST /is/ nothing else than a data structure, the source code (textual data) becomes a tree in RAM (lists of lists) o PicoLisp can modify this AST (the code data structure) during execution - primarily by interpret it as code or data, both is the exact same stuff in memory (more so in PicoLisp than some other lisps) * PicoLisp is a multi paradigm language o it is not purely functional (like e.g. Haskell) o operations can be immutable or mutable * In Lisp (*l**is*t *p*rocessor) languages are based on lists (everything with parentheses around it is a list) o A list is always a grouping of (possibly) multiple elements o Elements which are not lists themselves are called *atoms* (single things) * PicoLisp has only 3 fundamental data types: o Number + signed integer of arbitrary size + this is an atom (atomic type) o Lists + singly linked list (so always an ordered sequence of elements, not a unordered set) + may contain lists and atoms o Symbol + has a name + has a value + may have an arbitrary number of properties # a property consists of a name and a value + this is an atom (atomic type) o a /value/ always is of one of these 3 data types + other than in other languages, variables don't have a data type (the value has a data type) + variables in PicoLisp are just symbols + the data type of an value is both static and strong (cannot be changed) o all other data types (e.g. classes / objects) are based on these 3 fundamental PicoLisp types + non-fundamental types have practically no enforcement, no checks - unless explicitly called # not entirely true, the fundamental data types have built-in sub-variants for which certain rules apply (e.g. transient symbols, primarily used as string type) + so all non-fundamental types are *duck typed* <https://en.wikipedia.org/wiki/Duck_typing> + this enables easier code reuse o for example: the OOP system in PicoLisp is primarily based on the symbol data type + classes and objects are symbols which follow certain principles # member variables (attributes) are stored as properties of the symbol # the value of the symbol is a list containing parent classes and methods * PicoLisp database mechanisms is multi paradigm o *on lowest level: a /key-value store/* + just values of the symbol data type which are persisted to disk + the so-called *external symbols* - a sub-variant of the fundamental symbol data type + the name of such an external symbol is the logical block address of the data within the database file(s) + external symbols are automatically loaded into RAM on first access (lazy loading) o *external symbols combined with PicoLisp OOP system: /object database/* + persistent OOP objects + no translation/copying between objects in RAM and database (no ORM problem) + as the objects are just external symbols which follow certain principles, they're lazy loaded when accessed o *relationships between database classes (entities) defined in a schema: /graph database/* + not exactly following the strict academic definitions
Re: Graph database
I'm afraid at my level of CS theory I don't really know what is meant by a picolisp atom being persistent, much less across distributed picolisp instances. Could someone give me a concrete example of what you describe as: "Any named bag of items automatically represents a (directed, undirected) graph. The name then is the node, the items in the bag then there represent the edges." I do understand the tree structure of a lisp program. But that doesn't make it a graph database. When I tried to fathom the Picolisp "graph database" example, I was quickly confused. The GUI actually added confusion, AFAIC. I'm guessing from what I could ausknobeln from example that the Picolisp version of a CLOS object is a vertex, and the inheritance of that object from other (higher, more general?) objects is a sort of edge. Correct me if I'm wrong. But then there was talk of "records." Is creating a record the same as creating an object instance -- and this record/object is a vertex? Where, what are the edges? Don't get me wrong, I have long felt that Lisp -- with its parsing actually visible in the code you write -- is or could be very graphDB-friendly; however, Lisp is functional, i.e., you write functions. And even though they are set up as a graph-like tree in nested lists form, they are not in themselves data in the traditional sense, rather, code meant to take you from a domain/input to a range/result. This is not a "record" (or graph vertex?) creation/query/deletion paradigm. But this relates to a long-standing question I've had about software libraries. As it stands, they may be auto-indexed for our viewing pleasure, but they aren't in any real database form so that you might simply have your program "query-and-plug-in" a library. (Although I've heard Haskell's hlint almost writes your code for you!) No, you have to find the module, plug it in yourself. The whole "code is data", therefore, doesn't seem to get past the higher-order function trick of passing in a function as an argument. What more is there to "code is data?" In Fortran the data was in fact parked just below the code. At some point I'm just scared and rambling on On Wed, Mar 25, 2020 at 7:12 AM Guido Stepken wrote: > Lawrence, you haven't yet understood, that any Lisp, by default, is it's > own Graph Database. Especially Picolisp, where Alex has made any Picolisp > Atom persistent and even distributed across other Picolisp instances. 'Data > is code, code is data'. > > Any named bag of items automatically represents a (directed, undirected) > graph. The name then is the node, the items in the bag then there represent > the edges. Even Picolisp sources you can consider a (directed) graph, often > also called 'syntax tree'. > > If you like, you can put, group all "edges" with same properties into a > new, searchable bag of edges for fast lookup. Since it's all lazy evaluated > (even the persistent nodes), as Alex already pointed out, it's still ultra > fast. And since in Picolisp everything can be persisted distributed, > Picolisp automatically represents a distributed graph database (with > sharding and everything) which you can build, implement on your own with > just a few lines of code. It's a no-brainer! > > Picolisp is a genius strike, but most people can't see the forest for all > the trees. > > Have fun! > > Regards, Guido Stepken > > P.S. Keep away from Windows and other viruses! > > Am Donnerstag, 12. März 2020 schrieb Lawrence Bottorff >: > >> I take it the picolisp graph database follows more the Neo4j property >> graph idea than any RDF/OWL triples, correct? That seems obvious, but I >> thought I'd check. I haven't dived in deep, buy you seem to use Lisp >> objects to create a vertex. But then what are the edges? Again, I'm just >> getting started. >> >> LB >> Grand Marais, MN, Oberer See >> >
Re: Graph database
Lawrence, you haven't yet understood, that any Lisp, by default, is it's own Graph Database. Especially Picolisp, where Alex has made any Picolisp Atom persistent and even distributed across other Picolisp instances. 'Data is code, code is data'. Any named bag of items automatically represents a (directed, undirected) graph. The name then is the node, the items in the bag then there represent the edges. Even Picolisp sources you can consider a (directed) graph, often also called 'syntax tree'. If you like, you can put, group all "edges" with same properties into a new, searchable bag of edges for fast lookup. Since it's all lazy evaluated (even the persistent nodes), as Alex already pointed out, it's still ultra fast. And since in Picolisp everything can be persisted distributed, Picolisp automatically represents a distributed graph database (with sharding and everything) which you can build, implement on your own with just a few lines of code. It's a no-brainer! Picolisp is a genius strike, but most people can't see the forest for all the trees. Have fun! Regards, Guido Stepken P.S. Keep away from Windows and other viruses! Am Donnerstag, 12. März 2020 schrieb Lawrence Bottorff : > I take it the picolisp graph database follows more the Neo4j property > graph idea than any RDF/OWL triples, correct? That seems obvious, but I > thought I'd check. I haven't dived in deep, buy you seem to use Lisp > objects to create a vertex. But then what are the edges? Again, I'm just > getting started. > > LB > Grand Marais, MN, Oberer See >
Re: Graph database
On Fri, Mar 13, 2020 at 10:22:12PM +0100, Alexander Burger wrote: > Perhaps this helps? https://software-lab.de/doc/tut.html#ext > > It is really very simple. Understanding PicoLisp symbols is perhaps the important point. https://software-lab.de/doc/ref.html#symbol The rest is just making symbols persistent. ☺/ A!ex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Graph database
On Fri, Mar 13, 2020 at 04:02:44PM -0500, Lawrence Bottorff wrote: > Could you point me to a beginner's treatment of this topic, especially an > example of a graph database, and what exactly a picolisp pointer is? Perhaps this helps? https://software-lab.de/doc/tut.html#ext It is really very simple. > I'm afraid I don't even know what you mean by a pointer in this context. I > know from C what a pointer is, but a picolisp pointer is beyond my > comprehension. Exactly the same as C pointers. The 32-bit version of PicoLisp *is* even written in C. The external symbols *point* to each other when loaded inte memory, via properties or arbitrary Lisp structures. And they are loaded *into* memory on demand, implicitly when they are accessed (lazy loading). The higher DB levels then implement entity and relation classes, B-Trees etc., all on top of external symbols. ☺/ A!ex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Graph database
Could you point me to a beginner's treatment of this topic, especially an example of a graph database, and what exactly a picolisp pointer is? I'm afraid I don't even know what you mean by a pointer in this context. I know from C what a pointer is, but a picolisp pointer is beyond my comprehension. On Fri, Mar 13, 2020 at 3:28 PM Alexander Burger wrote: > Hi Lawrence, > > > I take it the picolisp graph database follows more the Neo4j property > graph > > idea than any RDF/OWL triples, correct? That seems obvious, but I thought > > I'd check. I haven't dived in deep, buy you seem to use Lisp objects to > > create a vertex. But then what are the edges? Again, I'm just getting > > started. > > Sorry, I know absolutely nothing about Neo4j or RDF/OWL, but the edges are > in > fact trivial. > > As everything in PicoLisp is a pointer (except short numbers), the edges > are > indeed just pointers to other symbols. All those symbols in a DB (called > "external symbols") are loaded lazily on demand. > > ☺/ A!ex > > > > > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe >
Re: Graph database
Hi Lawrence, > I take it the picolisp graph database follows more the Neo4j property graph > idea than any RDF/OWL triples, correct? That seems obvious, but I thought > I'd check. I haven't dived in deep, buy you seem to use Lisp objects to > create a vertex. But then what are the edges? Again, I'm just getting > started. Sorry, I know absolutely nothing about Neo4j or RDF/OWL, but the edges are in fact trivial. As everything in PicoLisp is a pointer (except short numbers), the edges are indeed just pointers to other symbols. All those symbols in a DB (called "external symbols") are loaded lazily on demand. ☺/ A!ex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Graph database
I take it the picolisp graph database follows more the Neo4j property graph idea than any RDF/OWL triples, correct? That seems obvious, but I thought I'd check. I haven't dived in deep, buy you seem to use Lisp objects to create a vertex. But then what are the edges? Again, I'm just getting started. LB Grand Marais, MN, Oberer See