it began with camping, Matju had been using Ruby in Gridflow since ages before, so he pointed me to poignant guide and i noticed the announcement on redhanded and tried out
> store them in some sort of indexy thing, where we could use filesystem locks > to keep from writing over eachother, and garbage collect / compress every > now and then. That could work really well, and could be nice pure ruby. > Mmmm. Is this crazy? Am I a nut for thinking that a simple multiprocess safe > key/value store would actually be really easy to do? I've played with the > filesystem as a storage medium a fair bit.. it seems like it should be > almost trivial! Maybe I should make this right now! eventualy i had wiped out all the varying parts with replacements, but it is important to remember Camping provided the scaffolding to get off the ground went 1.9x Ruby because of proper lexical scoping of blocks (mainly) + fast but that broke Markaby..and there was all this code in there with Builder and such and god knows what i was suposed to fix (multipled by metaprogramming tweak-ness) Ruby's Hash/Array connstructors obviated a custom template-language parser or meta-methodery hacks (magic?) http://element.rubyforge.org/git?p=element.git;a=blob;f=ruby/H.rb so sqlite databases being locked by other processes, mysql servers that werent running or had a wrong password (or hardpowerd and required myisamcks). then redland's SWIG wrappers segfaulting ruby with memory errors back to FS. i guess "E" class is sort of a "jquery for a filesystem" sitting at convergence of HTTP URIs, and filesystem paths so i want to read today's email (delivered by getmail, with a 1 line procmailrc rule to put into dirs by date, and cloud-persisted across phones/netbooks with rsync/ceph/nfs) so GET /mail, it goes to thiS: fn '/mail/GET',->e,r{[303,{Location: '/m/'+(Time.now.strftime '%Y/%m/%d')+'/*?'+(r ?r['QUERY_STRING']:'')}]} which constructs today's path, and redirects: GET /m/2010/07/20/*?view=threads there are no 'routes' just a mapping from URI to resourceSet. which includes globbing, 'fragments' of documents (after #), and depth-first traversal (for pagination of large quantities of stuff, or sorted values) so that glob all todays mails, extracts the triples and creates a (Hash) model alive for the request. views are specified in QS, so ?view=threads, you get a basic overview: http://i574.photobucket.com/albums/ss187/ix9/hyper/2010-03-27-051943_1280x800_scrot.png triple sources are functions that yield 3 values, and exist for most of the comon things. so your message, aanlktimtvv0c39kypjyj-ve1uxhgrh1tsc6x3q8g-...@mail.gmail.com has an ID, and URI and the Filesystem cant just store this as is, unless you want 3 million files in a dir. so using sometihng git-like: irb(main):005:0> E('aanlktimtvv0c39kypjyj-ve1uxhgrh1tsc6x3q8g-...@mail.gmail.com').d => "/var/E/ee/dc/QUFOTGtUaW10VlYwQzM5a3lQSllKLXZlMXVYSEdSSDFUc0M2eDNROEctSXBCQG1haWwuZ21haWwuY29t" does its best to use a path similar to the URI, to not nuke everything outright irb(main):006:0> E('http://camping.org').d => "/var/http://camping.org"; in addition to these paths, theres a path of metadata _about_ this path irb(main):007:0> E('http://camping.org').u => #<E:0x000000015ebfd8 @uri="/http://camping.org/<>", @graph=nil> so , in this way, you can create indexed properties: eg, mail references are ugly index paths like: /usr/src/index/<>/http:/rdfs.org/sioc/ns#reference/<>/E/e0/43/MTI3OTYyODYzMi4zMjcxLjEwLmNhbWVsQG1pZGdhcmQ= so when i request a message, provide a query in the QS: fn 'data/thread',->d,_,m{d.walk SIOC+'reference',m} this walks those index paths and constructs the entire thread def walk p,m={},v={} m.merge! memoModel v[uri]=true ((attr p)||[]).concat(((E p).po self)||[]).map{|r| r.E.walk p,m,v if !v[r.uri]} m end ..theres functions to go to/from memory models, lookup FS indexes, and so on, in probably camping-style (ive been told my code is 'obfuscated' anwyas) some other doc @ http://blog.whats-your.name/public/carmen.html creating a 265 message thread including finding all the messages and rendering a view takes about a second on my laptop, which is fine for my needs. you could use the resourceSet X mtimes as a cache key since all data is (convertable to/from) RDF you could go crazy with 4store and SPARQL if you needed more insane indexing options so yeah, let me know what you come up with, im interested in checking it out if a darn OS booted, you have a FS.., _______________________________________________ Camping-list mailing list Camping-list@rubyforge.org http://rubyforge.org/mailman/listinfo/camping-list
