madIS is an extensible relational database system built upon the SQLite database and with extensions written in Python (via APSW SQLite wrapper). Its is developed at:
http://madis.googlecode.com Due to madIS’ SQLite core, the database format of madIS is exactly the same as SQLite’s one. This means that all SQLite database files are directly usable with madIS. In usage, madIS, feels like a lightweight personal Hive+Hadoop programming environment, without the distributed processing capabilities of Hadoop. Nevertheless due to its low overhead while running on a single computer (compared to Hadoop), madIS can easily handle tens of millions of rows on a single desktop/laptop computer. In version 1.3 of madIS: - A great deal of testing has been done on Linux, Windows and Mac OSX. - madIS now only depends on APSW on all these systems, so it is easier to be installed - XMLPARSE was added. XMLPARSE processes its input in a streaming fashion, and has been tested with very large (~20+ GB) XML source files without problems. - JPACK functions were added. Jpacks are now the preferable way to store a set of values into a single tuple. For easy viewing and exporting of the jpacks, their format was based on the JSON format. - Heavy testing under Windows and Mac OSX. CLIPBOARD and CLIPOUT virtual tables work under all OSes. - CLIPOUT and CLIPBOARD, have been tested with Excel, Libre/Open Office Calc, and iWork Numbers. - Functions that return tables, can easily be coded now, by using Python's generators (yield) - A lot of completions (via TAB) have been added to mterm. Mterm's completion engine can automatically complete, tables, column names, table index names and database filenames in attach database. In detail: MTERM changes: While using madIS's terminal (mterm), mterm completes (via TAB) column names, tables names, etc. of the opened and attached databases. Also by default mterm colours column separators and if more than 4 columns are returned, mterm "tags" the columns with numbers: mterm> select * from deficit; [1|1 | People's Republic of China [3|272.500 |2010 [1|2 | Japan [3|166.500 |2010 [1|3 | Germany [3|162.300 |2010 --- Column names --- [1|Rank [2|Country [3|CAB [4|Year Note: In mterm the column number tags are coloured red in above example XMLPARSE: If i wished to retrieve the date and author of madIS project's Source Changes ATOM feed: mterm> select * from (XMLPARSE '<entry><updated>t</updated><author><name>t</name></author></entry>' select * from file('http://code.google.com/feeds/p/madis/hgchanges/basic') ) limit 3; 2011-07-25T14:07:07Z|est...@servum 2011-07-25T14:04:09Z|est...@servum 2011-07-22T14:08:11Z|est...@servum --- Column names --- [1|updated [2|author_name Query executed in 0 min. 0 sec 543 msec mterm> In above query, XMLPARSE is used in an "inverted form" which is easier to write, when chaining virtual tables. JPACKS: Frequently, the need to store multiple values into a tuple arises while processing data. Previously in madIS a lot of formats were used to store all these multiple values (space separated, comma separated, tab separated). Now JPACKs are the recommended way to store multiple values. JPACKs are based on the JSON format, with the exception that a JPACK of a single value is itself. Some examples are presented below: mterm> select jpack('a'); a mterm> select jpack('a', 'b'); ["a","b"] mterm> select jsplit(jpack('a','b','c')); a|b|c mterm> select jsplitv(jpack('a','b','c')); a b c mterm> select jgroup(c1) from (select 'a' as c1 union select 'b'); ["a","b"] CLIPBOARD and CLIPOUT: If a selection of data has been made in the web browser or a spreadsheet program then by executing the following in mterm we could access the clipboard data as a table: mterm> select * from CLIPBOARD(); Rank↓ |Country↓ |CAB (billion US dollars)↓ |Year↓ 1 | People's Republic of China |272.500 |2010 2 | Japan |166.500 |2010 3 | Germany |162.300 |2010 --- Column names --- [1|C1 [2|C2 [3|C3 [4|C4 Query executed in 0 min. 0 sec 204 msec * For above data i selected the top 3 rows while browsing http://en.wikipedia.org/wiki/List_of_sovereign_states_by_current_account_balance . Similarly if i wished to process and send the data to a spreadsheet program then i could execute the following in mterm: mterm> CLIPOUT select * from deficit; 1 --- Column names --- [1|return_value Query executed in 0 min. 0 sec 111 msec and paste in a spreadsheet program. Both CLIPBOARD and CLIPOUT are virtual tables. CLIPOUT is used in an "inverted form" which is easier to write, when chaining queries. -- Lefteris _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
