Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Dan Bishop
Michael Chen wrote: > --this is my first version for the purpose of storing sparse numerical > matrix in sql > --please let me know how to fix the bug at the end of the file, and how to > tune the performance > --or any better reference, thanks! > > .explain-- return

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Michael Chen
yes, that's what I am thinking of too. This big table is in charge of store all matrix, keep track of all index changes, and rollback when needed. I will only extract a tiny part from this big table in format like a sparse matrix, and put it in C array, then the available numerical routines, such a

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Stephan Wehner
On Fri, Oct 16, 2009 at 1:41 PM, Michael Chen wrote: > rdbms is indeed not a place for store a single sparse matrix like this. > However I have hundreds of them, and I need to break them and recombine them > frequently; furthermore, I need to drop a few rows or columns successively, > and need to

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Michael Chen
rdbms is indeed not a place for store a single sparse matrix like this. However I have hundreds of them, and I need to break them and recombine them frequently; furthermore, I need to drop a few rows or columns successively, and need to be able to trace back what's a row's original index. I think s

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Michael Chen
thanks Pavel ! On Fri, Oct 16, 2009 at 1:24 PM, Pavel Ivanov wrote: > > -- IA = [ 1 3 5 7 ] // IA(i) = Index of the first nonzero element > of > > row i in A > > Why 4th element if A has only 3 rows? > > > create temp view rowwiseC as > > select a1.rowid, sum(a2.ct) +1 as JA > > from rowwi

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Pavel Ivanov
> -- IA = [ 1 3 5 7 ] // IA(i) = Index of the first nonzero element of > row i in A Why 4th element if A has only 3 rows? > create temp view rowwiseC as > select a1.rowid, sum(a2.ct) +1 as JA > from rowwiseB a1, rowwiseB a2 > where a2.rowid < a1.rowid > group by a1.rowid > ; > --this is not

Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread P Kishor
This is not the answer you are looking for, and there are SQL geniuses on this list who will help you better, but really, is an rdbms really a good place to store a matrix the way you are trying to do? So convoluted. My approach, if I really was determined to store it in sqlite, would be to flatte

Re: [sqlite] Sparse matrix

2007-08-25 Thread
Hi All, Simon answered: Here's a less gruesome version - no cases. I've given no thought to performance comparisons. Thanks for the two great solutions you posted. Upon further investigation, those solutions assume that we want all like occurrences together, effectively sorting records i

Re: [sqlite] Sparse matrix

2007-08-22 Thread Darren Duncan
At 1:54 PM +1000 8/23/07, T&B wrote: Hi Darren, It seems to me that you have a flawed design. Displaying sparse like that should be a function of your application display code, not the database I had to chuckle that when I asked "How do I use this to do that", your solution was "you shouldn'

Re: [sqlite] Sparse matrix

2007-08-22 Thread
Hi Darren, It seems to me that you have a flawed design. Displaying sparse like that should be a function of your application display code, not the database I had to chuckle that when I asked "How do I use this to do that", your solution was "you shouldn't have that and you should do it

Re: [sqlite] Sparse matrix

2007-08-22 Thread
Hi Simon, Here's a less gruesome version - no cases. I've given no thought to performance comparisons. Thanks for the two great solutions you posted. They certainly achieve the desired result with the simplified sample I gave. However, in the broader reality, it doesn't quite satisfy my s

Re: [sqlite] Sparse matrix

2007-08-22 Thread Simon Davies
Hi Tom, Here's a less gruesome version - no cases. I've given no thought to performance comparisons. C:\Joinerysoft\JMS\dev\trunk> sqlite3 tst.db SQLite version 3.4.0 Enter ".help" for instructions sqlite> insert into List values( 'a' ); sqlite> insert into List values( 'a' ); sqlite> insert int

Re: [sqlite] Sparse matrix

2007-08-22 Thread Simon Davies
Hi Tom, Its a pretty gruesome bit of sql... C:\Joinerysoft\JMS\dev\trunk> sqlite3 tst.db SQLite version 3.4.0 Enter ".help" for instructions sqlite> create table List( Code text ); sqlite> insert into List values( 'a' ); sqlite> insert into List values( 'a' ); sqlite> insert into List values( 'a

Re: [sqlite] Sparse matrix

2007-08-21 Thread Darren Duncan
It seems to me that you have a flawed design. You should just have a 2 column database to begin with, with a table like this: Code Count a 4 b 2 c 3 Rather than plain inserting or deleting rows, just sometimes insert or delete, you should instead increment or decrement cou