Re: [sqlite] Limit on number of columns in SQLite table

2019-10-16 Thread Mitar
g regular structure back to the user. Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table

2019-10-16 Thread Mitar
Hi! On Wed, Oct 16, 2019 at 3:16 PM Hick Gunter wrote: > 100k distinct column names? Or is that 1 repeats of 10 attributes? 100k distinct names. Like each column a different gene expression. Mitar -- http://mitar.tnode.com/ https://twitter.com/mita

[sqlite] Limit on number of columns in SQLite table

2019-10-16 Thread Mitar
reasons why this limit is so low (even if we extend it during compiling, it can be at most 32k columns), while others are comfortably large? Any plans to extend this limit in the future? Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m

Re: [sqlite] Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
the main use case would be sub-sampling rows. Mitar On Thu, Oct 17, 2019 at 4:11 PM Donald Griggs wrote: > > So if character-separated values (CSV-ish) were originally your preferred > import format, would using that format for the blob's work for you? > > E.g., Suppose you need to inde

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
d decoding cells in over multiple rows? Mitar On Thu, Oct 17, 2019 at 3:38 PM Hick Gunter wrote: > > I have the impression that you still do not grasp the folly of a 100k column > schema. > > See the example below, which only has 6 fields. As you can see, each field > requires a

Re: [sqlite] Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
t modifying to original data too much. I do hear suggestions to do such transformation, but that is less ideal for our use case. Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.

Re: [sqlite] Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
embed, that approach would be useful. Like composite value types. Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
n keep the existing 2000 limit, but to support it without having to recompile, people could increase it all the way to 2 billion. Is there any significant performance downside to this? [1] https://www.sqlite.org/whentouse.html Mitar On Wed, Oct 16, 2019 at 8:21 PM Richard Hipp wrote: > > SQLite coul

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
Hi! In that case we would have to define a standard BLOB storage format, slightly defeating the idea of using SQLite to define such standard future-proof format. :-) Mitar On Thu, Oct 17, 2019 at 11:19 AM Hick Gunter wrote: > > Since your data is at least mostly opaque in the

Re: [sqlite] Limit on number of columns in SQLite table

2019-10-17 Thread Mitar
ised that it says "Many bioinformatics researchers use SQLite in this way." With limit on 2000 columns this is a very strange claim. I would love to see a reference here and see how they do that. I might learn something new. Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m ___