On Dec 5, 10:17 pm, Chris <[EMAIL PROTECTED]> wrote: > On Dec 5, 9:35 pm, [EMAIL PROTECTED] wrote: > > > > > I've been using the HyperText module for a while now > > (http://dustman.net/andy/python/HyperText/), and I really like it. > > > I've run into a situation where I have code to construct a table > > and while it is normally perfect, there are times where the table > > can get quite huge (e.g. 1000 columns, 100000 rows .... yes, the > > question of "how on earth would someone render this table?" comes > > up, but that's not the point here :) ), and the code I have generating > > this starts choking and dying from excessive RAM usage. > > > I'm curious if people see a better way of going about this task and/or > > believe that an alternative method of HTML generation here would be > > better. > > > A (possibly somewhat pseudocode, as I'm doing this by hand) small example > > of what I'm doing ... > > > inputs = [A, List, Of, Values, To, Go, Into, A, Table] > > numcolumns = howManyColumnsIWant > > > out = ht.TABLE() > > column = 0 > > for input in inputs: > > if (column == 0): > > tr = ht.TR() > > tr.append(ht.TD(input)) > > column += 1 > > if (column == numcolumns): > > out.append(tr) > > column = 0 > > > As I said, this works fine for normal cases, but I've run into some > > situations > > where I need this to scale not just into the hundreds of thousands but also > > well into the millions - and that's just not happening. Is there a better > > way to do this (which involves direct HTML generation in Python), or am > > I SOL here? > > for (i, input) in enumerate(inputs): > """Your Code > """ > if not i % 1000: > # Flush your data. > > It's logical that you will run out of space as the code just appends > data constantly instead of ever writing it out. How you flush the > data out is up to you or if it's as simple as you have there you could > do something like. > > file_out.write('<TABLE>\n') > for x in xrange(0, len(inputs)//numcolumns): > file_out.write('<TR>\n<TD>%s</TD>\n</TR>' % '</TD> > \n<TD>'.join(inputs[(x*numcolumns):((x+1)*numcolumns)]) ) > if not x % 500: file_out.flush() > file_out.write('<TR>\n<TD>%s</TD>\n</TR>\n</TABLE>' % '</TD> > \n<TD>'.join(inputs[x*numcolumns:]) ) > file_out.close()
Sorry, change the second last line from: > file_out.write('<TR>\n<TD>%s</TD>\n</TR>\n</TABLE>' % > '</TD>\n<TD>'.join(inputs[x*numcolumns:]) ) to: if len(inputs) % numcolumns: file_out.write('<TR>\n<TD>%s</TD>\n</TR>\n</TABLE>' % '</TD> \n<TD>'.join(inputs[x*numcolumns:]) ) else: file_out.write('\n</TABLE>') -- http://mail.python.org/mailman/listinfo/python-list