[web2py] Re: Speed of rendering html data (10,000+ rows)

[Vineet]

Ref. to issue 307 in google code.
http://code.google.com/p/web2py/issues/detail?id=307&sort=-id

As suggested by "Caleb", I prepared the HTML myself (in the
controller).
Then returned the generated html to the View and rendered it there.
==
My finding:---
==
Still it takes long time to load the page (using either of the
following 2 ways).

1) Without XML helper, it displays html code as it is (i.e.
MyNameetc.)
Takes long time to display the page.

2) With XML, it displays properely formatted table.
Takes long time to display the page.

Whether anybody else had a similar situation like the same?
Any suggestions/comments highly appreciated.

---Vineet

On Jun 15, 9:00 pm, VP  wrote:
> > for i in custdata:
> >    mystr2 = ''.join([mystr2, '' , str(i[0]) , ''])
>
> ...
>
> > lst = [ jn(['' , str(i[0]) , '',
> >            '' , str(i[1]) , '',
> >            '' , str(i[2]) , '',
> >         .. more omitted..]) for i in res]
>
> > mystr=jn(lst)
> > --
> > The performance boost is amazing..
> > It took ONLY 0.078 seconds
> > --
>
> This makes sense because a crude analysis shows that the first
> approach is O(n^3) while the second one is O(n).
>
> (I think)

[web2py] Re: Speed of rendering html data (10,000+ rows)

[pbreit]

If you really need to return 10,000 records to the browser, you should be 
returning json and then rending with some sort of data grid.

[web2py] Re: Speed of rendering html data (10,000+ rows)

[Vineet]

@Sebastian, @pbreit,
I understand what you mean to say.
If I get you rightly, I should fetch only a limited no. of rows
through pagination.
It makes sense for most of the situations.

But for some cases, rendering all the records on single page is
required.
e.g. consider a big automobile workshop.
A report for spare parts price list (with >12,000 parts) is to be
viewed.
For the customer, using pagination & clicking "next", "next",... is
unsuitable.
After rendering the total parts on single page, he/she may apply a
desired filter and analyze the data.


On Jun 21, 9:50 pm, "Sebastian E. Ovide" 
wrote:
> why you need to return 10k records in one go ?
>
> --
> Sebastian E. Ovide

[web2py] Re: Speed of rendering html data (10,000+ rows)

[pbreit]

We used YUI DataTables at my last company and they worked really well but we 
only got up to around 1000 records on a page (with very quick paging, 
filtering and sorting). I've been wanting to hook them up to Web2py but 
haven't needed it so far.

http://developer.yahoo.com/yui/datatable/

[web2py] Re: Speed of rendering html data (10,000+ rows)

[Chris May]

Have you tried pasring the data using your 0.078 second method into a
json format and using jQuery to parse it?

[web2py] Re: Speed of rendering html data (10,000+ rows)

[Anthony]

Maybe check out DataTables new virtual scrolling feature: 
http://datatables.net/blog/Introducing_Scroller_-_Virtual_Scrolling_for_DataTables.
 
I think jQgrid may have a similar feature as well.
 
Anthony
 

On Tuesday, June 21, 2011 1:26:50 PM UTC-4, Vineet wrote:

> @Sebastian, @pbreit, 
> I understand what you mean to say. 
> If I get you rightly, I should fetch only a limited no. of rows 
> through pagination. 
> It makes sense for most of the situations. 
>
> But for some cases, rendering all the records on single page is 
> required. 
> e.g. consider a big automobile workshop. 
> A report for spare parts price list (with >12,000 parts) is to be 
> viewed. 
> For the customer, using pagination & clicking "next", "next",... is 
> unsuitable. 
> After rendering the total parts on single page, he/she may apply a 
> desired filter and analyze the data. 
>
>
> On Jun 21, 9:50 pm, "Sebastian E. Ovide"  
> wrote: 
> > why you need to return 10k records in one go ? 
> > 
> > -- 
> > Sebastian E. Ovide

[web2py] Re: Speed of rendering html data (10,000+ rows)

[Vineet]

As suggested by the community, I am trying to export json data to
dataTable.
But only blank dataTable is rendered (with desired formatting, search
box, etc.)
Pl. introspect into my code as to where I have a mistake.
Data is not displayed (as returned by "get_data" method).
I have made sure that the tables in MySQL have been populated.

---
Controller
---
def show_data():
return dict()


def get_data():
custdata = db.executesql(qry, as_dict=True)
return dict(mydata=custdata)

# For testing purpose, I returned response.json(custdata) in a
separate method & validated the output on "jsonlint.com".
It is valid json.
But, here the returned value is dict(mydata=custdata).
How do I reference the custdata object in View?
Already tried {{=mydata}} in  tag in View

---
View (show_data.html)
---
{{extend 'layout.html'}}