Begin forwarded message:
> From: Daniel Beatty <danielbea...@mac.com> > Date: December 23, 2010 6:24:03 PM PST > To: Ramsey Gurley <ram...@xeotech.com> > Cc: Daniel Beatty <danielbea...@mac.com> > Subject: Re: Dynamically changing the number of columns included in a D2W > query/listing > > Greetings Ramsey, > Brilliant idea, and it works for the most part. There is another error that > crops up, and I can not seem to make heads or tails of it. It is a bounds > error: > > > IllegalArgumentException: Index (10) out of bounds [0, 9] > at com.webobjects.foundation.NSArray.objectAtIndex(NSArray.java:379) > at > er.extensions.eof.ERXEOControlUtilities.ensureSortOrdering(ERXEOControlUtilities.java:2422) > at > er.extensions.eof.ERXEOControlUtilities.objectsInRange(ERXEOControlUtilities.java:617) > at > er.extensions.batching.ERXBatchingDisplayGroup.objectsInRange(ERXBatchingDisplayGroup.java:368) > at > er.extensions.batching.ERXBatchingDisplayGroup.refetch(ERXBatchingDisplayGroup.java:407) > at > er.extensions.batching.ERXBatchingDisplayGroup.fetch(ERXBatchingDisplayGroup.java:459) > at > er.directtoweb.pages.ERD2WListPage._fetchDisplayGroup(ERD2WListPage.java:452) > at er.directtoweb.pages.ERD2WListPage.setupPhase(ERD2WListPage.java:552) > ... skipped 16 stack elements > > Any ideas? Weird part is that it happens for some of the bloated tables, but > not all. > > I ran a second test with > 160 : *true* => useBatchingDisplayGroup = "true" [BooleanAssignment] > > Some of the tables responded correctly, and with little memory bloat. Others > gave me the same Illegal Argument Exception. It did not crash the WO > program, but it does give a page that is hardly satisfying. In some cases, > it gave the Illegal Argument Exception on the first try, but the second try > it performed as expected. Any ideas? > > Later, > > Dan Beatty, M.S. CS (B.S. EECS) > Ph.D. Student > Texas Tech University > dan.bea...@mac.com > http://web.me.com/danielbeatty/My_Home_Page/Welcome.html > (806)438-6620 > > > > On Dec 22, 2010, at 8:49 AM, Ramsey Gurley wrote: > >> >> On Dec 22, 2010, at 11:09 AM, Daniel Beatty wrote: >> >>> Thank you for responding Ramsey, >>> >>> >>> On Dec 22, 2010, at 6:44 AM, Ramsey Gurley wrote: >>> >>>> >>>> On Dec 22, 2010, at 1:08 AM, Daniel Beatty wrote: >>>> >>>>> Greetings Gentlemen, Chuck, and Davids, >>>>> I am exploring how to limit the number of columns queried and used in for >>>>> a listing in D2W. My experiment uses ERModernLook, and has a the SDSS >>>>> DR1 data set. Several of the tables have an annoyingly large number of >>>>> columns (641 to be precise) and a large number of rows (770k of them). >>>>> If the JVM is tweaked to 3GB of memory, ERModernLook can return a list >>>>> from a blank query (taking nearly 2.5GB memory footprint). I have a >>>>> few more tables with just about as a bad of a scheme and size >>>>> arrangement. It would be nice to use this data set to measure just how >>>>> much abuse D2W can handle. >>>> >>>> It sounds like you're mainly interested in just thrashing the app to see >>>> what it can do under stress. That database table sounds sufficient for >>>> the job. Are you putting all 770K rows on a single page? (^_^) I think >>>> it would be interesting to see it using a "real world" setup with a >>>> batching display group also. That would probably reduce the memory usage >>>> a lot, but that's still a ton of columns to render. >>> >>> For the starting app, I literally reverse engineered the original database >>> from inside a framework and am driving the program from an ERModernLook >>> about as straight out of the box (by your example from WOWODC 2010), and >>> ran it. The next step I took was to increase the memory available to the >>> JVM to 3GB. >>> >>> One thing to be proud of, ERModernLook has a semi-batching display group, >>> but still loads the entire table into memory. >> >> Try >> >> 100 : (entity.name = 'BestTSField' and task = 'list') => >> useBatchingDisplayGroup = "true" [BooleanAssignment] >> >> and see if that changes anything. Note from the ERXBatchingDisplayGroup doc >> (since the javadocs are down today for relocation)... >> >> Extends WODisplayGroup in order to provide real batching. This is done by >> adding database specific code to the select statement from the >> EOFetchSpecification from the WODisplayGroup'sEODataSource which must be an >> EODatabaseDataSource. If used with other datasources, it reverts to the >> default behaviour. >> >> >> >>> >>> I am starting with the one of the large tables. Obviously, the small >>> tables load without flaw. The rule set I have for the large table (so far) >>> includes the following: >>> >>> "60 : (entity.name = 'BestTSField' and task = 'list') => batchSize = 5 >>> [com.webobjects.directtoweb.Assignment]", >>> "60 : (entity.name = 'BestTSField' and task = 'list') => fetchLimit = 5 >>> [com.webobjects.directtoweb.Assignment]", >>> "60 : (entity.name = 'BestTSField' and task = 'list') => >>> displayPropertyKeys = (run,rerun,camcol,field) >>> [com.webobjects.directtoweb.Assignment]", >>> "60 : entity.name = 'BestTSField' => queryDataSourceDelegate = >>> edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate >>> [ERDDelayedObjectCreationAssignment]", >>> >>> I currently have the last rule disabled since >>> "edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate" is not >>> implemented. While these are set the way they are, it does not seem to >>> have an effect on retrieval of this large table. It still basically loads >>> the entire table into memory. >>> >>> Since this is a "real world" database that even Microsoft Research tried >>> improve, it seems to be a worthy challenge to make ERModernLook tackle it. >>> I can podcast the next effort and place it on scorpius.hpcc.ttu.edu. That >>> machine is a Mac Pro running OSX Server 10.6.5 (developer seed) with about >>> 6GB or RAM. I still need to talk the powers that be to open the ports >>> podcast producer use to allow outside access, but I think that they can >>> honor a community of interest. >>> >>> >>> >>>> >>>>> A couple things would be handy for this experiment. >>>>> >>>>> 1. How do I use ERDQueryDataSourceDelegateInterface? It appears that >>>>> the interface requires implementation of the queryDataSource and >>>>> qualifierFromSender methods. >>>> >>>> >>>> Just the queryDataSource method really. In the documentation I put on the >>>> wiki, I included a qualifierFromSender() method since customizing the >>>> qualifier on a query page seemed to be a common request. >>>> >>>> >>>>> 2. Is there a way to specify different query data sources for different >>>>> tasks for the same entity (list versus inspect)? >>>> >>>> I think the datasource is passed from the query page to the list page. I >>>> may be mistaken, I'd have to look to be sure. Every inspect page I've >>>> seen just uses the object(). No datasource involved. >>> So the task would inherently be "query", it would not matter if the result >>> were list or inspect. >>> >>>> >>>>> 3. How do I get more run time logging from the WO application itself? I >>>>> thought Shark could do something with the JVM. Can it measure the >>>>> memory foot print over time? If so, how? >>>> >>>> >>>> I'm not sure, but I've heard Pascal mention he is using Nagios to monitor >>>> memory usage. >>> Well lets hope that Pascal has some ideas. >>> Thank you, >>> Dan Beatty, M.S. CS (B.S. EECS) >>> Ph.D. Student >>> Texas Tech University >>> dan.bea...@mac.com >>> http://web.me.com/danielbeatty/My_Home_Page/Welcome.html >>> (806)438-6620 >>> >>>> >>>> Ramsey >>>> >>>> >>>>> >>>>> Thank you, >>>>> >>>>> Dan Beatty, M.S. CS (B.S. EECS) >>>> >>>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Do not post admin requests to the list. They will be ignored. >>> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) >>> Help/Unsubscribe/Update your Subscription: >>> http://lists.apple.com/mailman/options/webobjects-dev/ramsey%40xeotech.com >>> >>> This email sent to ram...@xeotech.com >>> >> > > > > > > > > > > > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
