Larry, I wanted to get back to you yesterday, and I got bery busy at work. I don't personally have a problem with you commiting your changes to the BasicFeature class. We can start testing the change in our nightly build, and will be able to get an idea on how serious the memory impact is. In the meantime, I might tinker with some alternate implementations.
If none of the other programmers have a serious objection at this point, we could commit the change. It's not like we can't undo the change if it has serious impacts on memory usage. The Sunburned Surveyor On Mon, Apr 27, 2009 at 12:56 PM, Larry Becker <becker.la...@gmail.com> wrote: > Using a scheme with the container sound good, but I can't find an > appropriate listener. Layer#setFeatureCollection has featuresAdded and > featuresRemoved. Layer#getLayerListener() has the void featuresChanged() > method which does setFeatureCollectionModified(true), but this doesn't help > with tracking individual changes to features. > > If we added the appropriate listeners to BasicFeature, how would this change > the state data that needs to be stored? Wouldn't it essentially be the same > data? > > Larry > > On Mon, Apr 27, 2009 at 2:07 PM, Martin Davis <mbda...@refractions.net> > wrote: >> >> I see your point, although newly-created BasicFeatures aren't really an >> issue - they are always dirty by definition. It's just features which >> have had their attributes modified which are the problem. >> >> I'm not totally crazy about this MutableBasicFeature/BasicFeature >> dichotomy, but it might be workable. You would have to flip BFs to MBFs >> when they were written out, since they would then become clean again. >> >> I think I'd prefer a scheme where the container kept track of which >> features have been modified, and leave features as simple value >> objects. I believe this is what GeoTools does. Can anyone confirm that >> before I go digging through their codebase? >> >> Larry Becker wrote: >> > >> > Perhaps you could extend BasicFeature to be a MutableBasicFeature, >> > and >> > this is the concrete class that would be used by Writeable >> > DataStores. >> > >> > >> > This is a good idea, but tough to implement. A search for "new >> > BasicFeature" found 80 matches in the project. The fact is that most >> > tools create BasicFeatures. Now perhaps, if we are clever, we can use >> > that to our advantage. Hmm... >> > >> > Larry >> > >> > On Thu, Apr 23, 2009 at 6:16 PM, Martin Davis <mbda...@refractions.net >> > <mailto:mbda...@refractions.net>> wrote: >> > >> > Perhaps you could extend BasicFeature to be a MutableBasicFeature, >> > and >> > this is the concrete class that would be used by Writeable >> > DataStores. >> > Only MutableBasicFeature would incur the overhead of tracking >> > changes. >> > Whether the DataStore is writeable could be determined dynamically >> > as >> > well (ie by user input when layer is created). There are a lot of >> > use >> > cases where the writable aspect isn't needed. >> > >> > If it was me, I would still prefer the simpler single boolean >> > dirty flag >> > (although I would probably opt for still implementing >> > MutableBasicFeature, in case there was more behaviour that needed >> > to be >> > added down the road.) >> > >> > If I get a chance I'll try and see what GeoTools does here - they >> > must >> > have this same problem. >> > >> > Larry Becker wrote: >> > > I think the other developers need to understand that the cost of >> > > supporting per-feature modified status will be a per-feature cost >> > of >> > > (size of int) * number of attributes. I'm not sure I made that >> > > clear. Currently the memory cost is for all features on all >> > layers. >> > > Perhaps there is needed a way to opt-in to this cost and >> > functionality >> > > at layer creation time. Hmm... >> > > >> > > Larry >> > > >> > > On Thu, Apr 23, 2009 at 4:53 PM, Martin Davis >> > <mbda...@refractions.net <mailto:mbda...@refractions.net> >> > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>>> wrote: >> > > >> > > I guess the memory issue might be moot for DataStore-based >> > datasets >> > > anyway, since the whole point is to only keep in memory what >> > is >> > > currently displayed on the screen. >> > > >> > > But simpler is always better! Alarm bells start ringing for >> > me when I >> > > see counts being used - the semantics seems like they must >> > be more >> > > complicated than a simple flag. >> > > >> > > Larry Becker wrote: >> > > >> > >> > >> > > > I agree that a mechanism to reset the modified status is >> > needed, >> > > > especially after the changes to the layer have been >> > flushed to the >> > > > database. I'll add this to BasicFeature. >> > > > >> > > > Your comment about carrying around state data after it is >> > needed is >> > > > reasonable. The code could check for modified on-the-fly >> > while >> > > it is >> > > > incrementing the attribute mod count and if it is greater >> > than one, >> > > > set a boolean instead and dispose of the array of ints. >> > You would >> > > > lose the ability to determine individual attribute mods >> > (not sure if >> > > > it is needed), however you would only save memory in >> > modified >> > > records >> > > > which wouldn't usually be very much. >> > > > >> > > > I was hoping someone would come up with a clever hack that >> > wouldn't >> > > > require the int array. All of the optimizations that I have >> > > > considered break in one case or another. >> > > > >> > > > BTW, if you really want to save memory on attribute storage, >> > I >> > > have a >> > > > number of questionable schemes that load attributes on an >> > as-needed >> > > > basis. :-) SkyJUMP even has an optimization were layers >> > that >> > > aren't >> > > > visible are not loaded into memory until you make them >> > visible, >> > > > although it doesn't go as far as removing them when you >> > make them >> > > > invisible. Of course the most famous attribute >> > memory-saving scheme >> > > > is Michael's permgen attribute string scheme for dbf files. >> > > That one >> > > > saves a ton of memory on typically redundant data, until >> > you run out >> > > > of permgen memory. >> > > > >> > > > Larry >> > > > >> > > > On Thu, Apr 23, 2009 at 4:06 PM, Martin Davis >> > > <mbda...@refractions.net <mailto:mbda...@refractions.net> >> > <mailto:mbda...@refractions.net <mailto:mbda...@refractions.net>> >> > > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net> >> > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>>>> wrote: >> > > > >> > > > re Mutating Geometry in-place - JTS discourages this, >> > but >> > > does not >> > > > prevent it. Sometimes people do this when they are >> > transforming >> > > > coordinates (eg. translation or affine transform).. >> > It's >> > > quite >> > > > possible that all the JUMP code is clean, however. In >> > any >> > > case, this >> > > > will only be an issue if that particular function is >> > used. >> > > > >> > > > re tracking modifications - how about providing a method >> > > that lets you >> > > > clear the modified flag? Then the feature can be >> > constructed as >> > > > necessary and then marked as clean. >> > > > >> > > > It seems heavyweight to carry around a set of data >> > which is only >> > > > really >> > > > of use during the construction phase of a Feature. To >> > avoid >> > > this, I'd >> > > > even suggest constructing a Feature as needed, then >> > creating >> > > a new >> > > > Feature from it via a constructor which sets the >> > modified flag >> > > > appropriately. Anything to avoid requiring more >> > storage. >> > > > >> > > > How does GeoTools handle this? >> > > > >> > > > Larry Becker wrote: >> > > > > The tricky thing about modifications is not to find a >> > > record is >> > > > > modified just because you set the initial value. It >> > is only >> > > > modified >> > > > > if you set it more than once, otherwise all records >> > would >> > > be set as >> > > > > modified as soon as they are loaded. >> > > > > >> > > > > The next trick is to consider that you call >> > setAttribute >> > > multiple >> > > > > times with different attribute indexes, so it is >> > necessary >> > > to track >> > > > > the changes to each one separately. >> > > > > >> > > > > Mutating Geometries in place is a concern. I have >> > never >> > > seen code >> > > > > that does this, and certainly none of the edit tools >> > or >> > > any plugins >> > > > > that use transactions do this, but it may be >> > possible. Could >> > > > you just >> > > > > modify the Coordinate.x and y values? I'll try to >> > construct a >> > > > > Beanshell test for this, but I doubt that this is a >> > > serious concern. >> > > > > All of the tools and plugins that I have tried so >> > far play by >> > > > the rules. >> > > > > >> > > > > Larry >> > > > > >> > > > > On Thu, Apr 23, 2009 at 2:12 PM, Martin Davis >> > > > <mbda...@refractions.net >> > <mailto:mbda...@refractions.net> <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>> >> > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net> <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>>> >> > > > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net> >> > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>> >> > > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net> >> > > <mailto:mbda...@refractions.net >> > <mailto:mbda...@refractions.net>>>>> wrote: >> > > > > >> > > > > Larry, why do you use an int rather than a >> > boolean to flag >> > > > changed >> > > > > attributes? >> > > > > >> > > > > BTW, In order to track changes to Geometry >> > attributes >> > > > correctly, the >> > > > > JUMP codebase needs to be scrutinized to make >> > sure it >> > > isn't >> > > > mutating >> > > > > Geometries "in place". >> > > > > >> > > > > >> > > > > >> > > > > Larry Becker wrote: >> > > > > > Hi, >> > > > > > >> > > > > > As I mentioned in the other thread, before >> > the problem >> > > > of partial >> > > > > > database updates can be solved, we must first >> > be able to >> > > > > determine if >> > > > > > a Feature has been modified. This is not >> > currently >> > > > possible in >> > > > > all of >> > > > > > the JUMP variants that I am familiar with, >> > although >> > > Kosmo >> > > > may have >> > > > > > implemented it. >> > > > > > >> > > > > > The simplest way of implementing it that I can >> > see is to >> > > > modify >> > > > > > BasicFeature to include: >> > > > > > >> > > > > > private int[] attributeModCount; //this is a >> > parallel >> > > > array to >> > > > > > Object[] attributes >> > > > > > >> > > > > > Then it would be necessary to allocate the >> > > > attributeModCount array >> > > > > > when setAttributes(Object[] attributes) is >> > called. >> > > > > > >> > > > > > In addition each time setAttribute(int >> > > attributeIndex, Object >> > > > > > newAttribute) is called, add the line of code: >> > > > > > >> > > > > > attributeModCount[attributeIndex]++ >> > > > > > >> > > > > > With these modifications we could then define: >> > > > > > >> > > > > > public boolean isFeatureModified() { >> > > > > > for (int i=0; i<attributeModCount.length; >> > i++) { >> > > > > > if (attributeModCount[i] > 1) //modified >> > is >> > > defined as >> > > > > setting >> > > > > > an attribute more than once >> > > > > > return true; >> > > > > > } >> > > > > > return false; >> > > > > > } >> > > > > > >> > > > > > Would this work and does this seem like >> > something we >> > > should >> > > > > consider? >> > > > > > >> > > > > > regards, >> > > > > > Larry >> > > > > > -- >> > > > > > http://amusingprogrammer.blogspot.com/ >> > > > > > >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > > > >> > > > > > _______________________________________________ >> > > > > > Jump-pilot-devel mailing list >> > > > > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>> >> > > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>>> >> > > > > > >> > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > > > >> > > > > >> > > > > -- >> > > > > Martin Davis >> > > > > Senior Technical Architect >> > > > > Refractions Research, Inc. >> > > > > (250) 383-3022 >> > > > > >> > > > > >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > > > Crystal Reports - New Free Runtime and 30 >> > Day Trial >> > > > > Check out the new simplified licensign option >> > that enables >> > > > unlimited >> > > > > royalty-free distribution of the report >> > engine for >> > > > externally >> > > > > facing >> > > > > server and web deployment. >> > > > > http://p.sf.net/sfu/businessobjects >> > > > > _______________________________________________ >> > > > > Jump-pilot-devel mailing list >> > > > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>> >> > > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>>> >> > > > > >> > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > http://amusingprogrammer.blogspot.com/ >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > > >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > > > Crystal Reports - New Free Runtime and 30 Day >> > Trial >> > > > > Check out the new simplified licensign option that >> > enables >> > > unlimited >> > > > > royalty-free distribution of the report engine for >> > > > externally facing >> > > > > server and web deployment. >> > > > > http://p.sf.net/sfu/businessobjects >> > > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > > >> > > > > _______________________________________________ >> > > > > Jump-pilot-devel mailing list >> > > > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>> >> > > > > >> > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > > >> > > > >> > > > -- >> > > > Martin Davis >> > > > Senior Technical Architect >> > > > Refractions Research, Inc. >> > > > (250) 383-3022 >> > > > >> > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > > Crystal Reports - New Free Runtime and 30 Day Trial >> > > > Check out the new simplified licensign option that >> > enables >> > > unlimited >> > > > royalty-free distribution of the report engine for >> > > externally >> > > > facing >> > > > server and web deployment. >> > > > http://p.sf.net/sfu/businessobjects >> > > > _______________________________________________ >> > > > Jump-pilot-devel mailing list >> > > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>>> >> > > > >> > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > http://amusingprogrammer.blogspot.com/ >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > >> > > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > > Crystal Reports - New Free Runtime and 30 Day Trial >> > > > Check out the new simplified licensign option that enables >> > unlimited >> > > > royalty-free distribution of the report engine for >> > > externally facing >> > > > server and web deployment. >> > > > http://p.sf.net/sfu/businessobjects >> > > > >> > > >> > >> > ------------------------------------------------------------------------ >> > > > >> > > > _______________________________________________ >> > > > Jump-pilot-devel mailing list >> > > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > > >> > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > >> > > >> > > -- >> > > Martin Davis >> > > Senior Technical Architect >> > > Refractions Research, Inc. >> > > (250) 383-3022 >> > > >> > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > Crystal Reports - New Free Runtime and 30 Day Trial >> > > Check out the new simplified licensign option that enables >> > unlimited >> > > royalty-free distribution of the report engine for >> > externally >> > > facing >> > > server and web deployment. >> > > http://p.sf.net/sfu/businessobjects >> > > _______________________________________________ >> > > Jump-pilot-devel mailing list >> > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > <mailto:Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net>> >> > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > >> > > >> > > >> > > >> > > -- >> > > http://amusingprogrammer.blogspot.com/ >> > > >> > >> > ------------------------------------------------------------------------ >> > > >> > > >> > >> > ------------------------------------------------------------------------------ >> > > Crystal Reports - New Free Runtime and 30 Day Trial >> > > Check out the new simplified licensign option that enables >> > unlimited >> > > royalty-free distribution of the report engine for >> > externally facing >> > > server and web deployment. >> > > http://p.sf.net/sfu/businessobjects >> > > >> > >> > ------------------------------------------------------------------------ >> > > >> > > _______________________________________________ >> > > Jump-pilot-devel mailing list >> > > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > >> > >> > -- >> > Martin Davis >> > Senior Technical Architect >> > Refractions Research, Inc. >> > (250) 383-3022 >> > >> > >> > >> > ------------------------------------------------------------------------------ >> > Crystal Reports - New Free Runtime and 30 Day Trial >> > Check out the new simplified licensign option that enables unlimited >> > royalty-free distribution of the report engine for externally >> > facing >> > server and web deployment. >> > http://p.sf.net/sfu/businessobjects >> > _______________________________________________ >> > Jump-pilot-devel mailing list >> > Jump-pilot-devel@lists.sourceforge.net >> > <mailto:Jump-pilot-devel@lists.sourceforge.net> >> > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > >> > >> > >> > >> > -- >> > http://amusingprogrammer.blogspot.com/ >> > ------------------------------------------------------------------------ >> > >> > >> > ------------------------------------------------------------------------------ >> > Crystal Reports - New Free Runtime and 30 Day Trial >> > Check out the new simplified licensign option that enables unlimited >> > royalty-free distribution of the report engine for externally facing >> > server and web deployment. >> > http://p.sf.net/sfu/businessobjects >> > ------------------------------------------------------------------------ >> > >> > _______________________________________________ >> > Jump-pilot-devel mailing list >> > Jump-pilot-devel@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > >> >> -- >> Martin Davis >> Senior Technical Architect >> Refractions Research, Inc. >> (250) 383-3022 >> >> >> >> ------------------------------------------------------------------------------ >> Crystal Reports - New Free Runtime and 30 Day Trial >> Check out the new simplified licensign option that enables unlimited >> royalty-free distribution of the report engine for externally facing >> server and web deployment. >> http://p.sf.net/sfu/businessobjects >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > -- > http://amusingprogrammer.blogspot.com/ > > ------------------------------------------------------------------------------ > Crystal Reports - New Free Runtime and 30 Day Trial > Check out the new simplified licensign option that enables unlimited > royalty-free distribution of the report engine for externally facing > server and web deployment. > http://p.sf.net/sfu/businessobjects > _______________________________________________ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > ------------------------------------------------------------------------------ Register Now & Save for Velocity, the Web Performance & Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance & Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
