RE: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Peter B. West wrote: The last assessment of the situation that I can recall on this list was that property handling is working satisfactorily so there is no need to pay any attention to it. Yes, really. I don't think this is quite right. It is a matter of priorities. Property handling works, layout does not. I can't speak for others, but my hangup on alt-design properties that has prevented progress is that it so far has been presented as inextricably linked to pull-parsing and the other parts of alt-design. Until either 1) the case for that approach has been made more satisfactorily than it has until now, or 2) alt-design property handling can be unbundled from pull-parsing, it really is too much to ask to take the alt-design approach. Especially with the FO Tree isolated, I am glad to have someone step in and improve its performance, if (and only if) we nail down its API to the rest of FOP so that the rest of FOP doesn't have to know what is going on here. Victor Mote
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
John Austin wrote: On Tue, 2003-11-18 at 22:36, Peter B. West wrote: ... Overnight I ran another test, using xml-fop_20031118231549.tar.gz and the final summary of the call tree SUGGESTS that things are better (although the results may be incorrect). ... My focus is on performance. I came to the properties discussion from a measurement of program performance. In my experience, measurement is the most reliable approach to optimization. Entirely understandable. Let me know what conclusions you come to. I will fill you in off-line on the approach I am taking. My apologies to everyone on the list for the testy tone of my previous posting. Peter -- Peter B. West http://www.powerup.com.au/~pbwest/resume.html
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
On Wed, 2003-11-19 at 19:50, Peter B. West wrote: John Austin wrote: ... My apologies to everyone on the list for the testy tone ... You mean I might have help p*ss*ng people off ? Peter -- John Austin [EMAIL PROTECTED]
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Victor Mote wrote: Peter B. West wrote: The last assessment of the situation that I can recall on this list was that property handling is working satisfactorily so there is no need to pay any attention to it. Yes, really. I don't think this is quite right. It is a matter of priorities. Property handling works, layout does not. OK. Except that I would say, Property handling works as far as it goes. If you want a conforming implementation, property handling has to be complete. I can't speak for others, but my hangup on alt-design properties that has prevented progress is that it so far has been presented as inextricably linked to pull-parsing and the other parts of alt-design. Until either 1) the case for that approach has been made more satisfactorily than it has until now, or 2) alt-design property handling can be unbundled from pull-parsing, it really is too much to ask to take the alt-design approach. Not if it works. That is a bone of contention, of course. One of the problems with the Rec, I think, is that only by attempting an implementation do you gain an understanding of it. Especially with the FO Tree isolated, I am glad to have someone step in and improve its performance, if (and only if) we nail down its API to the rest of FOP so that the rest of FOP doesn't have to know what is going on here. Peter -- Peter B. West http://www.powerup.com.au/~pbwest/resume.html
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
No shortages here. Peter John Austin wrote: On Wed, 2003-11-19 at 19:50, Peter B. West wrote: John Austin wrote: ... My apologies to everyone on the list for the testy tone ... You mean I might have help p*ss*ng people off ? -- Peter B. West http://www.powerup.com.au/~pbwest/resume.html
RE: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Peter B. West wrote: [This message was originally on fop-user. I have moved it here as it is mostly a dev topic.] The approach I took was to resolve all of the properties at every node in the tree during the parsing. As the FO tree builder descends to the leaf nodes, a complete properties array is maintained at each level. Before the tree builder exits a node though, all properties not relevant to that particular node type are discarded (i.e. the property sets are reduced) so that the maximal data arrays are short-lived. The relevant properties are maintained in sparse arrays. FWIW, we now have the FO Tree tied together in a more tree-like way. We have several methods which rely on going up the tree to an appropriate level to get information. Right now, that is mostly high-level stuff like the logger document-level information. For properties, I recommend that we use appropriately-named get methods for each individual property. Then we can hide the data structures behind that modify them as needed. So if a get method sees that the property is inherited, it can get it from the parent node instead of needing to walk through the tree to resolve the value. There is a flaw in this approach which is exposed when markers are processed. Because marker subtrees are conceptually re-parented in the static-content tree of the page on which they are laid out, the properties within the marker tree cannot initially be resolved, and the properties of the static-content trees cannot be reduced. This situation is not catered for in the existing code, but is quite easy to accommodate by not reducing the property sets of nodes in the static-content, and by slicing the marker subtrees out and storing them unresolved for later processing, including property resolution, in the context of static-content. All of this assumes that property resolution cannot be completed independently of layout. This is a major point of difference between me and everyone else on the FOP team, but the implication is there in the Rec. if you read it carefully. I think I agree with your statement that property resolution cannot be completed independently of layout for the specific case that you mention. I think where we differ is about whether the FO Tree can simply store a To Be Determined At Layout value and go on about life, or whether the concept of the FO Tree should be scrapped entirely because of this issue. Victor Mote
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Looks like I really put my foot into it this time ;-) I have repeated the measurements I did yesterday and I think that it is a pretty reasonable conclusion that a lot of resources are consumed by FOP in its rather Byzantine property management code. I just spent a while trying to understand PropertyList and PropertyListBuilder and found out that I need to understand Property and Property.Maker as well. I think I am going to have to help with this part of the project but it is going to take a while. I offered (off-line) to look merging the Alt-Design code in to the main branch but I suspect that there are some different directions associated with this. Perhaps this is the reason it has not been done so far. I am still willing to work on this but I don't want to walk in to a firefight. I believe the measurements I did yesterday and I feel that a bit of algorithm replacement should produce a significant improvement in the program. I would also like to suggest that anyone interested in performance look at Java Memory Profiler at http://www.khelekore.org/jmp/performance.html I suspect there are still major memory leaks in FOP and this is one tool that will help you track them down. -- John Austin [EMAIL PROTECTED]
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
John Austin wrote: Looks like I really put my foot into it this time ;-) What makes you say that? I have repeated the measurements I did yesterday and I think that it is a pretty reasonable conclusion that a lot of resources are consumed by FOP in its rather Byzantine property management code. I just spent a while trying to understand PropertyList and PropertyListBuilder and found out that I need to understand Property and Property.Maker as well. I think I am going to have to help with this part of the project but it is going to take a while. I offered (off-line) to look merging the Alt-Design code in to the main branch but I suspect that there are some different directions associated with this. Perhaps this is the reason it has not been done so far. I am still willing to work on this but I don't want to walk in to a firefight. That way lies madness. When you look into the *actual* requirements for property processing, you may well draw the same conclusions as I did. In fact, when I got to the point of running some comparative timing tests, I had already decided that there were serious flaws in my design which I needed to address before proceeding. Some of those I have already mentioned. I will give you the benefit of my experiences off-line. You may be able to see a way to integrate the work I have done with the current directions of HEAD. I believe the measurements I did yesterday and I feel that a bit of algorithm replacement should produce a significant improvement in the program. The last assessment of the situation that I can recall on this list was that property handling is working satisfactorily so there is no need to pay any attention to it. Yes, really. I would also like to suggest that anyone interested in performance look at Java Memory Profiler at http://www.khelekore.org/jmp/performance.html I suspect there are still major memory leaks in FOP and this is one tool that will help you track them down. Peter -- Peter B. West http://www.powerup.com.au/~pbwest/resume.html
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
--- John Austin [EMAIL PROTECTED] wrote: I just spent a while trying to understand PropertyList and PropertyListBuilder and found out that I need to understand Property and Property.Maker as well. I think I am going to have to help with this part of the project but it is going to take a while. I offered (off-line) to look merging the Alt-Design code in to the main branch but I suspect that there are some different directions associated with this. Perhaps this is the reason it has not been done so far. I am still willing to work on this but I don't want to walk in to a firefight. It would be great if you can help us out--you may be one of the few that can understand both Alt-Design and our current approach in HEAD--plus you seem to be very knowledgable in code optimization--what we need here. May I suggest, start tentatively sketching out your ideas for properties on our FOP wiki page--you'll probably get comments from many of the other committers. I believe the measurements I did yesterday and I feel that a bit of algorithm replacement should produce a significant improvement in the program. I would also like to suggest that anyone interested in performance look at Java Memory Profiler at http://www.khelekore.org/jmp/performance.html That looks like a fantastic product for us--I did a Google search on it as soon as I saw your screen shots on it--thanks--we possibly should even add it to our team page so others can critique our code as well. I suspect there are still major memory leaks in FOP and this is one tool that will help you track them down. Yes, this is something that will help us get to them--Thanks! Glen __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree
