Re: Java components in blocks
On Apr 22, 2005, at 8:34 AM, Daniel Fagerstrom wrote: Pier allready have problems with this, that is one reason for his interest in this area. I have also have had problems with it from time to time. I have as well. So, I don't want to left it as a desicion to the blocks developers as it just don't scale. What Pier and I have suggested might be somewhat less powerfull, (but that is left to see), but it does scale. I don't want us to start with a contract with blocks developers that we have to restrict later, it is better to do it the other way around. I basically agree. On the other hand, I'm not sure this is a preventable problem. Once you have two or more apps depending on a single jar, you have a choice of either to deploy multiple copies or to put the jar in a central location. How so? What is the alternative? Have blocks that have no component dependencies? If a block depends on a component it depends on a component. If a sitemap uses a component it uses a component. Of course a block should be allowed to depend on and contain components, no one have said anything else. The discussion is about if a block should be able to expose any type of component to other blocks. Maybe, I didn't express myself well. This is what I think is a non-issue. I think it is a mistake to make it the block manager's responsibility to do this simply because of the way Java works. I understand that you want two deployment levels. I don't want a global unshielded level as I'm certain that it doesn't scale. You're right it doesn't. Personally,I would prefer just a single level, but I get enough complaints about multiple copies of jars that I think some alternative is necessary. The multiple dependencies of blocks gives us few if any alternatives. Initially, to have the choice of deploying Cocoon wide or sitemap wide solves some basic issues. The ultimate solution is much more difficult because of the multiple dependencies a block may have. It may be that it is impossible to solve all possible scenarios. I don't know. I think it would be possible to solve, but personally I don't find it worthwhile right now. OK. Your point is well taken, If I can find the time I will see what I can do. Lets just see what happens. I'm sure in the end all will be fine. Glen Ezkovich HardBop Consulting glen at hard-bop.com A Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers." - Thomas Pynchon Gravity's Rainbow
Re: Java components in blocks
Glen Ezkovich wrote: On Apr 18, 2005, at 12:07 PM, Daniel Fagerstrom wrote: Glen Ezkovich wrote: On Apr 18, 2005, at 7:05 AM, Daniel Fagerstrom wrote: The issue here is one of deployment, where to locate the class and which ClassLoader will load the class. It seems to me that if we have two component deployment levels, global and sitemap, we can pretty much accomplish the same thing as exposing or not exposing components. If all jars from all blocks are put in a global sitemap it just don't scale. It is hard enough to keep jars in different blocks in synch whithin the Cocoon SVN. When external developers start to develop and distribute their own blocks it will stop working. Agreed. Certain jars should be available cocoon wide and others not. Unfortunately, this must be left up to the blocks developer to decide how a particular jar gets deployed. Just as it is now. One of the points with real blocks is to scale the community by make it possible for people to develop Cocoon blocks independent of ASF and the current set of committers and without central control. If these external block developers can put jars in an common unshielded area in Cocoon we will get the situation where they put different vesrions of the same jar in this common area. Sy that you have a system that depends on block A and block B developed by different external organizations. When you update Cocoon or block A or block B you suddenly realized that the last version have started to export a jar that it didn't exported before and that allready was exported from one of the other blocks in another version. Pier allready have problems with this, that is one reason for his interest in this area. I have also have had problems with it from time to time. So, I don't want to left it as a desicion to the blocks developers as it just don't scale. What Pier and I have suggested might be somewhat less powerfull, (but that is left to see), but it does scale. I don't want us to start with a contract with blocks developers that we have to restrict later, it is better to do it the other way around. Because of that classloader isolation is needed so that a block only is exposed to the classes it actually depend on from its parent classloader. And that is complicated stuff, if you don't think so you can take a look at kernel in whiteboard. I know so. I didn't say it would be easy. And I have no personal interest in solving that problem right now. If someone else has, just go ahead, as long as you don't introduce concepts that we allready know that they doesn't scale. Things get more complicated and creates IMO unnecessary dependencies between blocks when you combine it with sitemap functionallity. How so? What is the alternative? Have blocks that have no component dependencies? If a block depends on a component it depends on a component. If a sitemap uses a component it uses a component. Of course a block should be allowed to depend on and contain components, no one have said anything else. The discussion is about if a block should be able to expose any type of component to other blocks. Because of that complexity Pier and I prefer to separate the problems. If you separate the problems thats fine with me, but to consider this a complete solution to the total problem which is to have real blocks is a mistake. When I deploy an app, I usually have a custom component or two and a bunch of POJOs. If I want to package an app as a block I would need to package the sitemap and associated files as a block and package the jars as a bundle. A block at the very least should include the bundle. As an app developer I want to have a single deployment solution. As a sys admin, I want to ensure that I don't have 30 copies of identical jars hanging around. Packaging is a separate concern. Of course we want it as convenient as possible. Large units has the benefit that you get everything at once and the drawback that different complete packages might overlap and be redundant. But if we have automatic deployment it is probably more practicalt to have fine grained packaging. As long as the block you happen to be intersted in has a list of it dependencies the block deployer will take care of the work anyway. I understand that this is a complex problem and breaking it up into smaller pieces is a wise thing to do. My basic points are that to have real blocks you need a single deployment and that there are two levels to deploy Java components, global and sitemap, parent sitemaps included. I understand that you want two deployment levels. I don't want a global unshielded level as I'm certain that it doesn't scale. Initially, to have the choice of deploying Cocoon wide or sitemap wide solves some basic issues. The ultimate solution is much more difficult because of the multiple dependencies a block may have. It may be that it is impossible to solve all possible scenarios. I don't know. I think it would be possible t
Re: Java components in blocks
On Apr 18, 2005, at 12:07 PM, Daniel Fagerstrom wrote: Glen Ezkovich wrote: On Apr 18, 2005, at 7:05 AM, Daniel Fagerstrom wrote: The portal definition files define how individual portlets are invoked and rendered. As I stated before, ideally these would be separate blocks. However, since many will contain java code, it sounds like many portlets would have to be a block with a matching bundle. A block can contain code. It is just (at least in the first step) not allowed to expose components. So if the portlet components need to be accessible from other parts of the system you need both a bundle and a block. But if the components only are needed internally and the portlet can expose all its functionality as pipeline functionality, it is enough with a block. Sorry to be late to the party, but I was unsure where this discussion was headed and choose to keep my mouth shut. I'm glad to see to that blocks can have code again.;-) I think the issue of exposing components is a non issue. Did you read http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=111339143403364&w=2 ? Yes We are after all talking about java byte code, if some one wants to use the jar/class file it just needs to be on the classpath. Sure, but what if two blocks contain different versions of a jar? I admit that this is a problem. The way I see this, is that if a block developer is using a jar that may have multiple versions deployed in the cocoon environment they should make sure to deploy that jar local to a sitemap. The issue here is one of deployment, where to locate the class and which ClassLoader will load the class. It seems to me that if we have two component deployment levels, global and sitemap, we can pretty much accomplish the same thing as exposing or not exposing components. If all jars from all blocks are put in a global sitemap it just don't scale. It is hard enough to keep jars in different blocks in synch whithin the Cocoon SVN. When external developers start to develop and distribute their own blocks it will stop working. Agreed. Certain jars should be available cocoon wide and others not. Unfortunately, this must be left up to the blocks developer to decide how a particular jar gets deployed. Just as it is now. Because of that classloader isolation is needed so that a block only is exposed to the classes it actually depend on from its parent classloader. And that is complicated stuff, if you don't think so you can take a look at kernel in whiteboard. I know so. I didn't say it would be easy. Things get more complicated and creates IMO unnecessary dependencies between blocks when you combine it with sitemap functionallity. How so? What is the alternative? Have blocks that have no component dependencies? If a block depends on a component it depends on a component. If a sitemap uses a component it uses a component. Because of that complexity Pier and I prefer to separate the problems. If you separate the problems thats fine with me, but to consider this a complete solution to the total problem which is to have real blocks is a mistake. When I deploy an app, I usually have a custom component or two and a bunch of POJOs. If I want to package an app as a block I would need to package the sitemap and associated files as a block and package the jars as a bundle. A block at the very least should include the bundle. As an app developer I want to have a single deployment solution. As a sys admin, I want to ensure that I don't have 30 copies of identical jars hanging around. I understand that this is a complex problem and breaking it up into smaller pieces is a wise thing to do. My basic points are that to have real blocks you need a single deployment and that there are two levels to deploy Java components, global and sitemap, parent sitemaps included. Initially, to have the choice of deploying Cocoon wide or sitemap wide solves some basic issues. The ultimate solution is much more difficult because of the multiple dependencies a block may have. It may be that it is impossible to solve all possible scenarios. I don't know. Glen Ezkovich HardBop Consulting glen at hard-bop.com A Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers." - Thomas Pynchon Gravity's Rainbow
Re: Java components in blocks
Reinhard Poetz wrote: Daniel Fagerstrom wrote: Reinhard Poetz wrote: Daniel Fagerstrom wrote: Maybe not, but those are areas that are rather vague anyway, we where the only ones who thought it had much importance the last time we discussed it. Maybe it had the appearance but I can't believe this. (Recently I had a discussion with Sylvain and he really wants this feature too. So we are at least three ;-) ) Might be, we still need to know the use cases to provide a good solution. Any comments to what I suggested in my response? BTW I rather meant: var result = cocoon.sendPageAndWait("block:myblockA://confirmDialog", {msg: "areYouSure"}); A part from that this changes the contract of sendPageAndWait (returns a continuation), The response object could be made available in an extra argument instead. Also we could have VPC flowscript functions as long as their return values are restricted to what is part of the "core" classloader. I would prefer the explicit call of flowscript functions of other blocks. Anything else would have been a suprise to me ;) I think the usecases are quite clear (reuse a flowscript function of another block and use it the same way as a local function) but we shouldn't discuss this ATM. Let's wait until Pier and you have set up the base infrastructure (Block[s]Manager + some basic classloader isoluation) and then we will see what is (easily) possible and what not. Good, it feels rather abstract to discuss whats missing in something we haven't implemented or tried. /Daiel
Re: Java components in blocks
Daniel Fagerstrom wrote: Reinhard Poetz wrote: Daniel Fagerstrom wrote: Maybe not, but those are areas that are rather vague anyway, we where the only ones who thought it had much importance the last time we discussed it. Maybe it had the appearance but I can't believe this. (Recently I had a discussion with Sylvain and he really wants this feature too. So we are at least three ;-) ) Might be, we still need to know the use cases to provide a good solution. Any comments to what I suggested in my response? BTW I rather meant: var result = cocoon.sendPageAndWait("block:myblockA://confirmDialog", {msg: "areYouSure"}); A part from that this changes the contract of sendPageAndWait (returns a continuation) I would prefer the explicit call of flowscript functions of other blocks. I think the usecases are quite clear (reuse a flowscript function of another block and use it the same way as a local function) but we shouldn't discuss this ATM. Let's wait until Pier and you have set up the base infrastructure (Block[s]Manager + some basic classloader isoluation) and then we will see what is (easily) possible and what not. -- Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach {Software Engineering, Open Source, Web Applications, Apache Cocoon} web(log): http://www.poetz.cc
Re: Java components in blocks
Glen Ezkovich wrote: On Apr 18, 2005, at 7:05 AM, Daniel Fagerstrom wrote: The portal definition files define how individual portlets are invoked and rendered. As I stated before, ideally these would be separate blocks. However, since many will contain java code, it sounds like many portlets would have to be a block with a matching bundle. A block can contain code. It is just (at least in the first step) not allowed to expose components. So if the portlet components need to be accessible from other parts of the system you need both a bundle and a block. But if the components only are needed internally and the portlet can expose all its functionality as pipeline functionality, it is enough with a block. Sorry to be late to the party, but I was unsure where this discussion was headed and choose to keep my mouth shut. I'm glad to see to that blocks can have code again.;-) I think the issue of exposing components is a non issue. Did you read http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=111339143403364&w=2 ? We are after all talking about java byte code, if some one wants to use the jar/class file it just needs to be on the classpath. Sure, but what if two blocks contain different versions of a jar? The issue here is one of deployment, where to locate the class and which ClassLoader will load the class. It seems to me that if we have two component deployment levels, global and sitemap, we can pretty much accomplish the same thing as exposing or not exposing components. If all jars from all blocks are put in a global sitemap it just don't scale. It is hard enough to keep jars in different blocks in synch whithin the Cocoon SVN. When external developers start to develop and distribute their own blocks it will stop working. Because of that classloader isolation is needed so that a block only is exposed to the classes it actually depend on from its parent classloader. And that is complicated stuff, if you don't think so you can take a look at kernel in whiteboard. Things get more complicated and creates IMO unnecessary dependencies between blocks when you combine it with sitemap functionallity. Because of that complexity Pier and I prefer to separate the problems. /Daniel
Re: Java components in blocks
On Apr 18, 2005, at 7:05 AM, Daniel Fagerstrom wrote: The portal definition files define how individual portlets are invoked and rendered. As I stated before, ideally these would be separate blocks. However, since many will contain java code, it sounds like many portlets would have to be a block with a matching bundle. A block can contain code. It is just (at least in the first step) not allowed to expose components. So if the portlet components need to be accessible from other parts of the system you need both a bundle and a block. But if the components only are needed internally and the portlet can expose all its functionality as pipeline functionality, it is enough with a block. Sorry to be late to the party, but I was unsure where this discussion was headed and choose to keep my mouth shut. I'm glad to see to that blocks can have code again.;-) I think the issue of exposing components is a non issue. We are after all talking about java byte code, if some one wants to use the jar/class file it just needs to be on the classpath. The issue here is one of deployment, where to locate the class and which ClassLoader will load the class. It seems to me that if we have two component deployment levels, global and sitemap, we can pretty much accomplish the same thing as exposing or not exposing components. WDYT? Glen Ezkovich HardBop Consulting glen at hard-bop.com A Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers." - Thomas Pynchon Gravity's Rainbow
Re: Java components in blocks
Reinhard Poetz wrote: Daniel Fagerstrom wrote: Maybe not, but those are areas that are rather vague anyway, we where the only ones who thought it had much importance the last time we discussed it. Maybe it had the appearance but I can't believe this. (Recently I had a discussion with Sylvain and he really wants this feature too. So we are at least three ;-) ) Might be, we still need to know the use cases to provide a good solution. Any comments to what I suggested in my response? BTW I rather meant: var result = cocoon.sendPageAndWait("block:myblockA://confirmDialog", {msg: "areYouSure"}); /Daniel
Re: Java components in blocks
Daniel Fagerstrom wrote: Maybe not, but those are areas that are rather vague anyway, we where the only ones who thought it had much importance the last time we discussed it. Maybe it had the appearance but I can't believe this. (Recently I had a discussion with Sylvain and he really wants this feature too. So we are at least three ;-) ) -- Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach {Software Engineering, Open Source, Web Applications, Apache Cocoon} web(log): http://www.poetz.cc
Re: Java components in blocks
Reinhard Poetz wrote: Daniel Fagerstrom wrote: So WDYT, does this make sense for your use cases? Yes, for the pipelines part but that's not the whole story. Just two thoughts: Don't know what we gain when we introduce two separate deployment units (bundles and blocks). Maybe it's more confusing than helping. If necessary we can have different types of blocks. Sure, I called them bundles and block to make the discussion more concrete, we could call them component blocks and sitemap block or whatever. From a repository and deployment perspective they will have much in common so we should of course use the same mechanism as long as possible. Second, I don't see our flowscript usecase var true = cocoon.blocks.myblockA.confirmDialog("Are you sure?") or This is a page flow and would fall into the reponsibility area of blocks. It could be implemented as some flowscript VPC. In that case we need to restrict what kind of objects that can be returned to those that are part of the "core" classloader. We could also have a more request/repsponse oriented approach and make it possible to put the true/false and other data in the reponse object, in that case we would rather get: var result = cocoon.sendPageAndWait("block:myblockA://areYouSure"); var newCustomer = cocoon.blocks.crm.createCustomer() This falls in the area of "bundles" and component management, if the Customer interface is defined in crm, the user of it must have access to the Customer interafce from its classloader. We could develop some mechanism that makes it possible to implement components in JS but that would not be related to flow IMO. is covered so far, isn't it?. Maybe not, but those are areas that are rather vague anyway, we where the only ones who thought it had much importance the last time we discussed it. /Daniel
Re: Java components in blocks
Ralph Goers wrote: Daniel Fagerstrom wrote: Blocks are of course also packaged, distributed and dependency resolved much like the bundles above. --- o0o --- So WDYT, does this make sense for your use cases? Thanks. Yes, this makes a lot more sense. I'm not sure how workable it will be, but it is possible it might be. The only real issue I see is for the person trying to construct a portal. They need: a. A main sitemap that defines the portal sitemap components, the component-configurations section that defines the name of the portal and its profiles, and the main portal pipeline. b. additions to cocoon.xconf (presumably through their own xconf) that configures components that are in the portal bundle, or possibly in their own bundle if they choose to implement their own features. c. the portal definition files (these are obtained via pipelines in the user's main portal sitemap and might be dynamically generated). The portal definition files define how individual portlets are invoked and rendered. As I stated before, ideally these would be separate blocks. However, since many will contain java code, it sounds like many portlets would have to be a block with a matching bundle. A block can contain code. It is just (at least in the first step) not allowed to expose components. So if the portlet components need to be accessible from other parts of the system you need both a bundle and a block. But if the components only are needed internally and the portlet can expose all its functionality as pipeline functionality, it is enough with a block. JSR-168 portlets I guess would have to be bundles, if they are packaged for Cocoon, as they don't contain a sitemap of any kind. Seem reasonable. So as I understand it, a block would have the sitemap and would require a companion bundle which contained the xconf file if it had any components that need to be configured, or if it needs to configure components provided in another bundle. Is that correct? See above. The block can import and configure components as much as it like. As I have described in previous discussions, one way to make block extensible and configurable is by using polymorphic sitemap rules. The block, e.g. a portlet, contains configurations and default content that are used through polymorphic sitemap rules. Then a user can extend the original block and overide the sitemap rules that are needed for the application. We could also let extending blocks get access to the component manager and classloader from the extended block whithout introducing to much complexity. Parent - child classloader dependencies is still easy enough to manage. The problem comes when the blocks start to have mutual depdencies on the Java level. My only concern here is simply how many moving parts it will take to construct a portal. If that becomes a problem we must of course solve that. The main point is that Pier allready has a working system that solves important parts of the "bundle" aspect. According to what he have writen, it is used at VNU and solves a number of our current problems in Cocoon. And I feel rather confident in start building the blocks part as soon as I find some time for it. We use some simple block like mechanism in our internal webapps, and we have very good experience in. So if we start with those aspects we might have something much more concrete to discuss about in a close future. If that is enough, that is fine, if not we do something about it. /Daniel
Re: Java components in blocks
Daniel Fagerstrom wrote: So WDYT, does this make sense for your use cases? Yes, for the pipelines part but that's not the whole story. Just two thoughts: Don't know what we gain when we introduce two separate deployment units (bundles and blocks). Maybe it's more confusing than helping. If necessary we can have different types of blocks. Second, I don't see our flowscript usecase var true = cocoon.blocks.myblockA.confirmDialog("Are you sure?") or var newCustomer = cocoon.blocks.crm.createCustomer() is covered so far, isn't it?. -- Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach {Software Engineering, Open Source, Web Applications, Apache Cocoon} web(log): http://www.poetz.cc
Re: Java components in blocks
Daniel Fagerstrom wrote: Blocks are of course also packaged, distributed and dependency resolved much like the bundles above. --- o0o --- So WDYT, does this make sense for your use cases? Thanks. Yes, this makes a lot more sense. I'm not sure how workable it will be, but it is possible it might be. The only real issue I see is for the person trying to construct a portal. They need: a. A main sitemap that defines the portal sitemap components, the component-configurations section that defines the name of the portal and its profiles, and the main portal pipeline. b. additions to cocoon.xconf (presumably through their own xconf) that configures components that are in the portal bundle, or possibly in their own bundle if they choose to implement their own features. c. the portal definition files (these are obtained via pipelines in the user's main portal sitemap and might be dynamically generated). The portal definition files define how individual portlets are invoked and rendered. As I stated before, ideally these would be separate blocks. However, since many will contain java code, it sounds like many portlets would have to be a block with a matching bundle. JSR-168 portlets I guess would have to be bundles, if they are packaged for Cocoon, as they don't contain a sitemap of any kind. So as I understand it, a block would have the sitemap and would require a companion bundle which contained the xconf file if it had any components that need to be configured, or if it needs to configure components provided in another bundle. Is that correct? My only concern here is simply how many moving parts it will take to construct a portal. Ralph
Re: Java components in blocks
On 17 Apr 2005, at 16:44, Ralph Goers wrote: Pier Fumagalli wrote: My vision is quite simple... How many _COMPONENT_ interfaces we need for Cocoon? Probably 3: "Generator" "Transformer" "Serializer" (well, you might have another 2 or 3 of them but do you see the point?) Please take a look at the Portal and then explain how the above works for that. There are a heck of a lot more components that someone building a portal needs to configure besides these. Sitemap components are only uses in Cocoon Portlets. While each Portal request is fired off by the PortalGenerator, it invokes a lot of elements that are configured by whoever puts together the portal. Reading through the portal (I'm no expert here), I can't really understand what ties the Portal (Pluto, and all that lot), with a Cocoon block... I think this is more in the ballpark of "components" rather than Cocoon blocks and pipeline services. But again, I might be wrong... Someone care to explain Pier
Re: Java components in blocks
Ralph Goers wrote: Pier Fumagalli wrote: My vision is quite simple... How many _COMPONENT_ interfaces we need for Cocoon? Probably 3: "Generator" "Transformer" "Serializer" (well, you might have another 2 or 3 of them but do you see the point?) Please take a look at the Portal and then explain how the above works for that. There are a heck of a lot more components that someone building a portal needs to configure besides these. Sitemap components are only uses in Cocoon Portlets. While each Portal request is fired off by the PortalGenerator, it invokes a lot of elements that are configured by whoever puts together the portal. Don't have time to take any detailed look at the portal ATM, but I give my view on the main concepts and then you can see if they seem to help your use case. In the original proposal was the idea that a block exposes both sitemap functionality and components. Our current idea is that we, at least initially handles the two areas separately. Let us for concretness call the sitemap aspect a block and the component aspect a bundle. Bundles --- A bundle contains one or more components and some meta info. The meta info can be about dependecies on components in other bundles it could also contain default configurations and possibly other stuff. There might be special interface bundles that only contain the apis. A bundle would typically be a jar and we would store them in repositories like ibiblio.org. We could use Maven for downloading them and resolve the dependencies and e.g. Pier's kernel for managing them an providing classloader shielding. A large part of our current blocks would be bundles rather than blocks. Most of the portal would probably be packaged as a bundle. Component management is not my main expertise, but I wouldn't be supprised if much of what we need in this area already is developed in some other projects. Blocks -- Blocks are at the sitemap level. It is like a small configurable and possibly extendable webapp the can be used as part of user webapps. Its main communication line with the rest of the application is the block protocol, it takes an URI and a request object and returns a response object. You can use it internally and/or mount its URI space so that Cocoon expose it to the external world. A block can also expose sitemap components. A block can depend on components from bundles but it doesn't expose components (at least we hope that it will be enough). A block can contain own Java code for internal use. Block can be full application frameworks like Forrest, Lenya, Linotype and possibly a portal application. A WebDAV repository could be packaged as a block. A set of webservices could also be packaged as a block. Blocks can also be smaller and more specialized things like skins or the examples Pier gave. Maybe prtlets could make sense as blocks. A large application block probably is put together from a number of small blocks. Blocks are of course also packaged, distributed and dependency resolved much like the bundles above. --- o0o --- So WDYT, does this make sense for your use cases? /Daniel
Re: Java components in blocks
Pier Fumagalli wrote: My vision is quite simple... How many _COMPONENT_ interfaces we need for Cocoon? Probably 3: "Generator" "Transformer" "Serializer" (well, you might have another 2 or 3 of them but do you see the point?) Please take a look at the Portal and then explain how the above works for that. There are a heck of a lot more components that someone building a portal needs to configure besides these. Sitemap components are only uses in Cocoon Portlets. While each Portal request is fired off by the PortalGenerator, it invokes a lot of elements that are configured by whoever puts together the portal. Ralph
Re: Java components in blocks
On 16 Apr 2005, at 13:56, Daniel Fagerstrom wrote: Torsten Curdt wrote: Reading these discussions after the fact, having Blocks provide only sitemap components seems to make a lot of sense ...not to me - sorry. But maybe I just missed something. Pier is totally right: we have two different concerns. One is the pipeline services interface and one is the component interface of a block. But reducing a block just to it's pipeline services basically gives us virtual sitemap components on steroids. What about its dependencies? Well, IIUC one argument in this discussion was that the dependency will be satisfied through a pipeline service - not through a component. Block A: provides: a pipeline service to transform xml requires: a pipeline service to transform xml with a STX stylesheet Block B: provides: a pipeline service to transform xml with a STX stylesheet So block B does not provide the component with hint "stx" but a service that could be anything doing the job satisfying the dependency. Ok. Now what about the component dependencies? Let's say in order to implement the "transform-via-stx" service block B requires a component of hint "stx". Since the block B has its own component manager and the component "stx" being declared in the block's sitemap all is left is a java class level dependency to the stx implementation. Now the question is - will the block B provide the stx jar? Let's say yes for the moment. So what if the "transform-via-stx" component needs another component? We could list all the required component in the components section for that very block. ...but still the classes need to be available! What if the classes are in a different block? Essentially this means to me: As long as we don't want to ship every block with every component it requires I cannot see how we can get around of having blocks also expose component services. The idea is that if you have a component that you want to be able to use in several places, you package it in a certain way. Then the improved component handling system that Pier feel the itch to build can found its jar in some repository somwhere, reolve its dependencies and offer a shielded parent classloader with the depndencies you have asked for. I don't know the exact details about Pier's vison about this, but you can take a look at the kernel in whiteboard for part of it and Maven is also a part of the equation for downöloading and dependency reolution. My vision is quite simple... How many _COMPONENT_ interfaces we need for Cocoon? Probably 3: "Generator" "Transformer" "Serializer" (well, you might have another 2 or 3 of them but do you see the point?) How many _BLOCK_ interfaces can we have on the other hand? "ForrestSkin", "WebMailRepository", hundreds. Then, if each implementation of a block interface is allowed to say "To provide this transformer pipeline service I need the SQL transformer component, the XSLT component and STX component" (all those three simply implement the component "Transformer" Java interface, the problem of dependancy resolution is so easy from both sides: On the block side, you can completely ignore classloaders and java code, on the component side, you're pretty much sure that the number of component interfaces is not going to be so big and so "volatile" in terms of version numbers (and therefore simplifies the classloading structure). I mean, the problem right now is not that we have two versions of the "o.a.c.s.Serializer" (Java) interface, but that we _need_ to have in the same Cocoon two versions of "Fop" implementing the _same_precisely_exact_ Java interface... See what I mean? Pier
Re: Java components in blocks
On 16 Apr 2005, at 12:43, Torsten Curdt wrote: Reading these discussions after the fact, having Blocks provide only sitemap components seems to make a lot of sense ...not to me - sorry. But maybe I just missed something. Pier is totally right: we have two different concerns. One is the pipeline services interface and one is the component interface of a block. But reducing a block just to it's pipeline services basically gives us virtual sitemap components on steroids. What about its dependencies? Well, IIUC one argument in this discussion was that the dependency will be satisfied through a pipeline service - not through a component. Block A: provides: a pipeline service to transform xml requires: a pipeline service to transform xml with a STX stylesheet Block B: provides: a pipeline service to transform xml with a STX stylesheet Hmm... As far as I can see, intra-block dependancy is "available" only through interface blocks, right? So, the "Forrest" block, requires the "ForrestSkin" interface, and this is injected into it by creating a new "instance" of the "CocoonSkin" (a block which implements ForrestSkin). So, now we end up with a problem: if to provide the pipeline service to transform XML, my "CocoonSkin" uses an XSLT, this block will depend on the "XSLTTransformer", if it uses STX it will depend on the "STXTransformer", if it uses XSLT2.0 and XQUERY I might want to use "SaxonTransformer" version 8.0 or greater... You see that if a block needs to specify its COMPONENT requirements (not BLOCK requirements) only through interface, we'll end up having one interface block probably per each implementation block... Mess... If we separate blocks and components, at this point, we can carve into stone that block requirements _must_ go through interfaces, components, actually are a completely different beast. A block exposing a PDF serializer for "BOOK.DTD" documents might require specifically a version of FOP, because its stylesheet works around some bugs into that particular implementation... Pier
Re: Java components in blocks
Torsten Curdt wrote: Reading these discussions after the fact, having Blocks provide only sitemap components seems to make a lot of sense ...not to me - sorry. But maybe I just missed something. Pier is totally right: we have two different concerns. One is the pipeline services interface and one is the component interface of a block. But reducing a block just to it's pipeline services basically gives us virtual sitemap components on steroids. What about its dependencies? Well, IIUC one argument in this discussion was that the dependency will be satisfied through a pipeline service - not through a component. Block A: provides: a pipeline service to transform xml requires: a pipeline service to transform xml with a STX stylesheet Block B: provides: a pipeline service to transform xml with a STX stylesheet So block B does not provide the component with hint "stx" but a service that could be anything doing the job satisfying the dependency. Ok. Now what about the component dependencies? Let's say in order to implement the "transform-via-stx" service block B requires a component of hint "stx". Since the block B has its own component manager and the component "stx" being declared in the block's sitemap all is left is a java class level dependency to the stx implementation. Now the question is - will the block B provide the stx jar? Let's say yes for the moment. So what if the "transform-via-stx" component needs another component? We could list all the required component in the components section for that very block. ...but still the classes need to be available! What if the classes are in a different block? Essentially this means to me: As long as we don't want to ship every block with every component it requires I cannot see how we can get around of having blocks also expose component services. The idea is that if you have a component that you want to be able to use in several places, you package it in a certain way. Then the improved component handling system that Pier feel the itch to build can found its jar in some repository somwhere, reolve its dependencies and offer a shielded parent classloader with the depndencies you have asked for. I don't know the exact details about Pier's vison about this, but you can take a look at the kernel in whiteboard for part of it and Maven is also a part of the equation for downöloading and dependency reolution. Then the responsibility for blocks is to deliver sitemap functionality rather than components. A block declare what components it depend on and the component system takes care of that part for it. It will, at least in my vision, be possible to put Java code and components within the block that only is intended for block internal usage. --- o0o --- So the situation rigth now is that Pier have the itch to start integrate improved component handling with classloader shielding ito Cocoon. And I have the itch to start building the sitemap part of the original blocks proposal (actually I have even started on it although nothing is committed yet). For the time beeing we are going to *assume* that is is separate concerns, although the blocks system will use the component system. In this way we reduce the complexity to a point where we feel comfortable to actually do things, and get experince from that, instead of spending a couple of more years on just discussing them. In a next step, anyone who feel that it is necessary can integrate the two parts as it was intended originally. My hope is that we don't feel the need, as it IMHO is mixing two different architectural levels and it, according my current state of knowledge, means that we are creating a complexity nightmare for ourselves. Also letting the block depend on each other in the way that they can export any interface and jar, means that they get a very tight coupling and all kinds of version problems. I can't see that it is scaleable to the level where different organizations develop blocks independently, that users easily can integrate. --- o0o --- But this is just my view, if you feel that it is FUD and that is is much easier than I say, it is just to go ahead and propose better ideas and maybe even implement them ;) /Daniel
Re: Java components in blocks
> Reading these discussions after the fact, having Blocks provide only > sitemap components seems to make a lot of sense ...not to me - sorry. But maybe I just missed something. Pier is totally right: we have two different concerns. One is the pipeline services interface and one is the component interface of a block. But reducing a block just to it's pipeline services basically gives us virtual sitemap components on steroids. What about its dependencies? Well, IIUC one argument in this discussion was that the dependency will be satisfied through a pipeline service - not through a component. Block A: provides: a pipeline service to transform xml requires: a pipeline service to transform xml with a STX stylesheet Block B: provides: a pipeline service to transform xml with a STX stylesheet So block B does not provide the component with hint "stx" but a service that could be anything doing the job satisfying the dependency. Ok. Now what about the component dependencies? Let's say in order to implement the "transform-via-stx" service block B requires a component of hint "stx". Since the block B has its own component manager and the component "stx" being declared in the block's sitemap all is left is a java class level dependency to the stx implementation. Now the question is - will the block B provide the stx jar? Let's say yes for the moment. So what if the "transform-via-stx" component needs another component? We could list all the required component in the components section for that very block. ...but still the classes need to be available! What if the classes are in a different block? Essentially this means to me: As long as we don't want to ship every block with every component it requires I cannot see how we can get around of having blocks also expose component services. cheers -- Torsten signature.asc Description: OpenPGP digital signature
Re: Java components in blocks
Le 14 avr. 05, à 17:33, Ralph Goers a écrit : ...Frankly, I'd love to see how this will even work. If someone could explain how the cron block can be a block without exposing components then maybe I could understand. I really want to understand how this could apply to the portal block, but the cron block is a lot simpler... Reading these discussions after the fact, having Blocks provide only sitemap components seems to make a lot of sense: it looks immediately useful, understandable and Cocoon-specific, whereas yet another (potentially complicated) general java component mechanism sounds a lot like reinventing the wheel (I'm overly simplifying here). About cron and similar blocks (JMS, embedded database server, etc.): thinking of them as unix "daemons" might help fit them nicely in the concept. Conceptually, I think such blocks need an initialization mechanism, a reconfiguration interface (Transformer?) and a status interface (Generator). Basing the blocks concept on sitemap components (the grep sed ps ls awk sort of Cocoon) and "daemon components" (the cron mysql sendmail of Cocoon) might help find the right balance here. -Bertrand smime.p7s Description: S/MIME cryptographic signature
Re: Java components in blocks
Stefano Mazzocchi wrote: I had the same reaction at first, and then it accurred to me that if we went down the path of having optional code and optional sitemap in blocks, not only people are likely to keep them separate (ending up with 'sitemap blocks' and 'code blocks') but that would force us to reinvent the maven wheel for code blocks. I'm not actually 100% of how this separation is going to work in real life, but I'm willing to sit back and watch how far along we can go while keeping things separate, because making them unified I fear was one of the reasons why blocks have been so hard of a problem to attack. At the end, we might end up understanding that we need to reinvent the maven wheel, but I really hope not. So, I suggest you to follow me in being a little more hopeful about it, sit back and watch the show and just trust the community in its collective wisdom. What do you mean by "reinvent the maven wheel"? If my boss ever lets me have some time to work on Cocoon, one of the first things I'd like to do is to try to introduce maven into the Cocoon build. IMO, this has needed to happen for quite a while regardless of what happens with blocks. We've been "reinventing" maven for far too long. I'm not trying to get in the way. It is just that my company will be using the Cocoon Portal to manage the layout of all our products. I really like the block concept, but if I can't apply it to the portal I just don't see the value of it. The Portal block is primarily a bunch of components - many of which are not sitemap components. They would actually need to be provided, or packaged in, the Portal block, but configured by the block that implements the application. In addition, it would make sense to have individual portlets packaged in their own blocks. These could be JSR-168 portlets, which can contain JSPs and/or Java code, or they could be Cocoon Portlets - which are primarily sitemaps. In short, "real blocks" could be a huge win for dealing with the Cocoon Portal, if they have the needed support. Ralph
Re: Java components in blocks
Ralph Goers wrote: Daniel Fagerstrom wrote: For the portal block, which I still don't know that much about, I would assume that it would become a number of components with some kind of dependency description for each and a block with sitemap functionality and a list on what components it depend on. Hopefully Pier can tell more about his ideas about component handling. My main interest is the sitemap part of the equation. /Daniel Hmmm. If blocks go down the path of only supporting sitemaps then I guess I'm not too interested, since none of the stuff I care about fits into that. I had the same reaction at first, and then it accurred to me that if we went down the path of having optional code and optional sitemap in blocks, not only people are likely to keep them separate (ending up with 'sitemap blocks' and 'code blocks') but that would force us to reinvent the maven wheel for code blocks. I'm not actually 100% of how this separation is going to work in real life, but I'm willing to sit back and watch how far along we can go while keeping things separate, because making them unified I fear was one of the reasons why blocks have been so hard of a problem to attack. At the end, we might end up understanding that we need to reinvent the maven wheel, but I really hope not. So, I suggest you to follow me in being a little more hopeful about it, sit back and watch the show and just trust the community in its collective wisdom. -- Stefano.
Re: Java components in blocks
Daniel Fagerstrom wrote: For the portal block, which I still don't know that much about, I would assume that it would become a number of components with some kind of dependency description for each and a block with sitemap functionality and a list on what components it depend on. Hopefully Pier can tell more about his ideas about component handling. My main interest is the sitemap part of the equation. /Daniel Hmmm. If blocks go down the path of only supporting sitemaps then I guess I'm not too interested, since none of the stuff I care about fits into that. Ralph
Re: Java components in blocks
Pier Fumagalli wrote: On 14 Apr 2005, at 13:32, Daniel Fagerstrom wrote: Does anyone have problems if I wipe the current "kernel" in SVN? I want to try and synchronize it with the changes I've made locally (and try to sort out a few hacks we've had to introduce in our local copy) No problem for me. Just go ahead :) In that case I would be interested in geting a short description of the involved concepts and how they cooperate. I'm thinking of: Block, Contract, Extension, Library, Module and Plugin. I'm actually thinking about renaming those as I feel that the naming convention might be extremely misleading. I'll try to Wikify some thoughs over the weekend (If I can log-in on MoinFrigginMoin) Great! /Daniel
Re: Java components in blocks
Pier Fumagalli wrote: On 14 Apr 2005, at 13:32, Daniel Fagerstrom wrote: After having read your mails I agree that we at least for the time being, should consider component management with classloader isolation, a separate concern. By _assuming_ independence between blocks and component management, anyone who feel the itch can start working on class loader isolation right away. Then we can integrate the component management stuff in the blocks later if we feel the need. And if we don't, we saved ourself from considerable complexity. I have that itch, I hoped so :) Does anyone have problems if I wipe the current "kernel" in SVN? I want to try and synchronize it with the changes I've made locally (and try to sort out a few hacks we've had to introduce in our local copy) no problems for me. and to tell you the truth, the kernel in SVN _already_ does a some level of classloader isolation. Thing is, I use it on a daily basis for an XML repository we have (in house) at VNU, and the more I use it, the more I see the problems of the _current_ approach. I hit a dead end with the implementation in CVS last week when I wanted to upgrade JAXP from 1.2 to 1.3, running on a 1.4 JVM, but _not_ using the -Djava.endorsed.dirs JVM flag. I can't tell you the issues on trying to isolate the JAXP 1.3 contract and kernel plug-in from the underlying JVM, but with a bit of luck, I found what's wrong, and we should be up and running with the new kernel (here at VNU) in a week-or-so... Same here, as long as you tell us what you are planning to do, so that especially Carsten and Sylvain can say if it works with their plans, you have my big +1. My problem is that I'm not "paid" to work on Cocoon, so I kinda "hide" all this work behind changes that are required for our in-house XML-repository. And some of the hacks that are in our current code are definitely something I don't want to add to the Cocoon SVN. Is it based on the whiteboard kernel? Yup... It's basically that, but I've spotted a few problems using it :-( In that case I would be interested in geting a short description of the involved concepts and how they cooperate. I'm thinking of: Block, Contract, Extension, Library, Module and Plugin. I'm actually thinking about renaming those as I feel that the naming convention might be extremely misleading. I'll try to Wikify some thoughs over the weekend (If I can log-in on MoinFrigginMoin) awesome. I'm starting to like this separation more and more. -- Stefano.
Re: Java components in blocks
On 14 Apr 2005, at 13:32, Daniel Fagerstrom wrote: After having read your mails I agree that we at least for the time being, should consider component management with classloader isolation, a separate concern. By _assuming_ independence between blocks and component management, anyone who feel the itch can start working on class loader isolation right away. Then we can integrate the component management stuff in the blocks later if we feel the need. And if we don't, we saved ourself from considerable complexity. I have that itch, I hoped so :) Does anyone have problems if I wipe the current "kernel" in SVN? I want to try and synchronize it with the changes I've made locally (and try to sort out a few hacks we've had to introduce in our local copy) and to tell you the truth, the kernel in SVN _already_ does a some level of classloader isolation. Thing is, I use it on a daily basis for an XML repository we have (in house) at VNU, and the more I use it, the more I see the problems of the _current_ approach. I hit a dead end with the implementation in CVS last week when I wanted to upgrade JAXP from 1.2 to 1.3, running on a 1.4 JVM, but _not_ using the -Djava.endorsed.dirs JVM flag. I can't tell you the issues on trying to isolate the JAXP 1.3 contract and kernel plug-in from the underlying JVM, but with a bit of luck, I found what's wrong, and we should be up and running with the new kernel (here at VNU) in a week-or-so... Same here, as long as you tell us what you are planning to do, so that especially Carsten and Sylvain can say if it works with their plans, you have my big +1. My problem is that I'm not "paid" to work on Cocoon, so I kinda "hide" all this work behind changes that are required for our in-house XML-repository. And some of the hacks that are in our current code are definitely something I don't want to add to the Cocoon SVN. Is it based on the whiteboard kernel? Yup... It's basically that, but I've spotted a few problems using it :-( In that case I would be interested in geting a short description of the involved concepts and how they cooperate. I'm thinking of: Block, Contract, Extension, Library, Module and Plugin. I'm actually thinking about renaming those as I feel that the naming convention might be extremely misleading. I'll try to Wikify some thoughs over the weekend (If I can log-in on MoinFrigginMoin) Pier
Re: Java components in blocks
Ralph Goers wrote: Stefano Mazzocchi wrote: Pier Fumagalli wrote: IMO, we should start simple and add functionality when needed. Remove things is much harder. Also even if we chose to go for alternative 4. an important message from Pier that we should take into account is that component exposure adds complexity. So if we go that way, we should have mechanisms that so that we can have detailed control over what components and classes a block exposes. A block that exposes much will be much more likely to create conflicts with other blocks. Glad that we have agreement on this! :-D Ok, you changed my mind. Let's keep the concerns separate and see what happens. Frankly, I'd love to see how this will even work. If someone could explain how the cron block can be a block without exposing components then maybe I could understand. The cron block seem to consist of a number of components and no sitemap functionality so it would, at least in the first step not be a block at all. Pier's idea IIUC is to have a possibly Maven driven automatic instalation and dependency resolution of components. A block, which at a start is mainly about sitemap functionallity, would declare what components it depend on. A big advantage with this approach is that we can start working on better component handling, (class loader isolation, automatic download and automatic dependency resolution), independently from the blocks and hopefully reuse some proven external technology for components. At a later stage when we have got some experience from the more advanced component handling and the block system we *might* want to integrate them according to the original plan. But then we will know much more about the concequnces. I really want to understand how this could apply to the portal block, but the cron block is a lot simpler. For the portal block, which I still don't know that much about, I would assume that it would become a number of components with some kind of dependency description for each and a block with sitemap functionality and a list on what components it depend on. Hopefully Pier can tell more about his ideas about component handling. My main interest is the sitemap part of the equation. /Daniel
Re: Java components in blocks
Stefano Mazzocchi wrote: Pier Fumagalli wrote: IMO, we should start simple and add functionality when needed. Remove things is much harder. Also even if we chose to go for alternative 4. an important message from Pier that we should take into account is that component exposure adds complexity. So if we go that way, we should have mechanisms that so that we can have detailed control over what components and classes a block exposes. A block that exposes much will be much more likely to create conflicts with other blocks. Glad that we have agreement on this! :-D Ok, you changed my mind. Let's keep the concerns separate and see what happens. Frankly, I'd love to see how this will even work. If someone could explain how the cron block can be a block without exposing components then maybe I could understand. I really want to understand how this could apply to the portal block, but the cron block is a lot simpler. Ralph
Re: Java components in blocks
Pier Fumagalli wrote: IMO, we should start simple and add functionality when needed. Remove things is much harder. Also even if we chose to go for alternative 4. an important message from Pier that we should take into account is that component exposure adds complexity. So if we go that way, we should have mechanisms that so that we can have detailed control over what components and classes a block exposes. A block that exposes much will be much more likely to create conflicts with other blocks. Glad that we have agreement on this! :-D Ok, you changed my mind. Let's keep the concerns separate and see what happens. -- Stefano.
Re: Java components in blocks
Pier Fumagalli wrote: On 14 Apr 2005, at 10:16, Daniel Fagerstrom wrote: Pier Fumagalli wrote: As far as I've understood Cocoon blocks (and I might be _ENTIRELY_ wrong here) Cocoon blocks do not expose (for example) an XSLT Java transformer, but expose a transformer converting a specified format into another. AFAIU, you are entirely wrong ;) Ha... I read this differently, then... (Subject: Re: Planning Cocoon's future) On 23 Nov 2004, at 19:32, Stefano Mazzocchi wrote: Are there any reasons not to make Pier's kernel the official base of Cocoon 2.3? If not, Pier should have the "permission" to move its kernel into an offical 2.3 directory in our SVN whenever he thinks it's time for it. (... I prepare a vote after the discussion) If the kernel is stable, should it really be 2.3 or can it be 2.2? my suggestion would be: leave the kernel out and let's keep working on allowing us to deploy and ship the blocks in binary format. classloading isolation will happen later, when the need will be impossible for people to avoid ;-) My mishap? Misinterpretation? Or just one of my usual brainfarts??? :-P I interpreted it as no classloader isolation rather than total classloader isolation. Anyway, lets focus on what we want rather on what people possibly thought once. We'll get a current update of peoples opinons soon enough ;) considering what have been discussed before. Neither the less, what you propose makes perfect sense to me. My idea was to start implementing the pipline part of the blocks management system, as we IMO understand enough about it to be able to work on that. Then add the possiblity to expose components and take care of class loader issues in a later step. On this you have my +1... Also because, I've been looking at a little tool called "Maven" :-P That already tells us the dependancies of one particular library against others... By using the (let's assume) "httpclient-x.y.z.jar" from Maven, the maven descriptor file _already_ tells us this "library" dependancies. We can wrap the library into a "block" (or proto-component), and use the maven descriptor file to calculate dependancies, and use those as instructions to build the classloader structure required by (but separate from) blocks. Given that you synchronize it with Reinhard who has worked on a block builder, you have my +1. After having read your mails I agree that we at least for the time being, should consider component management with classloader isolation, a separate concern. By _assuming_ independence between blocks and component management, anyone who feel the itch can start working on class loader isolation right away. Then we can integrate the component management stuff in the blocks later if we feel the need. And if we don't, we saved ourself from considerable complexity. I have that itch, I hoped so :) and to tell you the truth, the kernel in SVN _already_ does a some level of classloader isolation. Thing is, I use it on a daily basis for an XML repository we have (in house) at VNU, and the more I use it, the more I see the problems of the _current_ approach. I hit a dead end with the implementation in CVS last week when I wanted to upgrade JAXP from 1.2 to 1.3, running on a 1.4 JVM, but _not_ using the -Djava.endorsed.dirs JVM flag. I can't tell you the issues on trying to isolate the JAXP 1.3 contract and kernel plug-in from the underlying JVM, but with a bit of luck, I found what's wrong, and we should be up and running with the new kernel (here at VNU) in a week-or-so... Same here, as long as you tell us what you are planning to do, so that especially Carsten and Sylvain can say if it works with their plans, you have my big +1. Is it based on the whiteboard kernel? In that case I would be interested in geting a short description of the involved concepts and how they cooperate. I'm thinking of: Block, Contract, Extension, Library, Module and Plugin. --- o0o --- Ok, taking a look at the technical issues. IIUC, the problem that you describe does not depend on whether a block contains Java classes or libraries or not. The problem appears as soon as a block tries to expose a component. Then the user of the component must have get all the eposed interfaces from its parent classloader. And when we add mutual dependencies between blocks and different versions of the interfaces, it start feeling more like headache than fun. PRECISELY, you got this spot on... So let's take a look at some different levels of block issolation, and their properties (the pipeline is exposed in all cases): 1. No component exposing The block doesn't expose any components at all, not even VPCs. The block can still contain as much own Java code as it want. It gets a parent classloader with the core Cocoon functionality and adds it own classes in its own classloader. The block must obviously be compatible with the Cocoo
Re: Java components in blocks
On 14 Apr 2005, at 10:16, Daniel Fagerstrom wrote: Pier Fumagalli wrote: On 13 Apr 2005, at 18:08, Vadim Gritsenko wrote: Pier Fumagalli wrote: I'm just saying that the concerns are separate: a block provides generators, transformers, serializers, ... to other blocks, but those are de-coupled from the real java components (if any) doing the job. But *real* blocks, IIUC, *are* the only way to bring new (java) components into the system, because Cocoon core lacks all the optional components, and, at the minimum, consists only of environment, treeprocessor, and block manager. So in extreme case xslt processor component itself is optional and provided by the xslt real block. I think that we're mixing again the idea behind a Cocoon block and an Avalon block. As far as I've understood Cocoon blocks (and I might be _ENTIRELY_ wrong here) Cocoon blocks do not expose (for example) an XSLT Java transformer, but expose a transformer converting a specified format into another. AFAIU, you are entirely wrong ;) Ha... I read this differently, then... (Subject: Re: Planning Cocoon's future) On 23 Nov 2004, at 19:32, Stefano Mazzocchi wrote: Are there any reasons not to make Pier's kernel the official base of Cocoon 2.3? If not, Pier should have the "permission" to move its kernel into an offical 2.3 directory in our SVN whenever he thinks it's time for it. (... I prepare a vote after the discussion) If the kernel is stable, should it really be 2.3 or can it be 2.2? my suggestion would be: leave the kernel out and let's keep working on allowing us to deploy and ship the blocks in binary format. classloading isolation will happen later, when the need will be impossible for people to avoid ;-) My mishap? Misinterpretation? Or just one of my usual brainfarts??? :-P considering what have been discussed before. Neither the less, what you propose makes perfect sense to me. My idea was to start implementing the pipline part of the blocks management system, as we IMO understand enough about it to be able to work on that. Then add the possiblity to expose components and take care of class loader issues in a later step. On this you have my +1... Also because, I've been looking at a little tool called "Maven" :-P That already tells us the dependancies of one particular library against others... By using the (let's assume) "httpclient-x.y.z.jar" from Maven, the maven descriptor file _already_ tells us this "library" dependancies. We can wrap the library into a "block" (or proto-component), and use the maven descriptor file to calculate dependancies, and use those as instructions to build the classloader structure required by (but separate from) blocks. After having read your mails I agree that we at least for the time being, should consider component management with classloader isolation, a separate concern. By _assuming_ independence between blocks and component management, anyone who feel the itch can start working on class loader isolation right away. Then we can integrate the component management stuff in the blocks later if we feel the need. And if we don't, we saved ourself from considerable complexity. I have that itch, and to tell you the truth, the kernel in SVN _already_ does a some level of classloader isolation. Thing is, I use it on a daily basis for an XML repository we have (in house) at VNU, and the more I use it, the more I see the problems of the _current_ approach. I hit a dead end with the implementation in CVS last week when I wanted to upgrade JAXP from 1.2 to 1.3, running on a 1.4 JVM, but _not_ using the -Djava.endorsed.dirs JVM flag. I can't tell you the issues on trying to isolate the JAXP 1.3 contract and kernel plug-in from the underlying JVM, but with a bit of luck, I found what's wrong, and we should be up and running with the new kernel (here at VNU) in a week-or-so... --- o0o --- Ok, taking a look at the technical issues. IIUC, the problem that you describe does not depend on whether a block contains Java classes or libraries or not. The problem appears as soon as a block tries to expose a component. Then the user of the component must have get all the eposed interfaces from its parent classloader. And when we add mutual dependencies between blocks and different versions of the interfaces, it start feeling more like headache than fun. PRECISELY, you got this spot on... So let's take a look at some different levels of block issolation, and their properties (the pipeline is exposed in all cases): 1. No component exposing The block doesn't expose any components at all, not even VPCs. The block can still contain as much own Java code as it want. It gets a parent classloader with the core Cocoon functionality and adds it own classes in its own classloader. The block must obviously be compatible with the Cocoon core, but there are no version interdependencies on the
Re: Java components in blocks
Pier Fumagalli wrote: On 13 Apr 2005, at 18:08, Vadim Gritsenko wrote: Pier Fumagalli wrote: I'm just saying that the concerns are separate: a block provides generators, transformers, serializers, ... to other blocks, but those are de-coupled from the real java components (if any) doing the job. But *real* blocks, IIUC, *are* the only way to bring new (java) components into the system, because Cocoon core lacks all the optional components, and, at the minimum, consists only of environment, treeprocessor, and block manager. So in extreme case xslt processor component itself is optional and provided by the xslt real block. I think that we're mixing again the idea behind a Cocoon block and an Avalon block. As far as I've understood Cocoon blocks (and I might be _ENTIRELY_ wrong here) Cocoon blocks do not expose (for example) an XSLT Java transformer, but expose a transformer converting a specified format into another. AFAIU, you are entirely wrong ;) considering what have been discussed before. Neither the less, what you propose makes perfect sense to me. My idea was to start implementing the pipline part of the blocks management system, as we IMO understand enough about it to be able to work on that. Then add the possiblity to expose components and take care of class loader issues in a later step. After having read your mails I agree that we at least for the time being, should consider component management with classloader isolation, a separate concern. By _assuming_ independence between blocks and component management, anyone who feel the itch can start working on class loader isolation right away. Then we can integrate the component management stuff in the blocks later if we feel the need. And if we don't, we saved ourself from considerable complexity. --- o0o --- Ok, taking a look at the technical issues. IIUC, the problem that you describe does not depend on whether a block contains Java classes or libraries or not. The problem appears as soon as a block tries to expose a component. Then the user of the component must have get all the eposed interfaces from its parent classloader. And when we add mutual dependencies between blocks and different versions of the interfaces, it start feeling more like headache than fun. So let's take a look at some different levels of block issolation, and their properties (the pipeline is exposed in all cases): 1. No component exposing The block doesn't expose any components at all, not even VPCs. The block can still contain as much own Java code as it want. It gets a parent classloader with the core Cocoon functionality and adds it own classes in its own classloader. The block must obviously be compatible with the Cocoon core, but there are no version interdependencies on the Java level between blocks. It can even expose its classloader to extending blocks without problems. The extending block must of course be compatible at the Java level with the extended one, but still no circular dependencies. We can package the VPCs as (block) web services (ws) instead with: VPCReader - Stream -> Stream ws VPCGenerator - Stream -> XML (SAX) ws VPCTransformer - XML -> XML ws VPCSerializer - XML -> Stream ws by making the block protocol competent enough and introducing pipeline components that talk with the corresponding ws, we get something that works like and is as efficient as VPCs. 2. Exposing VPCs only - Like the previous alternative but exposing VPCs. If we go for this, Sylvain's proposal about separating VPCs from other componsnts in the sitemap is preferable IMO. 3. Exposing "sitemap components" only - In this alternative a block can expose components with interfaces that allready are defined in Cocoon core, or possibly a well defined subset of it. As all blocks gets a common parent classloader that contains these interfaces, we still don't get the "mutual dependency of forign blocks" nightmare. 4. Expose what you choose to This is, AFAIU, the original block idea. It requires classloader isolation and dependency resolution to work. 5. Expose your classes to everyone -- This is what we do today. All classes/jars from blocks are loaded into a common classloader that is available for everyone. We can combine the idea of having a block aware component manager with a common classloader. --- o0o --- I think we can rule out alternative 5. at once. Its hard enough to get it to work today when we as a community control all the blocks. It was no accident that we choosed to move all libraries to lib/optional. There is no chance that this will scale, when external organizations start to create blocks. Alternative 4. would maybe be nice, but as Pier I start to doubt that thi
Re: Java components in blocks
On 13 Apr 2005, at 18:08, Vadim Gritsenko wrote: Pier Fumagalli wrote: I'm just saying that the concerns are separate: a block provides generators, transformers, serializers, ... to other blocks, but those are de-coupled from the real java components (if any) doing the job. But *real* blocks, IIUC, *are* the only way to bring new (java) components into the system, because Cocoon core lacks all the optional components, and, at the minimum, consists only of environment, treeprocessor, and block manager. So in extreme case xslt processor component itself is optional and provided by the xslt real block. I think that we're mixing again the idea behind a Cocoon block and an Avalon block. As far as I've understood Cocoon blocks (and I might be _ENTIRELY_ wrong here) Cocoon blocks do not expose (for example) an XSLT Java transformer, but expose a transformer converting a specified format into another. Again, an example using Forrest. The forrest block (ForrestBlock) requires two different blocks implementing the following interfaces: - ForrestRepository: a block exposing a generator that given a uri path (/2.1/whatsnew) generate an XML document in the format - ForrestSkin: a block exposing a serializer converting a document in the above mentioned format into something usable by a browser and exposing a mountable pipeline containing the resources (images, css, javascripts, ...) that the client will see... The ForrestBlock might have a sitemap which might look something like this: Now, the "repository" implementation block might expose a virtual generator like this: but another implementation might specify the following: In the same way, the "skin" implementation block but again, another implementation might do different: width="20" height="20" scale="yes"/> So far, easily enough, we can see how the "ForrestBlock" interacts with the block interfaces it requires, and how those can be composed differently. Easy enough, so in the sitemap we want to use from cocoon: ... ... Now, without considering the real java component which are doing the job (XSLT transformer, IMAGE reader, STX transformer, ...) the block implementation can be entirely developed using the technology (Avalon's ECM/Carsten's ECM+++) without caring too much about Java components themselves. You can simply deploy several versions of the same interface block in the same container, and given that we're taking the problem of loading classes out of the picture, blocks can work as they're supposed to be without thinking much about the crummy Java machinery behind them. Also, think about one thing: is a transformer exposed by a cocoon block a specific XSL or STX or (you name it) transformer, or is it a pseudo-component which per se knows how to translate the input to its output? The java component "XSLT transformer" doesn't do anything "by itself", it becomes a "Cocoon transformer" (it transforms a specified document into another) only when the java XSLT component is associated with the XSLT. But the same "Cocoon transformer" can be created by associating the STX "java component" with an equivalent STX stylesheet. Now, there is also another concern that falls into this ballpark. In my view, I think that our "ForrestBlock" block will only be allowed declare its requirements through interface blocks. This means that for a block to work properly, its requirements must be abstracted entirely from the specific implementations. But a block implementation MIGHT (in my view) require a _very_specific_java_component_ to fulfill the contract it implements. For example, in the case of the "ForrestRepository" interface implementation, the generator exposed is abstracted entirely, but to implement this component I can require a SQLGenerator (Java component), a FileGenerator (Java component), a JcrGenerator (Java component), and so on and so forth. So, in my view, either we define an interface cocoon block for each possible component (95% of those will only have ONE implementation), or we allow blocks to require specific JAva components to do their jobs, without relying on interfaces. So, at the end of the day, the java componentization of how a generic generator (for example) exposed by a block is a completely separate, different, concern from the one implied in the blocks: I want a generator that given a URI produces a document of "book" doctype. My 0.02 £ Pier
Re: Java components in blocks
Pier Fumagalli wrote: I'm just saying that the concerns are separate: a block provides generators, transformers, serializers, ... to other blocks, but those are de-coupled from the real java components (if any) doing the job. But *real* blocks, IIUC, *are* the only way to bring new (java) components into the system, because Cocoon core lacks all the optional components, and, at the minimum, consists only of environment, treeprocessor, and block manager. So in extreme case xslt processor component itself is optional and provided by the xslt real block. Vadim
Re: Java components in blocks
On 13 Apr 2005, at 12:40, Reinhard Poetz wrote: Pier Fumagalli wrote: Ok, here is where I don't agree... By adding to blocks the capability of "bringing components" with them, we enter a quite big minefield, imposed by the restrictions of the Java VM. The complexity escalates at this point as now blocks must be aware of their class-loading mechanism, and the component manager must be aware of blocks and interfaces. For example, classes and libraries included in an interface block must be loaded by the parent ClassLoader of the blocks using (implementing, extending, requiring) them to avoid ClassCastException(s). So, alongside blocks and their deployment graph (block A extends B, implements C, ... blabla ...) we need to start keep tracking also the classes included in those blocks, and create a completely different tree based on the class loading structure that those represent. And then you get into the minefield of versioning... If it's easy to determine that two versions of the same block can co-exist (by using two separate class loaders children of the same parent) without creating too many problems, what happens in the case of two interfaces with different versions? The class loading tree should (at this point) become very fine-grained and start building multiple class-loaders for multiple versions of the same interface, and then build as children of those class loaders for the individual blocks "using" them, ... Hmmm, are we really the first project tackling this? How did Eclipse solved this? I'm not saying that we are the first, nor that it can't be solved. I'm just saying that the concerns are separate: a block provides generators, transformers, serializers, ... to other blocks, but those are de-coupled from the real java components (if any) doing the job. For example, a "mailprovider" implementation used for the block "sendmail" will have to provide a generator that accepts URI like "imap://user:[EMAIL PROTECTED]/folder/id", and generates a MimeDocument XML stream. It doesn't matter to the block what components will be used to generate the XML, I can use a single component to do this job, or I can use a virtual component calling god knows what generator and a number of different transformers to achieve the same output, it doesn't matter to the block using the interface implementation. And it shouldn't matter to the block deployment frameworks whether those components are written in Java or JavaScript or whatever... And neither it should matter what kind of Java ClassLoading components do require to be available to blocks, right? Pier
Java components in blocks
Pier Fumagalli wrote: Ok, here is where I don't agree... By adding to blocks the capability of "bringing components" with them, we enter a quite big minefield, imposed by the restrictions of the Java VM. The complexity escalates at this point as now blocks must be aware of their class-loading mechanism, and the component manager must be aware of blocks and interfaces. For example, classes and libraries included in an interface block must be loaded by the parent ClassLoader of the blocks using (implementing, extending, requiring) them to avoid ClassCastException(s). So, alongside blocks and their deployment graph (block A extends B, implements C, ... blabla ...) we need to start keep tracking also the classes included in those blocks, and create a completely different tree based on the class loading structure that those represent. And then you get into the minefield of versioning... If it's easy to determine that two versions of the same block can co-exist (by using two separate class loaders children of the same parent) without creating too many problems, what happens in the case of two interfaces with different versions? The class loading tree should (at this point) become very fine-grained and start building multiple class-loaders for multiple versions of the same interface, and then build as children of those class loaders for the individual blocks "using" them, ... Hmmm, are we really the first project tackling this? How did Eclipse solved this? -- Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach {Software Engineering, Open Source, Web Applications, Apache Cocoon} web(log): http://www.poetz.cc