Hi Pascal, We are glad that you are pleased with the results :) Until now we have been focused only on dependencies cleanup, our next step is to inspect and refactor bundles source code in order to shrink p2 more. For example metadata bundle seems too large and could probably be optimized by extracting unused features in additional bundle.
About the remarks you made: We agree that there should be an abstraction layer for the repository transport and compatibility layer for the extension points. We also kept the IAgentServiceFactory pattern. What we are currently doing is code refactoring to address your comments. Declarative services are a nice way to provide laziness, but we have noticed something strange in p2's ds. It seems that there are implicit (not declared) references and p2 strongly relies they would be satisfied in any case. For example both repositories rely on repository manager existence, but don't declare reference to it. Thus, it works fine in the lazy case , but it breaks when the bundles are eagerly started. It's not an accident, but rather a pattern - we want to revise those ds and make all their dependencies explicit. It will make them work in both cases. About the profile creation - currently we don't have use case to install bundles in the installer, but we want to keep that door opened. We are working on new OSGi service which will be able to restore the profile either on local or remote framework. So, our intentions are to submit the patches with abstraction and compatibility layers before submitting the isolated p2 back in the mainstream. It will confirm that the changes are backwards compatible. I'll get back to you when we are ready. What do you think of the plan? Thanks, Katya ________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Pascal Rapicault Sent: Saturday, July 24, 2010 10:21 PM To: P2 developer discussions Subject: Re: [p2-dev] P2 Isolation Hi Katya, Congratulations! This is a pretty impressive result, especially given that you are still able to install the SDK (I have not tried it myself). I wonder what else you'll be doing to p2 to shrink it even further :) I glanced through the archive you sent and I have the following remarks / questions: * Removal of ECF. At this point you have done it by changing the code in FileReader to not longer use the ECF APIs. However to be able to merge this back into HEAD, I think we would have to make Transport be API and making it pluggable using the IAgentServiceFactory. * Removal of extension point to the advantage of service. Given that the usage of extension / extension point is part of the p2 SPI, we have to make sure in the Eclipse setup that those are still read. For example in the case of the repository factories, we will likely have to introduce something like a RepositoryFactoryFactory (implementing the IAgentServiceFactory) in order to allow for the tiny setup you created to not depend on the extension registry. * Registration of services: in 3.6, we put a lot of efforts to make p2 be super lazy (introduction of IAgentServiceFactory, usage of DS, etc), and as such starting a lot of services in activators would be a regression. Here again I think that the factory approach or something along those lines could work. * Profile creation. You've added code to automatically a profile based on the set of running bundles. Does that mean that you need to be able to install bundles into the installer itself? In any event, you've effectively demonstrated that p2 could be shrunk and I think that with the appropriate level of refactoring we should be able to merge that back in p2 mainstream when you are ready. PaScaL On 2010-07-22, at 5:47 PM, Todorova, Katya wrote: Hello, After a while we are done with a version of the lean P2 installer. We have decided to keep the Status and ProgressMonitor in order to keep it api compatible and later eclipse consumable. The main refactoring we made is to substitute the usages of declarative services and extension points with standard osgi services. The total size is 2.538 Mbyte and 20 bundles. We will continue to further reduce the size of the installer by extracting the core functionalities and make the rest pluggable (for example pack200 support). We have adopted and run all current p2 available tests and they all pass. You can download the installer as a zip from : https://sapmats-de.sap-ag.de/download/download.cgi?id=5YBCYHAKL9DG29TQCBIU5GDTWL7IJVAST2TDSTUEUTWDFE7G0Z In order to run it, you should extract the zip and run the go.bat The installer is currently implemented as an OSGI command p2_install. Here are the parameters that it takes: p2_install - installs a unit and its dependencies parameters: -installIU - unit to install -destination - folder in which to install the unit -repository - comma separated list of repositories to be looked up for the bundle to install -profile - profile id containing the description of the targeted product. Default is SDKProfile -updater_debug - turns on detailed error info Example: p2_install -installIU org.eclipse.sdk.ide -destination C:\\eclipse<smb://eclipse> -repository http://download.eclipse.org/eclipse/updates/3.6 -profile SDKProfile If you are behind a proxy, add the required -Dhttp.proxy* properties in go.bat We will very much appreciate any feedback, both on execution and implementation. The sources are located at https://sapmats-de.sap-ag.de/download/download.cgi?id=890MWEA401A9TG27HR37WH1IGD12ATHMSDFNQE8MMA587CQ0E3 Looking forward for your comments, Katya ________________________________ From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Todorova, Katya Sent: Tuesday, May 04, 2010 3:47 PM To: P2 developer discussions Subject: RE: [p2-dev] P2 Isolation Hi guys, One more interesting question around that topic pops up. System provsioning via p2 results in a bundles.info<http://bundles.info> file, which contains all the bundles that should be installed on system startup. Since we use p2 for enterprise server provisioning, we expect to have war files in the bundles.info<http://bundles.info>. But war file is transformed to a bundle with appropriate headers at runtme and it's not necessarily persisted. On the other hand, bundles.info<http://bundles.info> entry contain bundle location, which is not standartized. Currently if web bundle location is written to bundles.info<http://bundles.info>, it is impossible for the simple configurator to construct a valid URL. I'm just curios how web bundles fit in p2 concepts - should they be treated as ordinary bundles or special care should be taken? Any comments would be wellcome. Thanks, Katya ________________________________ From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Todorova, Katya Sent: Wednesday, April 21, 2010 6:26 PM To: [email protected]<mailto:[email protected]> Subject: RE: [p2-dev] P2 Isolation Hi Pascal, Here are some details that may describe our requirements clearer. Our intention is to develop p2 compliant tool which is responsible to provision OSGi-based system - only installation and update of the system is required. During that operations there won't be any other interaction with the system. The tool is not supposed to update itself and should contain the minimal functionality that will allow us to do such provisioning. Our main goal is to restrict tool's dependencies as much as possible - the perfect case would be if it depends only on OSGi core specification. That's why we don't intend to use declarative services but will replace them with OSGi services accordingly. So coupling with eclipse encreases tool's complexity and that's a concern since we want it as simple as possible. This is briefly the background. Let me answer your questions: Status and Progress monitoring I agree that these features are very usable and serviceable in complex scenarios and environment (like eclipse). I agree that they provide fine grained control, but we don't have agents outside the system which could benefit from provided functionality. We can live with result "Success" or "Failure" and could get details from the logs - this serves well our current very simple scenario. And also gives us the opportunity to upgrade "simple" p2 to current p2 API if the scenario evolves. Profile registry We will work against one profile and we don't need the profile change history (although I agree that it's very usable to keep track of changes in a system). One more remark here - currently p2 relies on profile existence and could not handle the case when profile is missing. In eclipse infrastructure there are tools which prepare appropriate profile before p2 engine execution (for example), but in our scenario we don't have those. So to handle system installation case, we need to be able to start without profile and to create it when we have declarative description which plugins the system should contain. What do you think of it? ProfileScope I don't think we need to store repositories nor saving any preferences. You mentioned that we can modify eclipse.preferences to equinox.preferences - could you give me an example what worths to be stored in our use case? ProfileEvent This is coupled with monitoring and could be removed if there is no one to use that feature. Touchpoints We don't want to use extension registry for both reasons you mentioned but eclipse specific nature is what bothers us more - it introduces a lot of dependencies to non-core functionality. Touchpoint implementation as extension points could be easily replaced with OSGi standard services and this will make p2 a lot more portable and will decrease its footprint and complexity. Thus it could run on every OSGi framework implementation. We plan to reuse eclipse touchpoints since we want to achieve the same result - our intension is to adapt "touchpoint bundle to register the touchpoints it provides as OSGi services. Thus all consumers (p2 engine for example) could get them via OSGi service registry. This is a change which needs to be adopted by eclipse because otherwise we would end up with two "touchpoint representations" when embedding our product into eclipse. That's why we cannot modify only engine's TouchpointManager and ActionManager because they rely on functionality provided by other bundle. Do you think that such refactoring introduces big risk in current eclipse environment? Phases We agreed that it's good to have them separated from engine:) About the prototype you mentioned - we are very interested to see it built and running around simple APIs like those we have talked about. Where can it be found? Talking about our modification of engine - the listed bundles are all the bundles modified engine depends on (at least from resolving point of view). We have refactored "org.eclipse.equinox.p2.repository" not to use jobs and ECF. We are thinking of another file transfer implementation since ECF is too complex, but we don't have a solution yet. Modified p2 bundles still use equinox.common, but that dependency could be easily cut. Yes, we have target size - around 1MB, but we aim more at reducing dependencies than sticking to that size. I hope our use case is clearer now. Thank you very much for the help and comments. Looking forward to hear from you, Katya ________________________________ From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Pascal Rapicault Sent: Tuesday, April 20, 2010 7:14 PM To: P2 developer discussions Subject: Re: [p2-dev] P2 Isolation Hi, Thank you for your interest in p2. Minimizing p2's footprint, improving its embed-ability and seeing it taken to new places are themes that we are all pretty interested in. When we set out to do p2, it was not only meant to be used by Eclipse but as a general provisioning platform. However, as you have discovered, being the Eclipse guys, we reused the concepts that we were the most familiar with (extension registry, progress) because of expediency but also because our first delivery vehicle and target was the Eclipse SDK and a tight integration with the rest of the platform was necessary (ability to cancel, provide feedback to the user, etc). Now that p2 has matured and people see the value of the platform, feedback and request for enhancements like these are really welcome and I'm really interested in understanding the background for those changes and see how they can be accommodated. Now on to the specifics: You seem to be concerned about the coupling with Eclipse. I can see where this would happen, but would appreciate if you could provide more details as to why you see some of these parts being more binding to eclipse than others. For example I gathered that you would not want to see the extension registry in your final solution, but is this because of its size? of its eclipse specific nature? What about status and progress monitoring? >What is removed (as dependencies and functionality): > IStatus Status the eclipse way of returning errors in a structured way. Without it, how would you go about communicating back a more detailed error about what went wrong during the execution of the plan? > IProgressMonitor The progress monitoring has been added when we started working on the UI. Even though it "pollutes" the API, it turns out that it often a good way to be able to cancel a stuck download or also figure out where the system's at. Is this a matter of size? What is interesting about those 2 previous items is that it resemble the shape of the first prototype I did :) The API eventually evolved into what it is today because of the requirements of serviceability and usability. > IProfileEvent The profile event was meant as a mechanism to decouple some of the p2 entities and also provide for external monitoring. You can probably do without it if you had no external piece that needed to monitor the progress of the engine. I would really like to understand what you have in mind here? > IProfileRegistry Are you removing this because you don't want to store the profiles at all, or because you know you will only work against one profile? > ProfileScope This will likely be trickier to remove since this is used by the repository managers to store their repositories. Should the issue be with the dependency on eclipse.preferences, I'm sure we can find a way to replace the equinox.preferences. > ISizingPhaseSet This interface, as well as most of the implementation of the phases can be moved out of the engine bundle. [...] > We have an idea about refactoring touchpoints as OSGi services which are > standard instead of using eclipse specific extension points. We'd like to > hear your opinion about that. The actions and touchpoints are all encapsulated in the ActionManager and the TouchpointManager. The intent was for those to be "replaceable" and allow for actions and touchpoints to be contributed differently. I do not see any particular difficulty with this. Again should the goal be to remove the usage of the extension registry, then you will need something similar in the metadata repository bundle. >Modified in that way p2 engine depends only on (no other transitive >dependencies): > org.eclipse.equinox.p2.core > org.eclipse.equinox.p2.metadata > org.eclipse.equinox.p2.repository >(Except OSGi framework, services and declarative services implementation) In introduction you were mentioning how you were worried about size and coupling, what is your targeted size? With these set of changes you definitely cut the dependencies from the engine itself, but similar dependencies are brought in from p2.repository (for example it depends on jobs, ECF and equinox.common). Also are you thinking of writing your own touchpoints or reusing some of the ones we already provide? What are the things you absolutely don't want to see and why? Hope this helped and I'm really looking forward to hear your answers. PaScaL _______________________________________________ p2-dev mailing list [email protected]<mailto:[email protected]> https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev
