Re: Adding OSGi support to Geronimo spec jars.
On May 25, 2010, at 3:17 AM, Rick McGuire wrote: > 2) The ProviderLocator. This is how APIs that need to resolve providers > access the registry information. The provider registry is an OSGi service, so > the ProviderLocator needs a bundle context instance to resolve the service > instance. The Activator manages obtaining the bundle context at activation > and setting up a service tracker to give access to the registry service. The > ProviderLocator code manages the details of locating a loading a service > instance and will revert to classic classpath behavior if used outside of an > OSGi environment. On this note, I'm trying to determine what the performance impact might be in a large OSGi environment. Looks like the Activator always enables a ServiceTracker and that tracker appears to look for the ProviderRegistry service on every call. Is that a correct and do you have any idea what the general contract on ServiceTracker.getService() might be? Not sure if that's a quick call or might slow down the more bundles you have. -David
Re: Adding OSGi support to Geronimo spec jars.
Hey I got an idea. Since we're already modding spec jars, some of which will have to be installed into the endorsed dir directory anyway, Why don't we just mod the java.util.ServiceLoader class to do this optional OSGi search? You know, the whole one to rule them all approach. Then we don't have to mod each spec jar and it would work even for SAXParserFactory and other Java SE searches. And people could opt-in. "If you want this functionality, install this jar to your endorsed dir" -David On May 25, 2010, at 3:17 AM, Rick McGuire wrote: > On 5/24/2010 8:06 PM, David Blevins wrote: >> On May 24, 2010, at 3:49 PM, David Jencks wrote: >> >> >>> On May 24, 2010, at 2:18 PM, David Blevins wrote: >>> >>> 1. Using EJB as an example, how does one say "I am a provider". There is no "i am the EJB container" interface to implement so what exactly are we looking for? >>> >>> EJB is not an example. the provider stuff works for 2 situations: >>> >> The activator and locator were added to the EJB spec jar. Was that a >> mistake? >> > I don't believe it was. There are two sides to this generally. The first side > is the role played by the specs, which generally need to locate a provider. > Typically, the specification defines a search path that generally searches > for 1) a META-INF/services definition, 2) a class defined in a properties > file, and 3) a system property. Depending on the age of the specification, > the order in which these 3 are searched will vary, and the newest java ee > components only define step 1. Generally, this search order will resolve to > the name of a provider class that the API code will then instantiate to > create the provider instance. > > The second role is that of the provider class itself. Typically, a provider > jar would use the META-INF/services mechanism to advertise its availability. > The java EE defined mechanism for locating the provider is to search the > classpath for an appropriate META-INF/services file and load the first > definition it locates. > > In an OSGi environment, there are a number of problems associated with the > "search the classpath" step, so the ProviderLocator was created. There are > two pieces to this: > > 1) A provider registry. This is an extender that watches new bundles being > activated, and if the bundle contains the appropriate metadata, then then the > services it exports are made part of a framework-wide registry. > > 2) The ProviderLocator. This is how APIs that need to resolve providers > access the registry information. The provider registry is an OSGi service, so > the ProviderLocator needs a bundle context instance to resolve the service > instance. The Activator manages obtaining the bundle context at activation > and setting up a service tracker to give access to the registry service. The > ProviderLocator code manages the details of locating a loading a service > instance and will revert to classic classpath behavior if used outside of an > OSGi environment. > > There are two different models in play here for locating a provider instance. > In the first model, the API has been handed the name of the desired provider > directly as a class name. The API could would then attempt to instantiate > that class directly (typically by using the thread context classloader). In > the second model, an appropriate META-INF/services definition is located > using the interface that the provider is expected to implement as the search > key. The services definition file provides a mapping to the provider class > name, which is then loaded and instantiated. In terms of my search paths > defined in the first paragraph above, 1) is the META-INF/services mode, and > 2) and 3) are the first mode of direct loading. > > The provider registry can handle either of these locator modes. If your > provider is already using a META-INF/services model, then the registry then > generally all you need to do is add an SPI-Provider header to your jar file. > Openejb should have this already, but the EJB3 spec code is not strictly > following the EJB specification for locating the provider, so it is possible > it is not. > > If your provider needs to be located via a direct load mechanism where the > class name is provided, then you can use an Export-Service-Provider header to > advertise your provider class so the ProviderLocator code can locate it by > name. > > So, what should the openejb code be doing? The best solution would be to > create the META-INF/services file that maps EJBContainerProvider to your > implementation class and add the SPI-Provider header. Unfortunately, there is > a gotcha here, in the form of this Jira: > > https://issues.apache.org/jira/browse/GERONIMO-5186 > > The EJB specification says the EJBContainerProvider lookup mechanism should > use only the META-INF/services mechanism, and for each definition if finds, > it should
Re: Adding OSGi support to Geronimo spec jars.
On 5/24/2010 8:06 PM, David Blevins wrote: On May 24, 2010, at 3:49 PM, David Jencks wrote: On May 24, 2010, at 2:18 PM, David Blevins wrote: 1. Using EJB as an example, how does one say "I am a provider". There is no "i am the EJB container" interface to implement so what exactly are we looking for? EJB is not an example. the provider stuff works for 2 situations: The activator and locator were added to the EJB spec jar. Was that a mistake? I don't believe it was. There are two sides to this generally. The first side is the role played by the specs, which generally need to locate a provider. Typically, the specification defines a search path that generally searches for 1) a META-INF/services definition, 2) a class defined in a properties file, and 3) a system property. Depending on the age of the specification, the order in which these 3 are searched will vary, and the newest java ee components only define step 1. Generally, this search order will resolve to the name of a provider class that the API code will then instantiate to create the provider instance. The second role is that of the provider class itself. Typically, a provider jar would use the META-INF/services mechanism to advertise its availability. The java EE defined mechanism for locating the provider is to search the classpath for an appropriate META-INF/services file and load the first definition it locates. In an OSGi environment, there are a number of problems associated with the "search the classpath" step, so the ProviderLocator was created. There are two pieces to this: 1) A provider registry. This is an extender that watches new bundles being activated, and if the bundle contains the appropriate metadata, then then the services it exports are made part of a framework-wide registry. 2) The ProviderLocator. This is how APIs that need to resolve providers access the registry information. The provider registry is an OSGi service, so the ProviderLocator needs a bundle context instance to resolve the service instance. The Activator manages obtaining the bundle context at activation and setting up a service tracker to give access to the registry service. The ProviderLocator code manages the details of locating a loading a service instance and will revert to classic classpath behavior if used outside of an OSGi environment. There are two different models in play here for locating a provider instance. In the first model, the API has been handed the name of the desired provider directly as a class name. The API could would then attempt to instantiate that class directly (typically by using the thread context classloader). In the second model, an appropriate META-INF/services definition is located using the interface that the provider is expected to implement as the search key. The services definition file provides a mapping to the provider class name, which is then loaded and instantiated. In terms of my search paths defined in the first paragraph above, 1) is the META-INF/services mode, and 2) and 3) are the first mode of direct loading. The provider registry can handle either of these locator modes. If your provider is already using a META-INF/services model, then the registry then generally all you need to do is add an SPI-Provider header to your jar file. Openejb should have this already, but the EJB3 spec code is not strictly following the EJB specification for locating the provider, so it is possible it is not. If your provider needs to be located via a direct load mechanism where the class name is provided, then you can use an Export-Service-Provider header to advertise your provider class so the ProviderLocator code can locate it by name. So, what should the openejb code be doing? The best solution would be to create the META-INF/services file that maps EJBContainerProvider to your implementation class and add the SPI-Provider header. Unfortunately, there is a gotcha here, in the form of this Jira: https://issues.apache.org/jira/browse/GERONIMO-5186 The EJB specification says the EJBContainerProvider lookup mechanism should use only the META-INF/services mechanism, and for each definition if finds, it should instantiate the service instance and call createEJBContainer() passing the property bundle that was passed to EJBContainer.createEJBContainer(). The provider instance is then responsible for determining if it is the appropriate provider by examining the provider name in the properties bundle. Here's the wording from the spec: The EJBContainer bootstrap class will locate all of the container providers by their provider configuration files and call EJBContainerProvider.createEJBContainer(Map) on them in turn until an appropriate backing provider returns an EJBContainer. A provider may deem itself as appropriate for the embeddable application if any of the following are true : •The javax.ejb.embeddable.provider property
Re: Adding OSGi support to Geronimo spec jars.
On 5/24/2010 8:16 PM, David Jencks wrote: On May 24, 2010, at 5:06 PM, David Blevins wrote: On May 24, 2010, at 3:49 PM, David Jencks wrote: On May 24, 2010, at 2:18 PM, David Blevins wrote: 1. Using EJB as an example, how does one say "I am a provider". There is no "i am the EJB container" interface to implement so what exactly are we looking for? EJB is not an example. the provider stuff works for 2 situations: The activator and locator were added to the EJB spec jar. Was that a mistake? I was wrong... thinking of ejb 3. You need a META-INF/services file for EJBContainerProvider and the SPI-Provider header in the manifest. EJB 3.1 is an example. BTW the only way I've talked the maven-bundle-plugin into adding this header is to supply a value such astrue Anyone know a value-free way? I encountered the same problem and arrived at the same solution point you did. I never found any means of getting the header included without a value specified. Rick thanks david jencks -David
Re: Adding OSGi support to Geronimo spec jars.
On May 24, 2010, at 5:06 PM, David Blevins wrote: > > On May 24, 2010, at 3:49 PM, David Jencks wrote: > >> >> On May 24, 2010, at 2:18 PM, David Blevins wrote: >> >>> 1. Using EJB as an example, how does one say "I am a provider". There is >>> no "i am the EJB container" interface to implement so what exactly are we >>> looking for? >> >> >> EJB is not an example. the provider stuff works for 2 situations: > > The activator and locator were added to the EJB spec jar. Was that a mistake? I was wrong... thinking of ejb 3. You need a META-INF/services file for EJBContainerProvider and the SPI-Provider header in the manifest. EJB 3.1 is an example. BTW the only way I've talked the maven-bundle-plugin into adding this header is to supply a value such as true Anyone know a value-free way? thanks david jencks > > > -David >
Re: Adding OSGi support to Geronimo spec jars.
On May 24, 2010, at 3:49 PM, David Jencks wrote: > > On May 24, 2010, at 2:18 PM, David Blevins wrote: > >> 1. Using EJB as an example, how does one say "I am a provider". There is no >> "i am the EJB container" interface to implement so what exactly are we >> looking for? > > > EJB is not an example. the provider stuff works for 2 situations: The activator and locator were added to the EJB spec jar. Was that a mistake? -David
Re: Adding OSGi support to Geronimo spec jars.
On May 24, 2010, at 2:18 PM, David Blevins wrote: > Having a look at this code and trying to figure out what the impact is on > non-Geronimo consumers who might be using OSGi. > > The geronimo-osgi-registry seems fairly small (just 10 classes including the > inner classes) and only using the OSGi core and compendium jars. > > I grocked from the description that it is used to look for a provider of the > related API jar, but didn't get some specifics. > > 1. Using EJB as an example, how does one say "I am a provider". There is no > "i am the EJB container" interface to implement so what exactly are we > looking for? EJB is not an example. the provider stuff works for 2 situations: 1. Service providers, where you have a META-INF/services/ file containing one line, the name of an implementing class in the current jar (bundle) In this case, put a SPI-Provider manifest header in and your class will be accessible to service consumers via ProviderLocator.getServices(...) and related methods. 2. you need to make a class from your bundle available to others without them knowing about or getting inside your bundle. Put a Export-SPI-provider header in your manifest whose value is a comma separated list of the classes you want to make available. Consumers can load the classes by calling ProviderLocator.loadClass(className); > > 2. What is the effect if the Activator is activated and no provider is found? There are 2 activators. I guess you mean the one in provider-locator? If it isn't found, then you fall back on normal classloader tricks for trying to find services classes. These won't normally work in an osgi environment. thanks david jencks > > > Any insight? > > > -David > > > On Feb 26, 2010, at 5:35 AM, Rick McGuire wrote: > >> I've been taking a hard look at what Servicemix had done to the various spec >> jars to make them better behaved in an OSGi environment. This is being done >> with the intent of adding similar support to the base Geronimo spec jars. >> I'm taking a fresh approach to this rather than necessarily just copying >> what Servicemix is doing. I've found a number of interesting things during >> this process, so I thought it would be good to do a brain dump of what I've >> found and how I'm planning on implementing this. >> >> The basics of the Servicemix approach is to add an Activator to each of the >> spec bundle that maintains a registry of factory class information. Each >> spec bundle have a listener that tracks bundle activity and will check for >> factory information in the META-INF/services directory of each started >> bundle. Each started bundle has its own listener and own copy of the >> factory information. The registry information is used in the various places >> spec code needs to dynamically load provider classes for different >> subsystems. For example, loading a persistence provider. All classloading >> is done lazily when a request is made for a matching class file. The spec >> code retrieves the loaded classes and handles all details of creating the >> instances using the retrieved classes. >> >> Jarek Gawor suggested I might want to take a look at what the Aries project >> had for processing the META-INF/services information. This test >> implementation, called "spifly", uses the OSGi extender pattern to inspect >> the META-INF/services directories and uses that information to automatically >> register services in the OSGi services registry. In this situation, the >> classes are eagerly loaded, instances are created (which requires a >> no-argument constructor) and the services are registered in the OSGi >> registry. >> >> So, we have one set of information, but two different interpretations of how >> this information should be used. The new implementation I was working on >> was using the extender pattern to maintain a single registry of this >> information that could be accessed using a provider registry service. This >> would have a single listener, with a single version of the registry, and >> each bundle that required the service would just have a thin accessor layer >> to call the registry service if it was available. This is essentially >> combining the approaches used by Servicemix and spifly. >> >> However, I was becoming increasingly concerned about this dual >> interpretation of the META-INF/services information, and started researching >> what conventions were in play with this. What I found was there is a new >> feature in Java SE 6 called the ServiceLoader: >> >> http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html >> >> The service loader uses the META-INF/services information to create mappings >> between interface classes and concrete implementations of these interfaces. >> This is similar to the spifly approach, but there are a few fundamental >> differences. The biggest difference is that each instance of the >> ServiceLoader class will insta
Re: Adding OSGi support to Geronimo spec jars.
Having a look at this code and trying to figure out what the impact is on non-Geronimo consumers who might be using OSGi. The geronimo-osgi-registry seems fairly small (just 10 classes including the inner classes) and only using the OSGi core and compendium jars. I grocked from the description that it is used to look for a provider of the related API jar, but didn't get some specifics. 1. Using EJB as an example, how does one say "I am a provider". There is no "i am the EJB container" interface to implement so what exactly are we looking for? 2. What is the effect if the Activator is activated and no provider is found? Any insight? -David On Feb 26, 2010, at 5:35 AM, Rick McGuire wrote: > I've been taking a hard look at what Servicemix had done to the various spec > jars to make them better behaved in an OSGi environment. This is being done > with the intent of adding similar support to the base Geronimo spec jars. > I'm taking a fresh approach to this rather than necessarily just copying what > Servicemix is doing. I've found a number of interesting things during this > process, so I thought it would be good to do a brain dump of what I've found > and how I'm planning on implementing this. > > The basics of the Servicemix approach is to add an Activator to each of the > spec bundle that maintains a registry of factory class information. Each > spec bundle have a listener that tracks bundle activity and will check for > factory information in the META-INF/services directory of each started > bundle. Each started bundle has its own listener and own copy of the factory > information. The registry information is used in the various places spec > code needs to dynamically load provider classes for different subsystems. > For example, loading a persistence provider. All classloading is done lazily > when a request is made for a matching class file. The spec code retrieves > the loaded classes and handles all details of creating the instances using > the retrieved classes. > > Jarek Gawor suggested I might want to take a look at what the Aries project > had for processing the META-INF/services information. This test > implementation, called "spifly", uses the OSGi extender pattern to inspect > the META-INF/services directories and uses that information to automatically > register services in the OSGi services registry. In this situation, the > classes are eagerly loaded, instances are created (which requires a > no-argument constructor) and the services are registered in the OSGi registry. > > So, we have one set of information, but two different interpretations of how > this information should be used. The new implementation I was working on was > using the extender pattern to maintain a single registry of this information > that could be accessed using a provider registry service. This would have a > single listener, with a single version of the registry, and each bundle that > required the service would just have a thin accessor layer to call the > registry service if it was available. This is essentially combining the > approaches used by Servicemix and spifly. > > However, I was becoming increasingly concerned about this dual interpretation > of the META-INF/services information, and started researching what > conventions were in play with this. What I found was there is a new feature > in Java SE 6 called the ServiceLoader: > > http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html > > The service loader uses the META-INF/services information to create mappings > between interface classes and concrete implementations of these interfaces. > This is similar to the spifly approach, but there are a few fundamental > differences. The biggest difference is that each instance of the > ServiceLoader class will instantiate a new instance of the implementation > class when needed. For spifly, there is only ever a single instance of the > service created. Both spifly and Servicemix are only processing the first > line of the services files, while the ServiceLoader defines that an > individual definition file can define a one-to-many interface/implementation > mapping. So, now we're up to 3 different interpretations of the > META-INF/services information. > > Looking a little deeper into how Servicemix was using this information, I > found that it was bending the intent of the META-INF/services information a > bit. The ServiceLoader definitions are intended to create mappings between > interface classes and implementers of a given interface. The service mix > lookups were being used to directly resolve implementation classes. To do > this, the service definition file would need to use the same class as both > interface name and implementer class. This has a nice side effect of > allowing particular implementations to be selectively replaced, but this is a > usage that could cause problems if the informatio
Re: Adding OSGi support to Geronimo spec jars.
If you're working on a fresh implementation, I have some thoughts I gathered in the past months when writing / using the ServiceMix Specs. A few shortcomings of the current implementation are: * a given bundle does not have a way to select a particular implementation, i think we should define a header that may allow customizing the provider used for this very bundle * there is currently no isolation between the client and the provider. I think we should proxy the provider so that if it the bundle goes away, the implementation could be replaced underneath * specify in which order the provider is selected. One possible solution would be to default to the provider with the lowest bundle id (which is something used in several places in OSGi). This must be specified and easy to deal with (i.e. the user must be able to easily control which one will be used). Just my 2 cents. On Fri, Feb 26, 2010 at 14:35, Rick McGuire wrote: > I've been taking a hard look at what Servicemix had done to the various spec > jars to make them better behaved in an OSGi environment. This is being done > with the intent of adding similar support to the base Geronimo spec jars. > I'm taking a fresh approach to this rather than necessarily just copying > what Servicemix is doing. I've found a number of interesting things during > this process, so I thought it would be good to do a brain dump of what I've > found and how I'm planning on implementing this. > > The basics of the Servicemix approach is to add an Activator to each of the > spec bundle that maintains a registry of factory class information. Each > spec bundle have a listener that tracks bundle activity and will check for > factory information in the META-INF/services directory of each started > bundle. Each started bundle has its own listener and own copy of the > factory information. The registry information is used in the various places > spec code needs to dynamically load provider classes for different > subsystems. For example, loading a persistence provider. All classloading > is done lazily when a request is made for a matching class file. The spec > code retrieves the loaded classes and handles all details of creating the > instances using the retrieved classes. > > Jarek Gawor suggested I might want to take a look at what the Aries project > had for processing the META-INF/services information. This test > implementation, called "spifly", uses the OSGi extender pattern to inspect > the META-INF/services directories and uses that information to automatically > register services in the OSGi services registry. In this situation, the > classes are eagerly loaded, instances are created (which requires a > no-argument constructor) and the services are registered in the OSGi > registry. > > So, we have one set of information, but two different interpretations of how > this information should be used. The new implementation I was working on > was using the extender pattern to maintain a single registry of this > information that could be accessed using a provider registry service. This > would have a single listener, with a single version of the registry, and > each bundle that required the service would just have a thin accessor layer > to call the registry service if it was available. This is essentially > combining the approaches used by Servicemix and spifly. > > However, I was becoming increasingly concerned about this dual > interpretation of the META-INF/services information, and started researching > what conventions were in play with this. What I found was there is a new > feature in Java SE 6 called the ServiceLoader: > > http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html > > The service loader uses the META-INF/services information to create mappings > between interface classes and concrete implementations of these interfaces. > This is similar to the spifly approach, but there are a few fundamental > differences. The biggest difference is that each instance of the > ServiceLoader class will instantiate a new instance of the implementation > class when needed. For spifly, there is only ever a single instance of the > service created. Both spifly and Servicemix are only processing the first > line of the services files, while the ServiceLoader defines that an > individual definition file can define a one-to-many interface/implementation > mapping. So, now we're up to 3 different interpretations of the > META-INF/services information. > > Looking a little deeper into how Servicemix was using this information, I > found that it was bending the intent of the META-INF/services information a > bit. The ServiceLoader definitions are intended to create mappings between > interface classes and implementers of a given interface. The service mix > lookups were being used to directly resolve implementation classes. To do > this, the service definition file would need to use the same class as both > interface name and implemen
Adding OSGi support to Geronimo spec jars.
I've been taking a hard look at what Servicemix had done to the various spec jars to make them better behaved in an OSGi environment. This is being done with the intent of adding similar support to the base Geronimo spec jars. I'm taking a fresh approach to this rather than necessarily just copying what Servicemix is doing. I've found a number of interesting things during this process, so I thought it would be good to do a brain dump of what I've found and how I'm planning on implementing this. The basics of the Servicemix approach is to add an Activator to each of the spec bundle that maintains a registry of factory class information. Each spec bundle have a listener that tracks bundle activity and will check for factory information in the META-INF/services directory of each started bundle. Each started bundle has its own listener and own copy of the factory information. The registry information is used in the various places spec code needs to dynamically load provider classes for different subsystems. For example, loading a persistence provider. All classloading is done lazily when a request is made for a matching class file. The spec code retrieves the loaded classes and handles all details of creating the instances using the retrieved classes. Jarek Gawor suggested I might want to take a look at what the Aries project had for processing the META-INF/services information. This test implementation, called "spifly", uses the OSGi extender pattern to inspect the META-INF/services directories and uses that information to automatically register services in the OSGi services registry. In this situation, the classes are eagerly loaded, instances are created (which requires a no-argument constructor) and the services are registered in the OSGi registry. So, we have one set of information, but two different interpretations of how this information should be used. The new implementation I was working on was using the extender pattern to maintain a single registry of this information that could be accessed using a provider registry service. This would have a single listener, with a single version of the registry, and each bundle that required the service would just have a thin accessor layer to call the registry service if it was available. This is essentially combining the approaches used by Servicemix and spifly. However, I was becoming increasingly concerned about this dual interpretation of the META-INF/services information, and started researching what conventions were in play with this. What I found was there is a new feature in Java SE 6 called the ServiceLoader: http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html The service loader uses the META-INF/services information to create mappings between interface classes and concrete implementations of these interfaces. This is similar to the spifly approach, but there are a few fundamental differences. The biggest difference is that each instance of the ServiceLoader class will instantiate a new instance of the implementation class when needed. For spifly, there is only ever a single instance of the service created. Both spifly and Servicemix are only processing the first line of the services files, while the ServiceLoader defines that an individual definition file can define a one-to-many interface/implementation mapping. So, now we're up to 3 different interpretations of the META-INF/services information. Looking a little deeper into how Servicemix was using this information, I found that it was bending the intent of the META-INF/services information a bit. The ServiceLoader definitions are intended to create mappings between interface classes and implementers of a given interface. The service mix lookups were being used to directly resolve implementation classes. To do this, the service definition file would need to use the same class as both interface name and implementer class. This has a nice side effect of allowing particular implementations to be selectively replaced, but this is a usage that could cause problems if the information was picked up by either spifly or ServiceLoader. This violated the fundamental assumption that this information defined interface-to-implementation mappings. In addition, the javamail changes were using this information to define protocol-to-provider mappings. For example, an "smtp" javamail provider implementation class. In this case, the mapping did not even start with the name of a Java class. This definitely conflicted with both spifly and ServiceLoader. A lot of these difficulties go away if I decouple the Servicemix semantics by moving the information to a different location so that we're not seeing multiple interpretations of what the data in META-INF/services means. The code I'm working on will be looking in OSGI-INF/providers, and the mapping information is defined in terms of a provider identifi
