Re: [VOTE] Felix framework 3.0.4 and releated subproject releases
+1 Carsten -- Carsten Ziegeler cziege...@apache.org
Re: Bundle cache changes
Hi On 05.10.2010 15:13, Richard S. Hall wrote: > On 10/5/10 2:59, Felix Meschberger wrote: >> Hi, >> >> On 04.10.2010 22:26, Richard S. Hall wrote: >>> On 10/4/10 12:31, Pierre De Rop wrote: This is surprising, because I double checked (I did the same kind of tests of differents machines), and I always see that loading a populated cache is a bit longer than an empty cache. Ok, it's confusing for now, I will do further investigation and will get back to you later, if I find something. >>> Let's keep investigating this. >>> >>> In the meantime, I decided to rollback the change for the next release >>> so we can have more time to investigate it because right now I need to >>> get a bug fix out. So, we won't introduce any bundle cache structure >>> changes in 3.0.4, but maybe in 3.0.5. >> This sounds like a minor-release-number-worthy change ;-) > > I'm not sure that's necessary. It is still backward compatible, but it > isn't forward compatible. This means: once you ran the framework with the new bundle cache and installed and/or updated bundles you cannot run the existing cache with an older version of the framework any longer. Right ? In this case, IMHO a minor version number increase might provide a hint at this situation. Regards Felix > > -> richard > >> Regards >> Felix >> >>> -> richard >>> On Mon, Oct 4, 2010 at 5:05 PM, Richard S. Hallwrote: >On 10/4/10 10:52, Richard S. Hall wrote: > >> Thanks for the installer bundle, I did some tests locally on my Mac >> using >> 230 bundles from GlassFish, this is what I see (note: the "purge" >> command >> flushes disk buffers): >> >> Empty cache >> - >> [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar >> bin/felix.jar >> Mon Oct 4 10:43:09 EDT 2010 >> 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for >> FrameworkEvent.STARTED event ... >> 2010-10-04 10:43:24,822 - Framework started: Installing bundles from >> ./load dir >> 2010-10-04 10:43:54,464 - Done. >> >> Populated cache >> --- >> [heavyweight main]$ purge; date; java -jar bin/felix.jar >> Mon Oct 4 10:44:49 EDT 2010 >> 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for >> FrameworkEvent.STARTED event ... >> 2010-10-04 10:45:13,340 - Framework started: Installing bundles from >> ./load dir >> 2010-10-04 10:45:13,342 - Done. >> >> You can see that the empty cache scenario takes about 45 seconds, >> whereas >> the populated cache takes about 24 seconds. So, on my machine, it >> takes a >> lot less time to re-load cached bundles...could be a function of my >> disk >> speed, I suppose, but it is what I would expect. >> >> Not sure how to proceed, other than trying to get more samples. >> > Interestingly, I tried the same test using the 3.0.3 release (the > above was > with trunk) and I got the follow results: > > Empty cache (3.0.3) > - > > rm -rf felix-cache; purge; date; java -jar bin/felix.jar > Mon Oct 4 10:58:23 EDT 2010 > 2010-10-04 10:58:43,340 - BundleInstaller starting and waiting for > FrameworkEvent.STARTED event ... > 2010-10-04 10:58:43,557 - Framework started: Installing bundles from > ./load > dir > 2010-10-04 10:59:03,373 - Done. > > Populated cache (3.0.3) > --- > > purge; date; java -jar bin/felix.jar > Mon Oct 4 10:59:50 EDT 2010 > 2010-10-04 11:00:24,738 - BundleInstaller starting and waiting for > FrameworkEvent.STARTED event ... > 2010-10-04 11:00:26,736 - Framework started: Installing bundles from > ./load > dir > 2010-10-04 11:00:26,738 - Done. > > So, the populated cache takes nearly as long as the empty cache (36 > seconds > compared to 40 seconds)...this also makes sense, since the trunk has > patches > specifically designed to improve re-loading cached bundles. > > -> richard > > > >>>On Fri, Oct 1, 2010 at 11:30 PM, wrote: >On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop< > pierre.de...@gmail.com> > >> wrote: >> >>On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall >>> wrote: >>> >>> One thing, was your performance comparison based on using >>> the new >>> cache >>>or >>> old? I assume you are using a newly created cache for the framework snapshot and [obviously] an old cache for the 3.0.2, correct? yes: >>> old cache with 3.0.2, and new cache with trunk. >>> but I also checked that new fwk/trunk is properly working when >
[jira] Issue Comment Edited: (FELIX-2572) JRE system packages should include "uses" constraints
[ https://issues.apache.org/jira/browse/FELIX-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918154#action_12918154 ] Richard S. Hall edited comment on FELIX-2572 at 10/5/10 4:46 PM: - I've taken the attached example JRE metadata and used it to create the system bundle exports. The resulting framework still passed the OSGi CT, our internal tests, and the GlassFish test suite. So, it definitely seems like we could do this, we just have to decide if we should. It could definitely help in situations where bundles are trying to provide alternative versions of JRE supplied packages. This situation currently causes lots of issues since to the resolver system bundle packages look completely unconstrained since they lack all "uses" constraints. I am leaning in favor of adding this metadata. Thoughts? was (Author: rickhall): I taken the attached example JRE metadata and used it to create the system bundle exports. The resulting framework still passed the OSGi CT, our internal tests, and the GlassFish test suite. So, it definitely seems like we could do this, we just have to decide if we should. It could definitely help in situations where bundles are trying to provide alternative versions of JRE supplied packages. This situation currently causes lots of issues since to the resolver system bundle packages look completely unconstrained since they lack all "uses" constraints. I am leaning in favor of adding this metadata. Thoughts? > JRE system packages should include "uses" constraints > - > > Key: FELIX-2572 > URL: https://issues.apache.org/jira/browse/FELIX-2572 > Project: Felix > Issue Type: Improvement > Components: Framework >Affects Versions: framework-3.0.2 >Reporter: Richard S. Hall >Assignee: Richard S. Hall >Priority: Minor > Fix For: framework-3.2.0 > > Attachments: jre-package-linux.txt > > > The framework is configured by default to export all JRE packages. Currently, > this doesn't include "uses" constraints, which can lead to resolutions that > result in execution-time issues (e.g., LinkageErrors) that are hard to > diagnose. If we include "uses" constraints on the system packages, then we > can avoid this. We should be able to use BND to generate this information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (FELIX-2572) JRE system packages should include "uses" constraints
[ https://issues.apache.org/jira/browse/FELIX-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918154#action_12918154 ] Richard S. Hall commented on FELIX-2572: I taken the attached example JRE metadata and used it to create the system bundle exports. The resulting framework still passed the OSGi CT, our internal tests, and the GlassFish test suite. So, it definitely seems like we could do this, we just have to decide if we should. It could definitely help in situations where bundles are trying to provide alternative versions of JRE supplied packages. This situation currently causes lots of issues since to the resolver system bundle packages look completely unconstrained since they lack all "uses" constraints. I am leaning in favor of adding this metadata. Thoughts? > JRE system packages should include "uses" constraints > - > > Key: FELIX-2572 > URL: https://issues.apache.org/jira/browse/FELIX-2572 > Project: Felix > Issue Type: Improvement > Components: Framework >Affects Versions: framework-3.0.2 >Reporter: Richard S. Hall >Assignee: Richard S. Hall >Priority: Minor > Fix For: framework-3.2.0 > > Attachments: jre-package-linux.txt > > > The framework is configured by default to export all JRE packages. Currently, > this doesn't include "uses" constraints, which can lead to resolutions that > result in execution-time issues (e.g., LinkageErrors) that are hard to > diagnose. If we include "uses" constraints on the system packages, then we > can avoid this. We should be able to use BND to generate this information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (FELIX-2572) JRE system packages should include "uses" constraints
[ https://issues.apache.org/jira/browse/FELIX-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard S. Hall reassigned FELIX-2572: -- Assignee: Richard S. Hall > JRE system packages should include "uses" constraints > - > > Key: FELIX-2572 > URL: https://issues.apache.org/jira/browse/FELIX-2572 > Project: Felix > Issue Type: Improvement > Components: Framework >Affects Versions: framework-3.0.2 >Reporter: Richard S. Hall >Assignee: Richard S. Hall >Priority: Minor > Fix For: framework-3.2.0 > > Attachments: jre-package-linux.txt > > > The framework is configured by default to export all JRE packages. Currently, > this doesn't include "uses" constraints, which can lead to resolutions that > result in execution-time issues (e.g., LinkageErrors) that are hard to > diagnose. If we include "uses" constraints on the system packages, then we > can avoid this. We should be able to use BND to generate this information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (FELIX-2572) JRE system packages should include "uses" constraints
[ https://issues.apache.org/jira/browse/FELIX-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard S. Hall updated FELIX-2572: --- Attachment: jre-package-linux.txt Example JRE metadata generated using BND. > JRE system packages should include "uses" constraints > - > > Key: FELIX-2572 > URL: https://issues.apache.org/jira/browse/FELIX-2572 > Project: Felix > Issue Type: Improvement > Components: Framework >Affects Versions: framework-3.0.2 >Reporter: Richard S. Hall >Priority: Minor > Fix For: framework-3.2.0 > > Attachments: jre-package-linux.txt > > > The framework is configured by default to export all JRE packages. Currently, > this doesn't include "uses" constraints, which can lead to resolutions that > result in execution-time issues (e.g., LinkageErrors) that are hard to > diagnose. If we include "uses" constraints on the system packages, then we > can avoid this. We should be able to use BND to generate this information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Bundle cache changes
Thanks for the feedback. Just so you know you aren't crazy, I tested it on my EEE PC and I am seeing similar results as you. It is taking around 18 seconds with an empty cache and about 23 seconds with a populated cache. In the end, though, the patch does show improvement over the non-patched version, so it is probably still worthwhile...but very counter intuitive. -> richard On 10/5/10 12:21, Pierre De Rop wrote: Hello Richard, I progress on my testing, and I understand now why I did not get the same results than you ... I did two tests: one test on a machine with a relatively slow disk, and another test on a host whose disk is faster. I did the test with the 213 bundles from glassfish 3.0.1 (glassfishv3/glassfish/modules/*.jar) Test on the host with the slow disk: I first measured the time needed to open a single file when disk buffers are flushed: 32 millis. (I did this to check that the disk io speed). Here are the results of the tests: * with felix 3.0.3 -> time to load an empty felix cache (with disk buffers flushed) = 9.2 millis time to load an existing felix cache (with dsk buffers flushed) = 14.9 millis * with felix trunk (optimized version that you uncommitted yesterday) -> time to load an empty felix cache (with disk buffers flushed) = 9.1 millis time to load an existing felix cache (with dsk buffers flushed) = 12.3 millis (here, we notice that it takes a longer time to load an existing felix cache, than an empty one, either with felix 3.0.3 or optimized felix trunk). Test on the host with the faster disk: == Time needed to open a single file when disk buffers are flushed: 1 millis. * with felix 3.0.3 -> time to load an empty felix cache (with disk buffers flushed) = 8.1 millis time to load an existing felix cache (with dsk buffers flushed) = 5.3 millis * with felix trunk (optimized version that you recently uncommited) -> time to load an empty felix cache (with disk buffers flushed) = 7.9 millis time to load an existing felix cache (with dsk buffers flushed) = 4.7 millis (here, loading an existing felix cache is faster then an empty felix cache, as expected). So, in any cases, the optimized version of the trunk is faster than the 2.0.3. However, with a low disk IO rate, then it seems that it takes a longer time to load an existing felix cache than an empty one. /pierre On Tue, Oct 5, 2010 at 3:13 PM, Richard S. Hallwrote: On 10/5/10 2:59, Felix Meschberger wrote: Hi, On 04.10.2010 22:26, Richard S. Hall wrote: On 10/4/10 12:31, Pierre De Rop wrote: This is surprising, because I double checked (I did the same kind of tests of differents machines), and I always see that loading a populated cache is a bit longer than an empty cache. Ok, it's confusing for now, I will do further investigation and will get back to you later, if I find something. Let's keep investigating this. In the meantime, I decided to rollback the change for the next release so we can have more time to investigate it because right now I need to get a bug fix out. So, we won't introduce any bundle cache structure changes in 3.0.4, but maybe in 3.0.5. This sounds like a minor-release-number-worthy change ;-) I'm not sure that's necessary. It is still backward compatible, but it isn't forward compatible. -> richard Regards Felix -> richard On Mon, Oct 4, 2010 at 5:05 PM, Richard S. Hallwrote: On 10/4/10 10:52, Richard S. Hall wrote: Thanks for the installer bundle, I did some tests locally on my Mac using 230 bundles from GlassFish, this is what I see (note: the "purge" command flushes disk buffers): Empty cache - [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar bin/felix.jar Mon Oct 4 10:43:09 EDT 2010 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:43:24,822 - Framework started: Installing bundles from ./load dir 2010-10-04 10:43:54,464 - Done. Populated cache --- [heavyweight main]$ purge; date; java -jar bin/felix.jar Mon Oct 4 10:44:49 EDT 2010 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:45:13,340 - Framework started: Installing bundles from ./load dir 2010-10-04 10:45:13,342 - Done. You can see that the empty cache scenario takes about 45 seconds, whereas the populated cache takes about 24 seconds. So, on my machine, it takes a lot less time to re-load cached bundles...could be a function of my disk speed, I suppose, but it is what I would expect. Not sure how to proceed, other than trying to get more samples. Interestingly, I tried the same test using the 3.0.3 release (the above was with trunk) and I got the follow results: Empty cache (3.0.3) - rm -rf felix-cache; purge; date; java -jar bin/felix.jar Mon Oct 4 10:58:23 EDT 2010 2010-10-04 10:58:43,340 - BundleInstaller starting and waitin
Re: Bundle cache changes
(the results in the above post are in seconds, not in millis ...) On Tue, Oct 5, 2010 at 6:21 PM, Pierre De Rop wrote: > Hello Richard, > > I progress on my testing, and I understand now why I did not get the same > results than you ... > I did two tests: one test on a machine with a relatively slow disk, and > another test on a > host whose disk is faster. > > I did the test with the 213 bundles from glassfish 3.0.1 > (glassfishv3/glassfish/modules/*.jar) > > Test on the host with the slow disk: > > > I first measured the time needed to open a single file when disk buffers > are flushed: 32 millis. > (I did this to check that the disk io speed). > > Here are the results of the tests: > > * with felix 3.0.3 -> > time to load an empty felix cache (with disk buffers flushed) = 9.2 millis > time to load an existing felix cache (with dsk buffers flushed) = 14.9 > millis > > * with felix trunk (optimized version that you uncommitted yesterday) -> > time to load an empty felix cache (with disk buffers flushed) = 9.1 millis > time to load an existing felix cache (with dsk buffers flushed) = 12.3 > millis > > (here, we notice that it takes a longer time to load an existing felix > cache, than an empty one, either with felix 3.0.3 or optimized felix trunk). > > Test on the host with the faster disk: > == > > Time needed to open a single file when disk buffers are flushed: 1 millis. > > * with felix 3.0.3 -> > time to load an empty felix cache (with disk buffers flushed) = 8.1 millis > time to load an existing felix cache (with dsk buffers flushed) = 5.3 > millis > > * with felix trunk (optimized version that you recently uncommited) -> > time to load an empty felix cache (with disk buffers flushed) = 7.9 millis > time to load an existing felix cache (with dsk buffers flushed) = 4.7 > millis > > (here, loading an existing felix cache is faster then an empty felix cache, > as expected). > > So, in any cases, the optimized version of the trunk is faster than the > 2.0.3. > However, with a low disk IO rate, then it seems that it takes a longer time > to load an existing felix cache > than an empty one. > > > /pierre > > > > > > On Tue, Oct 5, 2010 at 3:13 PM, Richard S. Hall wrote: > >> On 10/5/10 2:59, Felix Meschberger wrote: >> >>> Hi, >>> >>> On 04.10.2010 22:26, Richard S. Hall wrote: >>> On 10/4/10 12:31, Pierre De Rop wrote: > This is surprising, because I double checked (I did the same kind of > tests > of differents machines), and I always see that loading a populated > cache is > a bit longer than an empty cache. > Ok, it's confusing for now, I will do further investigation and will > get > back to you later, if I find something. > Let's keep investigating this. In the meantime, I decided to rollback the change for the next release so we can have more time to investigate it because right now I need to get a bug fix out. So, we won't introduce any bundle cache structure changes in 3.0.4, but maybe in 3.0.5. >>> This sounds like a minor-release-number-worthy change ;-) >>> >> >> I'm not sure that's necessary. It is still backward compatible, but it >> isn't forward compatible. >> >> -> richard >> >> >> Regards >>> Felix >>> >>> -> richard On Mon, Oct 4, 2010 at 5:05 PM, Richard S. > Hallwrote: > >On 10/4/10 10:52, Richard S. Hall wrote: >> >> Thanks for the installer bundle, I did some tests locally on my Mac >>> using >>> 230 bundles from GlassFish, this is what I see (note: the "purge" >>> command >>> flushes disk buffers): >>> >>> Empty cache >>> - >>> [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar >>> bin/felix.jar >>> Mon Oct 4 10:43:09 EDT 2010 >>> 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for >>> FrameworkEvent.STARTED event ... >>> 2010-10-04 10:43:24,822 - Framework started: Installing bundles from >>> ./load dir >>> 2010-10-04 10:43:54,464 - Done. >>> >>> Populated cache >>> --- >>> [heavyweight main]$ purge; date; java -jar bin/felix.jar >>> Mon Oct 4 10:44:49 EDT 2010 >>> 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for >>> FrameworkEvent.STARTED event ... >>> 2010-10-04 10:45:13,340 - Framework started: Installing bundles from >>> ./load dir >>> 2010-10-04 10:45:13,342 - Done. >>> >>> You can see that the empty cache scenario takes about 45 seconds, >>> whereas >>> the populated cache takes about 24 seconds. So, on my machine, it >>> takes a >>> lot less time to re-load cached bundles...could be a function of my >>> disk >>> speed, I suppose, but it is what I would expect. >>> >>> Not sure how to proceed, other than trying to get more samples. >>> >>> Interestingly, I tried the same test us
Re: Bundle cache changes
Hello Richard, I progress on my testing, and I understand now why I did not get the same results than you ... I did two tests: one test on a machine with a relatively slow disk, and another test on a host whose disk is faster. I did the test with the 213 bundles from glassfish 3.0.1 (glassfishv3/glassfish/modules/*.jar) Test on the host with the slow disk: I first measured the time needed to open a single file when disk buffers are flushed: 32 millis. (I did this to check that the disk io speed). Here are the results of the tests: * with felix 3.0.3 -> time to load an empty felix cache (with disk buffers flushed) = 9.2 millis time to load an existing felix cache (with dsk buffers flushed) = 14.9 millis * with felix trunk (optimized version that you uncommitted yesterday) -> time to load an empty felix cache (with disk buffers flushed) = 9.1 millis time to load an existing felix cache (with dsk buffers flushed) = 12.3 millis (here, we notice that it takes a longer time to load an existing felix cache, than an empty one, either with felix 3.0.3 or optimized felix trunk). Test on the host with the faster disk: == Time needed to open a single file when disk buffers are flushed: 1 millis. * with felix 3.0.3 -> time to load an empty felix cache (with disk buffers flushed) = 8.1 millis time to load an existing felix cache (with dsk buffers flushed) = 5.3 millis * with felix trunk (optimized version that you recently uncommited) -> time to load an empty felix cache (with disk buffers flushed) = 7.9 millis time to load an existing felix cache (with dsk buffers flushed) = 4.7 millis (here, loading an existing felix cache is faster then an empty felix cache, as expected). So, in any cases, the optimized version of the trunk is faster than the 2.0.3. However, with a low disk IO rate, then it seems that it takes a longer time to load an existing felix cache than an empty one. /pierre On Tue, Oct 5, 2010 at 3:13 PM, Richard S. Hall wrote: > On 10/5/10 2:59, Felix Meschberger wrote: > >> Hi, >> >> On 04.10.2010 22:26, Richard S. Hall wrote: >> >>> On 10/4/10 12:31, Pierre De Rop wrote: >>> This is surprising, because I double checked (I did the same kind of tests of differents machines), and I always see that loading a populated cache is a bit longer than an empty cache. Ok, it's confusing for now, I will do further investigation and will get back to you later, if I find something. >>> Let's keep investigating this. >>> >>> In the meantime, I decided to rollback the change for the next release >>> so we can have more time to investigate it because right now I need to >>> get a bug fix out. So, we won't introduce any bundle cache structure >>> changes in 3.0.4, but maybe in 3.0.5. >>> >> This sounds like a minor-release-number-worthy change ;-) >> > > I'm not sure that's necessary. It is still backward compatible, but it > isn't forward compatible. > > -> richard > > > Regards >> Felix >> >> -> richard >>> >>> On Mon, Oct 4, 2010 at 5:05 PM, Richard S. Hallwrote: On 10/4/10 10:52, Richard S. Hall wrote: > > Thanks for the installer bundle, I did some tests locally on my Mac >> using >> 230 bundles from GlassFish, this is what I see (note: the "purge" >> command >> flushes disk buffers): >> >> Empty cache >> - >> [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar >> bin/felix.jar >> Mon Oct 4 10:43:09 EDT 2010 >> 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for >> FrameworkEvent.STARTED event ... >> 2010-10-04 10:43:24,822 - Framework started: Installing bundles from >> ./load dir >> 2010-10-04 10:43:54,464 - Done. >> >> Populated cache >> --- >> [heavyweight main]$ purge; date; java -jar bin/felix.jar >> Mon Oct 4 10:44:49 EDT 2010 >> 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for >> FrameworkEvent.STARTED event ... >> 2010-10-04 10:45:13,340 - Framework started: Installing bundles from >> ./load dir >> 2010-10-04 10:45:13,342 - Done. >> >> You can see that the empty cache scenario takes about 45 seconds, >> whereas >> the populated cache takes about 24 seconds. So, on my machine, it >> takes a >> lot less time to re-load cached bundles...could be a function of my >> disk >> speed, I suppose, but it is what I would expect. >> >> Not sure how to proceed, other than trying to get more samples. >> >> Interestingly, I tried the same test using the 3.0.3 release (the > above was > with trunk) and I got the follow results: > > Empty cache (3.0.3) > - > > rm -rf felix-cache; purge; date; java -jar bin/felix.jar > Mon Oct 4 10:58:23 EDT 2010 > 2010-10-04 10:58:43,340 - BundleInstaller starting and waiting
Re: [VOTE] Felix framework 3.0.4 and releated subproject releases
+1 -> richard On 10/4/10 16:44, Karl Pauls wrote: I would like to call a vote on the following subproject releases: framework 3.0.4 main 3.0.4 main.distribution 3.0.4 Staging repositories: https://repository.apache.org/content/repositories/orgapachefelix-001/ You can use this UNIX script to download the release and verify the signatures: http://svn.apache.org/repos/asf/felix/trunk/check_staged_release.sh Usage: sh check_staged_release.sh 001 /tmp/felix-staging Please vote to approve this release: [ ] +1 Approve the release [ ] -1 Veto the release (please provide specific comments)
Re: Bundle cache changes
On 10/5/10 2:59, Felix Meschberger wrote: Hi, On 04.10.2010 22:26, Richard S. Hall wrote: On 10/4/10 12:31, Pierre De Rop wrote: This is surprising, because I double checked (I did the same kind of tests of differents machines), and I always see that loading a populated cache is a bit longer than an empty cache. Ok, it's confusing for now, I will do further investigation and will get back to you later, if I find something. Let's keep investigating this. In the meantime, I decided to rollback the change for the next release so we can have more time to investigate it because right now I need to get a bug fix out. So, we won't introduce any bundle cache structure changes in 3.0.4, but maybe in 3.0.5. This sounds like a minor-release-number-worthy change ;-) I'm not sure that's necessary. It is still backward compatible, but it isn't forward compatible. -> richard Regards Felix -> richard On Mon, Oct 4, 2010 at 5:05 PM, Richard S. Hallwrote: On 10/4/10 10:52, Richard S. Hall wrote: Thanks for the installer bundle, I did some tests locally on my Mac using 230 bundles from GlassFish, this is what I see (note: the "purge" command flushes disk buffers): Empty cache - [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar bin/felix.jar Mon Oct 4 10:43:09 EDT 2010 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:43:24,822 - Framework started: Installing bundles from ./load dir 2010-10-04 10:43:54,464 - Done. Populated cache --- [heavyweight main]$ purge; date; java -jar bin/felix.jar Mon Oct 4 10:44:49 EDT 2010 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:45:13,340 - Framework started: Installing bundles from ./load dir 2010-10-04 10:45:13,342 - Done. You can see that the empty cache scenario takes about 45 seconds, whereas the populated cache takes about 24 seconds. So, on my machine, it takes a lot less time to re-load cached bundles...could be a function of my disk speed, I suppose, but it is what I would expect. Not sure how to proceed, other than trying to get more samples. Interestingly, I tried the same test using the 3.0.3 release (the above was with trunk) and I got the follow results: Empty cache (3.0.3) - rm -rf felix-cache; purge; date; java -jar bin/felix.jar Mon Oct 4 10:58:23 EDT 2010 2010-10-04 10:58:43,340 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:58:43,557 - Framework started: Installing bundles from ./load dir 2010-10-04 10:59:03,373 - Done. Populated cache (3.0.3) --- purge; date; java -jar bin/felix.jar Mon Oct 4 10:59:50 EDT 2010 2010-10-04 11:00:24,738 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 11:00:26,736 - Framework started: Installing bundles from ./load dir 2010-10-04 11:00:26,738 - Done. So, the populated cache takes nearly as long as the empty cache (36 seconds compared to 40 seconds)...this also makes sense, since the trunk has patches specifically designed to improve re-loading cached bundles. -> richard On Fri, Oct 1, 2010 at 11:30 PM, wrote: On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop< pierre.de...@gmail.com> wrote: On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall wrote: One thing, was your performance comparison based on using the new cache or old? I assume you are using a newly created cache for the framework snapshot and [obviously] an old cache for the 3.0.2, correct? yes: old cache with 3.0.2, and new cache with trunk. but I also checked that new fwk/trunk is properly working when being started with the old cache (from 3.0.2) You should also be testing an already created cache (i.e., a framework restart), not an empty cache. In the previous test, I compared with already created cache, not with empty caches. With empty caches, here are the results: - old fwk 3.0.2 / fresh empty cache -> 15.8 sec. (this is surprising: I would expect to get a longer delay, since the cache is empty when I start the fwk) 2010-10-01 23:10:17,403 Starting Felix 3.0.2 2010-10-01 23:10:23,885 Starting cluster node "test" 2010-10-01 23:10:33,208 Cluster node "test" ready - new fwk / fresh empty cache ->15.2 sec 2010-10-01 23:15:46,2 Starting Felix 3.1.0.SNAPSHOT 2010-10-01 23:15:52,417 Starting cluster node "test" 2010-10-01 23:16:01,225 Cluster node "test" ready hmmm, I don't understand why the fwk startup time is better when the cache is empty (15.2 sec) than when the cache is non-empty (21.2 sec from previous mail) Ok, I will do more tests ... may be I did something wrong ? ... to be continued ... Hmm. Yeah, that seems a little odd. In that case, you should just delete your cache each time to get better startup performance! ;-) Definitel
Re: Bundle cache changes
Hi, On 04.10.2010 22:26, Richard S. Hall wrote: > On 10/4/10 12:31, Pierre De Rop wrote: >> This is surprising, because I double checked (I did the same kind of >> tests >> of differents machines), and I always see that loading a populated >> cache is >> a bit longer than an empty cache. >> Ok, it's confusing for now, I will do further investigation and will get >> back to you later, if I find something. > > Let's keep investigating this. > > In the meantime, I decided to rollback the change for the next release > so we can have more time to investigate it because right now I need to > get a bug fix out. So, we won't introduce any bundle cache structure > changes in 3.0.4, but maybe in 3.0.5. This sounds like a minor-release-number-worthy change ;-) Regards Felix > > -> richard > >> >> On Mon, Oct 4, 2010 at 5:05 PM, Richard S. >> Hallwrote: >> >>> On 10/4/10 10:52, Richard S. Hall wrote: >>> Thanks for the installer bundle, I did some tests locally on my Mac using 230 bundles from GlassFish, this is what I see (note: the "purge" command flushes disk buffers): Empty cache - [heavyweight main]$ rm -rf felix-cache; purge; date; java -jar bin/felix.jar Mon Oct 4 10:43:09 EDT 2010 2010-10-04 10:43:24,541 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:43:24,822 - Framework started: Installing bundles from ./load dir 2010-10-04 10:43:54,464 - Done. Populated cache --- [heavyweight main]$ purge; date; java -jar bin/felix.jar Mon Oct 4 10:44:49 EDT 2010 2010-10-04 10:45:12,212 - BundleInstaller starting and waiting for FrameworkEvent.STARTED event ... 2010-10-04 10:45:13,340 - Framework started: Installing bundles from ./load dir 2010-10-04 10:45:13,342 - Done. You can see that the empty cache scenario takes about 45 seconds, whereas the populated cache takes about 24 seconds. So, on my machine, it takes a lot less time to re-load cached bundles...could be a function of my disk speed, I suppose, but it is what I would expect. Not sure how to proceed, other than trying to get more samples. >>> Interestingly, I tried the same test using the 3.0.3 release (the >>> above was >>> with trunk) and I got the follow results: >>> >>> Empty cache (3.0.3) >>> - >>> >>> rm -rf felix-cache; purge; date; java -jar bin/felix.jar >>> Mon Oct 4 10:58:23 EDT 2010 >>> 2010-10-04 10:58:43,340 - BundleInstaller starting and waiting for >>> FrameworkEvent.STARTED event ... >>> 2010-10-04 10:58:43,557 - Framework started: Installing bundles from >>> ./load >>> dir >>> 2010-10-04 10:59:03,373 - Done. >>> >>> Populated cache (3.0.3) >>> --- >>> >>> purge; date; java -jar bin/felix.jar >>> Mon Oct 4 10:59:50 EDT 2010 >>> 2010-10-04 11:00:24,738 - BundleInstaller starting and waiting for >>> FrameworkEvent.STARTED event ... >>> 2010-10-04 11:00:26,736 - Framework started: Installing bundles from >>> ./load >>> dir >>> 2010-10-04 11:00:26,738 - Done. >>> >>> So, the populated cache takes nearly as long as the empty cache (36 >>> seconds >>> compared to 40 seconds)...this also makes sense, since the trunk has >>> patches >>> specifically designed to improve re-loading cached bundles. >>> >>> -> richard >>> >>> >>> > On Fri, Oct 1, 2010 at 11:30 PM,wrote: >>> On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop< >>> pierre.de...@gmail.com> >>> wrote: On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall > wrote: > >One thing, was your performance comparison based on using > the new > cache > or > >> old? >> >> I assume you are using a newly created cache for the framework >> snapshot >> and >> [obviously] an old cache for the 3.0.2, correct? >> >> yes: >> > old cache with 3.0.2, and new cache with trunk. > but I also checked that new fwk/trunk is properly working when > being > started > with the old cache (from 3.0.2) > > You should also be testing an already created cache (i.e., a > framework > >> restart), not an empty cache. >> >> In the previous test, I compared with already created cache, >> not >> with >> > empty caches. > With empty caches, here are the results: > > > - old fwk 3.0.2 / fresh empty cache ->15.8 sec. (this is > surprising: > > I would expect to get a longer delay, since the cache is empty when > I > start > the fwk) > > 2010-10-01 23:10:17,403 Starting Felix 3.
