Well, I hope this subject does not get lost for the OSGi-dev list ...
I do think this is a very relevant subject for OSGi developments that
goes much wider than PDE, even though PDE is a primary player in
this area.
Kind regards,
Peter Kriens
JV> I'm including pde-build-dev in the To: list in hopes of moving
JV> this to a more proper place. Please send replies to that list. :)
JV> Jeff,
JV> I think for optimal performance, the build server would keep a
JV> local copy of the repository and simply perform an update on it,
JV> instead of checking out projects each time. Our current system
JV> keeps local checkouts of each project, but it does so in a way
JV> that requires per-project updating. The optimal way would be to
JV> tell the builder to checkout a particular subversion path ( e.g.
JV> ,http://svn.company.com/osgi-project/bundles) which contains some
JV> or all of the bundles within subdirectories (e.g.,
JV> http://svn.company.com/osgi-project/bundles/com.company.somebundle),
JV> and then determine which bundles are contained within, if any. If
JV> the bundles path contained 150 bundle subdirectories, it would be
JV> much faster to update the root bundles path than to update each of
JV> the 150 bundles paths individually.
JV> Perhaps our SVN environment is slow for some reason, or JavaSVN
JV> is slow, I haven't investigated very closely. At any rate, I
JV> would imagine an extended PDE Build would allow stuff like this to be
configurable.
JV> -Jeremy
JV> On 4/30/07, Jeff McAffer <[EMAIL PROTECTED]> wrote:
JV>
JV> There are a number of changes in 3.3 but fundamentally the
JV> structure is the same. As for your performance issue...
JV> Interesting. per-project check out with SVN is slow? that does
JV> not bode well for incremental building of only those parts that
JV> need to be built. The incremental approach seems to imply that
JV> you would only check out the projects that need building and those
JV> which are already built can just be used directly.
JV>
JV> In any event, with the new provisioning work that we are doing in
JV> Equinox we are looking to see if there are opportunities to
JV> improve the overall build story. Input from other people with
JV> realworld build problems (and solutions) would be great. That
JV> should likely go on the equinox-dev or pde-build-dev lists as appropriate.
JV>
JV> Jeff
JV>
JV>
JV>
JV>
JV> "Jeremy Volkman" <[EMAIL PROTECTED]>
JV> Sent by: [EMAIL PROTECTED]
JV> 04/30/2007 11:35 AM
JV>
JV> Please respond to
JV> OSGi Developer Mail List <[email protected]>
JV>
JV>
JV> To
JV> "OSGi Developer Mail List" <[email protected]>
JV> cc
JV>
JV>
JV> Subject
JV> Re: [osgi-dev] Continuous Integration of OSGi Bundles
JV>
JV>
JV>
JV>
JV>
JV> Jeff,
JV>
JV> Perhaps I'll look towards starting with the current PDE build,
JV> then. I'll try to catch up on its lists. Are you aware of any
JV> large changes since 3.2 was released?
JV>
JV> The PDE part of our build really isn't the problem. The problem
JV> is that I'm storing each bundle as a project checked out into an
JV> Eclipse workspace (all headlessly). Updating the projects from
JV> subversion is much like highlighting all projects in a workspace
JV> and selecting Team -> Update: the update happens
JV> project-per-project. The much more efficient way to do this would
JV> be to simply update the root directory of all the bundle projects,
JV> which Subversion does very well. However, this change would
JV> require some rearchitecting, and since we have other larger needs
JV> anyway we want to just redo the whole thing.
JV>
JV> One issue with the PDE export is that it's a mysterious black
JV> box of long running processes. I'm sure that PDE Build has better
JV> build output, since it's not intended as a UI operation. Also,
JV> the PDE export job fails completely if any of the bundles have
JV> unresolved manifests (with a CoreException). It will complete
JV> successfully if a bundle has compilation problems, though.
JV> Optimally the export job would treat manifest resolution failures like
compilation failures.
JV>
JV> Thanks,
JV> Jeremy
JV>
JV> On 4/30/07, Jeff McAffer <[EMAIL PROTECTED] > wrote:
JV> Jeremy,
JV>
JV> There has been some discussion in the area of updating PDE build
JV> to do much of what you have described. Currently however there
JV> are not enough people in that community to do the required work.
JV> Do you think you/your organization would have time to work in this area?
JV>
JV> Jeff
JV>
JV> p.s., In case you didn't know, the Export Deployable Plugins and
JV> Fragments option in the Eclipse UI actually runs PDE build under
JV> the covers. Not sure why performance is degrading. You might
JV> check that you are not doing a build in the workspace (Look at the
JV> Project > Build Automatically menu setting). That would be a
JV> waste since PDE build does its own build.
JV>
JV>
JV> "Jeremy Volkman" <[EMAIL PROTECTED] >
JV> Sent by: [EMAIL PROTECTED]
JV> 04/30/2007 10:22 AM
JV>
JV>
JV> Please respond to
JV> OSGi Developer Mail List <[email protected] >
JV>
JV>
JV>
JV>
JV> To
JV> [EMAIL PROTECTED]
JV> cc
JV>
JV>
JV> Subject
JV> [osgi-dev] Continuous Integration of OSGi Bundles
JV>
JV>
JV>
JV>
JV>
JV>
JV> Greetings,
JV>
JV> We're looking to re-implement/replace our current continuous
JV> integration system, which has been building our OSGi project for the
JV> past 10 months or so. The current builder was quickly hacked together
JV> over a period of a week, and we're starting to run into its
JV> limitations now that our project has surpassed 150 bundles.
JV>
JV> Some of the big requirements that we set for our current system were:
JV> 1. Automatic discovery of new bundle projects in the source
JV> repository. We didn't want to configure the builder each time we
JV> added a new bundle to the project.
JV> 2. Build as much as needed, but no more. Based on what was changed
JV> since the previous build, we wanted to use OSGi dependency analysis to
JV> figure out what needed to be built (i.e., changed bundles and bundles
JV> depending upon them). Because of this need, the builder must be able
JV> to fetch dependencies required for build from our bundle repository.
JV> 3. Resolve build dependencies based upon OSGi metadata (not, for
JV> instance, Maven dependencies).
JV> 4. Build Eclipse PDE bundle projects, since that's what we use as our
JV> development environment.
JV>
JV> To cover these requirements, we wrote a custom continuous integration
JV> system consisting of:
JV> 1. A repository scanner (written using JavaSVN) able to detect new
JV> projects in our repository; and
JV> 2. A bundle builder based around Eclipse PDE export operations.
JV>
JV> We did not use the PDE build system but rather do the equivalent of
JV> launching an Eclipse instance, selecting bundles and invoking "Export
JV> Deployable plug-ins and fragments".
JV>
JV> This system has worked fairly well for us. However, it's becoming
JV> slower and slower, and it is not easily extensible. We'd like to be
JV> able to make use of some existing builder for integrated
JV> testing/reporting/etc.
JV>
JV> I've noticed that a lot of people are using Maven to build OSGi
JV> bundles. However, as far as I can tell, Maven requires its own
JV> dependency format, and cannot be extended resolve based upon OSGi
JV> dependency metadata. Please correct me if this is incorrect.
JV>
JV> Does anyone have any systems similar to this, or any pointers to build
JV> systems that can be easily extended to support these types of
JV> features?
JV>
JV> Thanks,
JV> Jeremy Volkman
JV> _______________________________________________
JV> OSGi Developer Mail List
JV> [email protected]
JV> http://www2.osgi.org/mailman/listinfo/osgi-dev
JV>
JV>
JV> _______________________________________________
JV> OSGi Developer Mail List
JV> [email protected]
JV> http://www2.osgi.org/mailman/listinfo/osgi-dev
JV> _______________________________________________
JV> OSGi Developer Mail List
JV> [email protected]
JV> http://www2.osgi.org/mailman/listinfo/osgi-dev
JV> _______________________________________________
JV> OSGi Developer Mail List
JV> [email protected]
JV> http://www2.osgi.org/mailman/listinfo/osgi-dev
JV>
--
Peter Kriens Tel +33467542167
9C, Avenue St. Drézéry AOL,Yahoo: pkriens
34160 Beaulieu, France ICQ 255570717
Skype pkriens Fax +1 8153772599
_______________________________________________
OSGi Developer Mail List
[email protected]
http://www2.osgi.org/mailman/listinfo/osgi-dev
