Re: [DISCUSS] Feature package, feature generation and validation

[Stuart McCulloch]

We use the ‘feature’ packaging along with the ‘feature-generate-descriptor’ goal
and it has been working very well for our particular use-case.

So while I’m ok with removing it from the default lifecycle (or better 
introducing a
flag to control it) I really wouldn’t want to see it totally go away as then 
we’d have
to maintain the same information twice - both in the Maven pom, and again in the
feature descriptor.

On Thursday, 13 October 2016 at 10:07, Guillaume Nodet wrote:

> The feature packaging is a nice thing, as it allows automatic attachment of
> the feature file.
> However, it always use the feature-generate-descriptor, which produces a
> lot of weird results.
> Afaik, the feature packaging is not much used and all projects i've seen,
> such as pax projects, camel, cxf, and even karaf itself (including
> decanter, cellar, karaf container...).
>  
> I think part of the problem comes from the feature descriptor generation,
> which is difficult to control. I have always found much easier to simply
> write the feature manually.
> Although the generation process rewrites the xml entirely, so that any xml
> comments or license header is lost.
>  
> Overall, I'm not sure that it makes our users life really easier.
>  
> So I'd like to propose to get rid of the feature-generate-descriptor from
> inside the feature packaging and replace it with the verify goal to
> validate the hand-written features instead.
>  
> Thoughts ?
>  
> --  
> 
> Guillaume Nodet
> 
> Red Hat, Open Source Integration
>  
> Email: gno...@redhat.com (mailto:gno...@redhat.com)
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>  
>  

Re: [DISCUSS] Feature package, feature generation and validation

[Guillaume Nodet]

I'm wondering about something in addition to the below items.
The problem I have with the feature-generate-descriptor is that I don't
think the result can be trusted as is, i.e. I don't think we should blindly
use the output.  However, it could be of a real help getting started when
writing the features.
So instead of a goal that would be use during build time, I'm thinking that
it could be used at development time : the output features would be
reviewed by the user and used as a starting point for the "real" feature.
Thoughts ?

2016-10-19 14:40 GMT+02:00 Guillaume Nodet :

> So I'll go ahead with the following:
>   * change the verify goal to be a default goal for the feature packaging
>   * change the feature-generate-descriptor to be a non default goal for
> the feature packaging
>
> The benefits is that people who want to hand-write features will be able
> to use the feature packaging, while those who want to use the generator
> will have to add this goal to their build.
> I've raised KARAF-4787 for that.
>
>
> 2016-10-13 11:07 GMT+02:00 Guillaume Nodet :
>
>> The feature packaging is a nice thing, as it allows automatic attachment
>> of the feature file.
>> However, it always use the feature-generate-descriptor, which produces a
>> lot of weird results.
>> Afaik, the feature packaging is not much used and all projects i've seen,
>> such as pax projects, camel, cxf, and even karaf itself (including
>> decanter, cellar, karaf container...).
>>
>> I think part of the problem comes from the feature descriptor generation,
>> which is difficult to control.  I have always found much easier to simply
>> write the feature manually.
>> Although the generation process rewrites the xml entirely, so that any
>> xml comments or license header is lost.
>>
>> Overall, I'm not sure that it makes our users life really easier.
>>
>> So I'd like to propose to get rid of the feature-generate-descriptor from
>> inside the feature packaging and replace it with the verify goal to
>> validate the hand-written features instead.
>>
>> Thoughts ?
>>
>> --
>> 
>> Guillaume Nodet
>> 
>> Red Hat, Open Source Integration
>>
>> Email: gno...@redhat.com
>> Web: http://fusesource.com
>> Blog: http://gnodet.blogspot.com/
>>
>>
>
>
> --
> 
> Guillaume Nodet
> 
> Red Hat, Open Source Integration
>
> Email: gno...@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>
>


-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/

Re: [DISCUSS] Feature package, feature generation and validation

[Jean-Baptiste Onofré]

+1

It sounds reasonable.

Regards
JB

On 10/19/2016 02:40 PM, Guillaume Nodet wrote:

So I'll go ahead with the following:
  * change the verify goal to be a default goal for the feature packaging
  * change the feature-generate-descriptor to be a non default goal for the
feature packaging

The benefits is that people who want to hand-write features will be able to
use the feature packaging, while those who want to use the generator will
have to add this goal to their build.
I've raised KARAF-4787 for that.


2016-10-13 11:07 GMT+02:00 Guillaume Nodet :


The feature packaging is a nice thing, as it allows automatic attachment
of the feature file.
However, it always use the feature-generate-descriptor, which produces a
lot of weird results.
Afaik, the feature packaging is not much used and all projects i've seen,
such as pax projects, camel, cxf, and even karaf itself (including
decanter, cellar, karaf container...).

I think part of the problem comes from the feature descriptor generation,
which is difficult to control.  I have always found much easier to simply
write the feature manually.
Although the generation process rewrites the xml entirely, so that any xml
comments or license header is lost.

Overall, I'm not sure that it makes our users life really easier.

So I'd like to propose to get rid of the feature-generate-descriptor from
inside the feature packaging and replace it with the verify goal to
validate the hand-written features instead.

Thoughts ?

--

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/







--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [DISCUSS] Feature package, feature generation and validation

[David Daniel]

This custom resource repository is what I would love to see as far as
bndtools and karaf integration.  I think bndtools had made a huge leap with
their new maven integration in the bnd pom repository.  I would love to see
a feature repository built off of that. Ideally when I run karaf in pax
exam I could include it as a repo and my unit tests could share the same
repo as my ide that I debug in.

On Wed, Oct 19, 2016 at 11:17 AM, Christian Schneider <
ch...@die-schneider.net> wrote:

> On 19.10.2016 16:57, Guillaume Nodet wrote:
>
>>
>>
>> Something like:
>>mvn:...
>>mvn:...
>>...
>>
>> That sounds like a feature to me ;-)
>>
> The advantage would be that you can use IDE tooling for poms like M2E. So
> you get search and completion for the artifacts. You get nice introspection
> for the transitive dependencies.
> I think one thing we are missing in karaf feature files is IDE support.
>
>
>> Anyway, a custom osgi resource repository can be easily created for an
>> external file by inheriting
>> org.apache.karaf.features.internal.repository.BaseRepository.  Just add
>> your new repository into org.apache.karaf.features.internal.region.
>> getRepository  and it should work.
>>
> Sounds interesting. I will play with that.
>
>>
>> Again, in the simple cases, if this list of bundles can be generated, it
>> means that the feature itself can be generated, so I'm not sure where the
>> benefit is...
>>
> I have not used the generated features a lot. One thing that might be
> problematic is that a feature file can contain several
> features while a pom can only contain one list of bundles. That is why I
> would treat the list of bundles from the pom rather as a
> backing repository than as a feature.
>
> Again I think the main benefit would come from tooling. Bndtools allows to
> search in the index and just drag bundles into the requirements.
> Such a mechanism would also be great for features. Unfortunately we do not
> have a lot of good Eclipse RCP or Intellij plugin developers at Apache.
> So I guess the dream of having decent IDE tooling is quite far away :-(
>
>
> Christian
>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>

Re: [DISCUSS] Feature package, feature generation and validation

[Christian Schneider]

On 19.10.2016 16:57, Guillaume Nodet wrote:



Something like:
   mvn:...
   mvn:...
   ...

That sounds like a feature to me ;-)
The advantage would be that you can use IDE tooling for poms like M2E. 
So you get search and completion for the artifacts. You get nice 
introspection for the transitive dependencies.

I think one thing we are missing in karaf feature files is IDE support.



Anyway, a custom osgi resource repository can be easily created for an
external file by inheriting
org.apache.karaf.features.internal.repository.BaseRepository.  Just add
your new repository into org.apache.karaf.features.internal.region.
getRepository  and it should work.

Sounds interesting. I will play with that.


Again, in the simple cases, if this list of bundles can be generated, it
means that the feature itself can be generated, so I'm not sure where the
benefit is...
I have not used the generated features a lot. One thing that might be 
problematic is that a feature file can contain several
features while a pom can only contain one list of bundles. That is why I 
would treat the list of bundles from the pom rather as a

backing repository than as a feature.

Again I think the main benefit would come from tooling. Bndtools allows 
to search in the index and just drag bundles into the requirements.
Such a mechanism would also be great for features. Unfortunately we do 
not have a lot of good Eclipse RCP or Intellij plugin developers at Apache.

So I guess the dream of having decent IDE tooling is quite far away :-(

Christian


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: [DISCUSS] Feature package, feature generation and validation

[Guillaume Nodet]

2016-10-19 16:13 GMT+02:00 Christian Schneider :

> On 19.10.2016 15:53, Guillaume Nodet wrote:
>
>> 2016-10-19 15:28 GMT+02:00 Christian Schneider :
>>
>> On 19.10.2016 15:22, Guillaume Nodet wrote:
>>>
>>> I disagree.

 All the problems come when you start using maven transitive dependencies
 in
 real projects and hit lots of dependencies which are not OSGi bundles or
 not OSGi ready.  Think about simple examples like spring, or all the
 bundles that we do re-wrap in servicemix.
 I think this idea is nice in theory, but it just does not work in real
 life.

 The idea is to not simply use the transitive dependencies of an existing
>>> project.
>>> Instead you create a pom where you tune the dependencies using excludes
>>> that only the bundles remain that you really want.
>>> I agree that just using any pom will produce bad results.
>>>
>>
>> That sounds like years ago when we migrated from maven 1 to maven 2.
>> Maven
>> 1 did not use transitive dependencies, so suddenly, a lot of unwanted
>> artifacts were included in the build and we had to use exclusions and
>> such.  The main difference, is that the fact that everyone would be
>> migrating from maven 1 to maven 2 was quite sure, so it was a transition.
>> However, I doubt everyone will support OSGi ever, so that's a state, not a
>> transition.
>> I think that's the difference.
>>
>
> In order to push user to a direction, we need to make sure it scales to
>> real projects.  Nice tools for beginners can become a pain when you
>> realize
>> you need to change to something else because they are too limited.  Maven
>> metadata will never be able to support the metadata than a karaf feature
>> can carry...
>>
> I agree it is quite some work. The good thing is that such repository poms
> can be used by karaf and bndtools.
> I already started providing such poms in Aries RSA and CXF DOSGi. I was
> able to build some nice demos for CXF DOSGi using these
> poms. So I think it can work and at least scale to support CXF.
>
> I think it will not work as well with camel as the nature of camel is to
> integrate many things. So for camel writing features by hand might be the
> better option.
> On the other hand I found that writing a pom for the depenencies you need
> from camel in an actual project is not that difficult. I did this for the
> OSGi chat example
> were I used camel-irc.
>
> I agree with the problem that maven is not the only build system. So
> people who build with gradle will not like the idea of using a pom as a
> repo.
> Maybe we can supply a simple maven plugin that just outputs a list of the
> bundles with maven coordinates. That would also remove the need to fully
> resolve pom files.
> For gradle we could then simply have a gradle plugin that does the same.
>
> So the effort in karaf would only be to create an OBR index on the fly
> from a simple text file with mvn urls.


Something like:
  mvn:...
  mvn:...
  ...

That sounds like a feature to me ;-)

Anyway, a custom osgi resource repository can be easily created for an
external file by inheriting
org.apache.karaf.features.internal.repository.BaseRepository.  Just add
your new repository into org.apache.karaf.features.internal.region.
getRepository  and it should work.

Again, in the simple cases, if this list of bundles can be generated, it
means that the feature itself can be generated, so I'm not sure where the
benefit is...


>
>
> Christian
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>


-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/

Re: [DISCUSS] Feature package, feature generation and validation

[Guillaume Nodet]

2016-10-19 15:28 GMT+02:00 Christian Schneider :

> On 19.10.2016 15:22, Guillaume Nodet wrote:
>
>> I disagree.
>>
>> All the problems come when you start using maven transitive dependencies
>> in
>> real projects and hit lots of dependencies which are not OSGi bundles or
>> not OSGi ready.  Think about simple examples like spring, or all the
>> bundles that we do re-wrap in servicemix.
>> I think this idea is nice in theory, but it just does not work in real
>> life.
>>
> The idea is to not simply use the transitive dependencies of an existing
> project.
> Instead you create a pom where you tune the dependencies using excludes
> that only the bundles remain that you really want.
> I agree that just using any pom will produce bad results.


That sounds like years ago when we migrated from maven 1 to maven 2.  Maven
1 did not use transitive dependencies, so suddenly, a lot of unwanted
artifacts were included in the build and we had to use exclusions and
such.  The main difference, is that the fact that everyone would be
migrating from maven 1 to maven 2 was quite sure, so it was a transition.
However, I doubt everyone will support OSGi ever, so that's a state, not a
transition.
I think that's the difference.


>
>
>> Also, in the past years, several attemps have been made at using pure
>> maven
>> metadata to do provisioning, it has always failed to my knowledge, so I'd
>> really want to avoid going that road again.
>>
> The idea is not to use the maven metadata. It should still work like an
> OBR index but the index could be generated on the fly.
>
>>
>> Keep in mind that Karaf always create OBR repositories on the fly, based
>> on
>> the bundles listed in the features.
>>
> So the main change would be to move the definition of the bundles from the
> feature file to a pom. Eventually this could be the pom where the feature
> file resides.


That could be have been an interesting idea when people were working on
Tycho.  That's not the case anymore, and maven is not the only build tool
people use (think graddle, ivy, etc.., though I think it's still the more
popular).

I'm really not convinced that it will really help.  For simple projects,
the feature can already be generated automatically, we may be able to
slighly improve it.  For more complicated cases, projects like cxf, camel,
karaf, pax web, etc..., I'm not sure it will be usable.

In order to push user to a direction, we need to make sure it scales to
real projects.  Nice tools for beginners can become a pain when you realize
you need to change to something else because they are too limited.  Maven
metadata will never be able to support the metadata than a karaf feature
can carry...


>
>
> Christian
>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>


-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/

Re: [DISCUSS] Feature package, feature generation and validation

[Guillaume Nodet]

I disagree.

All the problems come when you start using maven transitive dependencies in
real projects and hit lots of dependencies which are not OSGi bundles or
not OSGi ready.  Think about simple examples like spring, or all the
bundles that we do re-wrap in servicemix.
I think this idea is nice in theory, but it just does not work in real life.

Also, in the past years, several attemps have been made at using pure maven
metadata to do provisioning, it has always failed to my knowledge, so I'd
really want to avoid going that road again.

Keep in mind that Karaf always create OBR repositories on the fly, based on
the bundles listed in the features.

If you maven metadata is clean, you can simply use the generate goal and
you're good to go.  If it's not clean enough, because it contains non OSGi
dependencies, or because you need additional informations, keep your
features hand-written.

As for pure resource repositories, they can already be used and referenced
from feature files using the  elements, so we can
already experiment a bit.


2016-10-19 15:04 GMT+02:00 Christian Schneider :

> I agree. Currently the feature files are too low level. Basically you have
> to list all bundles.
> We need something like bndtools resolution.
>
> So I think a feature should be backed by an index. I have made some good
> experiences with pom based indexes. This is a pom that depends on bundles
> (including transitive deps).
> The goal is to provide a full list of bundles as a basis of the feature
> file.
> See this as an example https://github.com/apache/arie
> s-rsa/blob/master/repository/pom.xml.
> In the pom above an OBR index is created by maven. Eventually this could
> be skipped and karaf could create such an index on the fly as the maven
> index plugin is not working very well for me.
>
> Each feature would then only need to list the top level bundles and other
> requirements.
>
> The karaf maven plugin could then either validate the features against the
> repository. This works if karaf can work with such index backed feature
> files.
> If karaf can not do this then the plugin could create the full list of
> bundles per feature and write "traditional" feature files. The additional
> resolved bundles would then
> have dependency=true.
>
> Christian
>
>
> On 13.10.2016 11:19, Milen Dyankov wrote:
>
>> I have 2 things to say to that
>> - I agree with all the pain points you've identified (experienced them
>> myself)
>> - I'd prefer to fix things instead of claim them useless due to
>> malfunctioning
>>
>> Perhaps a middle ground would be a good starting point? Something like how
>> bndrun resolution works. I mean:
>>   - developer says - this is what I care to run (perhaps a prototype
>> feature
>> or something ...)
>>   - feature-generate-descriptor takes it from there and fills in the gaps
>>   - developer can change/fix things by tweaking the prototype if not happy
>> with what feature-generate-descriptor did
>>
>> This is just my first thought and I'm pretty sure reality is not that
>> simple. Just wanted to vote against removing it and suggest to start
>> looking for better solution instead.
>>
>> Best,
>> Milen
>>
>>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>


-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/

Re: [DISCUSS] Feature package, feature generation and validation

[Christian Schneider]

On 19.10.2016 15:22, Guillaume Nodet wrote:

I disagree.

All the problems come when you start using maven transitive dependencies in
real projects and hit lots of dependencies which are not OSGi bundles or
not OSGi ready.  Think about simple examples like spring, or all the
bundles that we do re-wrap in servicemix.
I think this idea is nice in theory, but it just does not work in real life.
The idea is to not simply use the transitive dependencies of an existing 
project.
Instead you create a pom where you tune the dependencies using excludes 
that only the bundles remain that you really want.

I agree that just using any pom will produce bad results.


Also, in the past years, several attemps have been made at using pure maven
metadata to do provisioning, it has always failed to my knowledge, so I'd
really want to avoid going that road again.
The idea is not to use the maven metadata. It should still work like an 
OBR index but the index could be generated on the fly.


Keep in mind that Karaf always create OBR repositories on the fly, based on
the bundles listed in the features.
So the main change would be to move the definition of the bundles from 
the feature file to a pom. Eventually this could be the pom where the 
feature file resides.


Christian

--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: [DISCUSS] Feature package, feature generation and validation

[Richard Nicholson]

Christian - sounds like a good idea ;) 

Paremus have always pursued this sort of approach -  See 
https://docs.paremus.com/pages/viewpage.action?pageId=8060997 
 for a simple 
example. 

Cheers




> On 19 Oct 2016, at 14:04, Christian Schneider  wrote:
> 
> I agree. Currently the feature files are too low level. Basically you have to 
> list all bundles.
> We need something like bndtools resolution.
> 
> So I think a feature should be backed by an index. I have made some good 
> experiences with pom based indexes. This is a pom that depends on bundles 
> (including transitive deps).
> The goal is to provide a full list of bundles as a basis of the feature file.
> See this as an example 
> https://github.com/apache/aries-rsa/blob/master/repository/pom.xml.
> In the pom above an OBR index is created by maven. Eventually this could be 
> skipped and karaf could create such an index on the fly as the maven index 
> plugin is not working very well for me.
> 
> Each feature would then only need to list the top level bundles and other 
> requirements.
> 
> The karaf maven plugin could then either validate the features against the 
> repository. This works if karaf can work with such index backed feature files.
> If karaf can not do this then the plugin could create the full list of 
> bundles per feature and write "traditional" feature files. The additional 
> resolved bundles would then
> have dependency=true.
> 
> Christian
> 
> On 13.10.2016 11:19, Milen Dyankov wrote:
>> I have 2 things to say to that
>> - I agree with all the pain points you've identified (experienced them
>> myself)
>> - I'd prefer to fix things instead of claim them useless due to
>> malfunctioning
>> 
>> Perhaps a middle ground would be a good starting point? Something like how
>> bndrun resolution works. I mean:
>>  - developer says - this is what I care to run (perhaps a prototype feature
>> or something ...)
>>  - feature-generate-descriptor takes it from there and fills in the gaps
>>  - developer can change/fix things by tweaking the prototype if not happy
>> with what feature-generate-descriptor did
>> 
>> This is just my first thought and I'm pretty sure reality is not that
>> simple. Just wanted to vote against removing it and suggest to start
>> looking for better solution instead.
>> 
>> Best,
>> Milen
>> 
> 
> 
> -- 
> Christian Schneider
> http://www.liquid-reality.de
> 
> Open Source Architect
> http://www.talend.com
> 

Re: [DISCUSS] Feature package, feature generation and validation

[Christian Schneider]

I agree. Currently the feature files are too low level. Basically you 
have to list all bundles.

We need something like bndtools resolution.

So I think a feature should be backed by an index. I have made some good 
experiences with pom based indexes. This is a pom that depends on 
bundles (including transitive deps).
The goal is to provide a full list of bundles as a basis of the feature 
file.
See this as an example 
https://github.com/apache/aries-rsa/blob/master/repository/pom.xml.
In the pom above an OBR index is created by maven. Eventually this could 
be skipped and karaf could create such an index on the fly as the maven 
index plugin is not working very well for me.


Each feature would then only need to list the top level bundles and 
other requirements.


The karaf maven plugin could then either validate the features against 
the repository. This works if karaf can work with such index backed 
feature files.
If karaf can not do this then the plugin could create the full list of 
bundles per feature and write "traditional" feature files. The 
additional resolved bundles would then

have dependency=true.

Christian

On 13.10.2016 11:19, Milen Dyankov wrote:

I have 2 things to say to that
- I agree with all the pain points you've identified (experienced them
myself)
- I'd prefer to fix things instead of claim them useless due to
malfunctioning

Perhaps a middle ground would be a good starting point? Something like how
bndrun resolution works. I mean:
  - developer says - this is what I care to run (perhaps a prototype feature
or something ...)
  - feature-generate-descriptor takes it from there and fills in the gaps
  - developer can change/fix things by tweaking the prototype if not happy
with what feature-generate-descriptor did

This is just my first thought and I'm pretty sure reality is not that
simple. Just wanted to vote against removing it and suggest to start
looking for better solution instead.

Best,
Milen




--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: [DISCUSS] Feature package, feature generation and validation

[James Carman]

I am +1 on removing it, especially if nobody wants to maintain it.  I tried
to use it at one point and it just never really worked well.  Hand-crafted
features files are always the best option, IMHO.  It might be nice to have
a Maven archetype or something that would generate a "features module" from
scratch, to give folks a starting point.

On Thu, Oct 13, 2016 at 5:08 AM Guillaume Nodet  wrote:

> The feature packaging is a nice thing, as it allows automatic attachment of
> the feature file.
> However, it always use the feature-generate-descriptor, which produces a
> lot of weird results.
> Afaik, the feature packaging is not much used and all projects i've seen,
> such as pax projects, camel, cxf, and even karaf itself (including
> decanter, cellar, karaf container...).
>
> I think part of the problem comes from the feature descriptor generation,
> which is difficult to control.  I have always found much easier to simply
> write the feature manually.
> Although the generation process rewrites the xml entirely, so that any xml
> comments or license header is lost.
>
> Overall, I'm not sure that it makes our users life really easier.
>
> So I'd like to propose to get rid of the feature-generate-descriptor from
> inside the feature packaging and replace it with the verify goal to
> validate the hand-written features instead.
>
> Thoughts ?
>
> --
> 
> Guillaume Nodet
> 
> Red Hat, Open Source Integration
>
> Email: gno...@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>

Re: [DISCUSS] Feature package, feature generation and validation

[Stephen Kitt]

On Thu, 13 Oct 2016 12:18:04 +0200
Stephen Kitt  wrote:
> I've been working on improving feature-generate-descriptor to support
> our use-cases, and I am still planning on improving it (e.g. to handle
> aggregate feature repositories). We still need hand-written
> feature.xml snippets in some cases, and I'm not sure we'll ever be
> able to handle them... But overall I've found the generator useful.

That being said, I don't mind of course if the generator is dropped
from the default execution!

Regards,

Stephen


pgpeKbB9aViQB.pgp
Description: OpenPGP digital signature

Re: [DISCUSS] Feature package, feature generation and validation

[Stephen Kitt]

Hi Guillaume,

On Thu, 13 Oct 2016 11:07:50 +0200
Guillaume Nodet  wrote:
[...]
> So I'd like to propose to get rid of the feature-generate-descriptor
> from inside the feature packaging and replace it with the verify goal
> to validate the hand-written features instead.

I've started migrating OpenDaylight to generated features (if you're
really keen, https://git.opendaylight.org/gerrit/39539 is the first
patch in the migration), for a variety of reasons:
* the dependencies are only specified once (in the POM)
* bundles are wrapped automatically
* dependencies aren't forgotten (this is caught by automated tests, but
  if the plugin can handle it for us that's even better)

The main reason is the first one; we've hit all sorts of pain points
dealing with duplicate dependency definitions, and I'd really like to
avoid them.

I've been working on improving feature-generate-descriptor to support
our use-cases, and I am still planning on improving it (e.g. to handle
aggregate feature repositories). We still need hand-written feature.xml
snippets in some cases, and I'm not sure we'll ever be able to handle
them... But overall I've found the generator useful.

Regards,

Stephen


pgpqnVGLj2eMe.pgp
Description: OpenPGP digital signature

Re: [DISCUSS] Feature package, feature generation and validation

[Jean-Baptiste Onofré]

+1

It would remove or at least limit the generate feature execution. It's too 
simple and never cleanly cover the use cases. Writing the features XML by hand 
is always better IMHO.

Regards
JB



On Oct 13, 2016, 11:08, at 11:08, Guillaume Nodet  wrote:
>The feature packaging is a nice thing, as it allows automatic
>attachment of
>the feature file.
>However, it always use the feature-generate-descriptor, which produces
>a
>lot of weird results.
>Afaik, the feature packaging is not much used and all projects i've
>seen,
>such as pax projects, camel, cxf, and even karaf itself (including
>decanter, cellar, karaf container...).
>
>I think part of the problem comes from the feature descriptor
>generation,
>which is difficult to control.  I have always found much easier to
>simply
>write the feature manually.
>Although the generation process rewrites the xml entirely, so that any
>xml
>comments or license header is lost.
>
>Overall, I'm not sure that it makes our users life really easier.
>
>So I'd like to propose to get rid of the feature-generate-descriptor
>from
>inside the feature packaging and replace it with the verify goal to
>validate the hand-written features instead.
>
>Thoughts ?
>
>-- 
>
>Guillaume Nodet
>
>Red Hat, Open Source Integration
>
>Email: gno...@redhat.com
>Web: http://fusesource.com
>Blog: http://gnodet.blogspot.com/

Re: [DISCUSS] Feature package, feature generation and validation

[Achim Nierbeck]

Hi,

as I'm the one with the most pain right now, I'm +1 on a pure validation
goal.
OTH I know a lot of people use the Feature generation as a starting point
to actually get going.
Especially since the "hand" writing for starters of using Karaf is
cumbersome in the beginning.
May I introduce another posibility?
Could we seperate the two concerns?
Have a validation which actually only validates but the plugin still
attaches the feature artifact to the build process so
it can be deployed.
And a second non-default generation goal, which uses all compile
dependencies to generate a new feature file which then
can also be used as attached artifact?

regards, Achim


2016-10-13 11:18 GMT+02:00 Grzegorz Grzybek :

> Good idea in my opinion.
>
> Feature descriptors should be (are) first-class artifacts and should be
> carefully maintained. Relying simply on underlying dependencies of another
> category (like Maven) is not enough.
>
> Manual creation + build time verification is much better idea.
>
> regards
> Grzegorz Grzybek
>
> 2016-10-13 11:07 GMT+02:00 Guillaume Nodet :
>
> > The feature packaging is a nice thing, as it allows automatic attachment
> of
> > the feature file.
> > However, it always use the feature-generate-descriptor, which produces a
> > lot of weird results.
> > Afaik, the feature packaging is not much used and all projects i've seen,
> > such as pax projects, camel, cxf, and even karaf itself (including
> > decanter, cellar, karaf container...).
> >
> > I think part of the problem comes from the feature descriptor generation,
> > which is difficult to control.  I have always found much easier to simply
> > write the feature manually.
> > Although the generation process rewrites the xml entirely, so that any
> xml
> > comments or license header is lost.
> >
> > Overall, I'm not sure that it makes our users life really easier.
> >
> > So I'd like to propose to get rid of the feature-generate-descriptor from
> > inside the feature packaging and replace it with the verify goal to
> > validate the hand-written features instead.
> >
> > Thoughts ?
> >
> > --
> > 
> > Guillaume Nodet
> > 
> > Red Hat, Open Source Integration
> >
> > Email: gno...@redhat.com
> > Web: http://fusesource.com
> > Blog: http://gnodet.blogspot.com/
> >
>



-- 

Apache Member
Apache Karaf  Committer & PMC
OPS4J Pax Web  Committer &
Project Lead
blog 
Co-Author of Apache Karaf Cookbook 

Software Architect / Project Manager / Scrum Master

Re: [DISCUSS] Feature package, feature generation and validation

[Milen Dyankov]

I have 2 things to say to that
- I agree with all the pain points you've identified (experienced them
myself)
- I'd prefer to fix things instead of claim them useless due to
malfunctioning

Perhaps a middle ground would be a good starting point? Something like how
bndrun resolution works. I mean:
 - developer says - this is what I care to run (perhaps a prototype feature
or something ...)
 - feature-generate-descriptor takes it from there and fills in the gaps
 - developer can change/fix things by tweaking the prototype if not happy
with what feature-generate-descriptor did

This is just my first thought and I'm pretty sure reality is not that
simple. Just wanted to vote against removing it and suggest to start
looking for better solution instead.

Best,
Milen

On Thu, Oct 13, 2016 at 11:07 AM, Guillaume Nodet  wrote:

> The feature packaging is a nice thing, as it allows automatic attachment of
> the feature file.
> However, it always use the feature-generate-descriptor, which produces a
> lot of weird results.
> Afaik, the feature packaging is not much used and all projects i've seen,
> such as pax projects, camel, cxf, and even karaf itself (including
> decanter, cellar, karaf container...).
>
> I think part of the problem comes from the feature descriptor generation,
> which is difficult to control.  I have always found much easier to simply
> write the feature manually.
> Although the generation process rewrites the xml entirely, so that any xml
> comments or license header is lost.
>
> Overall, I'm not sure that it makes our users life really easier.
>
> So I'd like to propose to get rid of the feature-generate-descriptor from
> inside the feature packaging and replace it with the verify goal to
> validate the hand-written features instead.
>
> Thoughts ?
>
> --
> 
> Guillaume Nodet
> 
> Red Hat, Open Source Integration
>
> Email: gno...@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>



-- 
http://about.me/milen

Re: [DISCUSS] Feature package, feature generation and validation

[Grzegorz Grzybek]

Good idea in my opinion.

Feature descriptors should be (are) first-class artifacts and should be
carefully maintained. Relying simply on underlying dependencies of another
category (like Maven) is not enough.

Manual creation + build time verification is much better idea.

regards
Grzegorz Grzybek

2016-10-13 11:07 GMT+02:00 Guillaume Nodet :

> The feature packaging is a nice thing, as it allows automatic attachment of
> the feature file.
> However, it always use the feature-generate-descriptor, which produces a
> lot of weird results.
> Afaik, the feature packaging is not much used and all projects i've seen,
> such as pax projects, camel, cxf, and even karaf itself (including
> decanter, cellar, karaf container...).
>
> I think part of the problem comes from the feature descriptor generation,
> which is difficult to control.  I have always found much easier to simply
> write the feature manually.
> Although the generation process rewrites the xml entirely, so that any xml
> comments or license header is lost.
>
> Overall, I'm not sure that it makes our users life really easier.
>
> So I'd like to propose to get rid of the feature-generate-descriptor from
> inside the feature packaging and replace it with the verify goal to
> validate the hand-written features instead.
>
> Thoughts ?
>
> --
> 
> Guillaume Nodet
> 
> Red Hat, Open Source Integration
>
> Email: gno...@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>

[DISCUSS] Feature package, feature generation and validation

[Guillaume Nodet]

The feature packaging is a nice thing, as it allows automatic attachment of
the feature file.
However, it always use the feature-generate-descriptor, which produces a
lot of weird results.
Afaik, the feature packaging is not much used and all projects i've seen,
such as pax projects, camel, cxf, and even karaf itself (including
decanter, cellar, karaf container...).

I think part of the problem comes from the feature descriptor generation,
which is difficult to control.  I have always found much easier to simply
write the feature manually.
Although the generation process rewrites the xml entirely, so that any xml
comments or license header is lost.

Overall, I'm not sure that it makes our users life really easier.

So I'd like to propose to get rid of the feature-generate-descriptor from
inside the feature packaging and replace it with the verify goal to
validate the hand-written features instead.

Thoughts ?

-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/