Re: What is OFBiz public API?

[Michael Brohl]

HI Piere,

inline...


Am 21.01.20 um 15:56 schrieb Pierre Smits:

Hi all,

The mission of the project is, in line with the ASF, to provide software
for the greater good and offering (potential) adopters a choice. In the
projects case this is done through providing the OFBiz works (each and all
products, as well as the supporting tooling for enabling contributors like
JIRA, Github services, etc.)

Agreed and in no way questioned in the preceding discussions.


The project has since its inception as a TLP made it clear, with the above
in mind, that the the project is not responsible for the choices a
(potential) adopter has made, nor that it will be held responsible for the
future choices of an adopter.


Of course we are not held responsible in a legal sense. But we (should) 
feel responsibility towards the users of the project and maintain it in 
a way which is in good balance between the user base needs and the 
contributions made to improve it.



This means, when talking about the code of the main OFBiz product, the
project does not care whether an adopter (being it either an enterprise
having OFBiz implemented, or a service integrator offering services and/or
solutions for payment or not) uses, enhances or diminishes the OFBiz
functionalities in upstream activities.


Where do I find this anywhere in the statutes of the project?

I strongly disagree with this statement. If we propagate that we don't 
care about the users and change the codebase, functionality and 
mechanisms without thinking about backwards compatibility, breaking 
existing projects and even don't support users with the documentation of 
a migration path, we will damage the projects' reputation.


This is not a playground project, it is used in serious businesses and 
we should have this in mind when changing things.


This is my main concern with the approach leading to the preceding 
discussion.




The project has also made clear in communications to every kind of
community member and adopter, that for use, enhancement or diminishment of
OFBiz functionalities in his environment of development and service
delivery the adopter should always start from releases and not from the
volatile trunk repositories.
Unfortunately, there have been (and are) several prominent members of the
OFBiz community that have made assertions that it was quite ok for adopters
to start with the code in the trunk branch of the repo for their own
develop and/or implementation.


Can you please point us to the source of these claims?


When these community members propagate this start-with-trunk paradigm in
their upstream needs-and-wants model, they tend to shoot in their own foot.
But instead of acknowledging that they took a wrong turn and remediate that
situation, they elect to bring their problem down to the OFBiz project and
demand that the other contributor removes this undesirable change from the
OFBiz code base, in order to keep the upstream deviation minimally
disrupted.


This seems to be very speculative to me. Where does the information for 
this claim come from?


The trunk is the source for the next stable branch and every change made 
there will sooner or later be in a next release and will have effects on 
users who will use it.


As a responsible developer, changes made to the codebase, especially if 
they are made to support a future change/improvement/refactoring, should 
be well planned, discussion, proved if they are really wanted/needed etc.


Another problem is that trunk receives a mix of functionality 
enhancements and improvements for the business part and also core 
improvements which might need a long time to be implemented (an example 
might be the initiative to make the framework container independent).


If we want to have shorter release cycles to bring the functionality 
updates to the users, this is in conflict with the improvements which 
needs more time to be implemented and thorougly tested. This is the 
reason why I would prefer to do these implementations in a feature 
branch which will be held up-to-date with trunk and allows experimental 
implementations without breaking anything. This is the classical and 
logical way when maintaining a product.


If you would see the project as a software development company which 
sells a product, this company would never start long-lasting 
developments in the base branch for the next release.


Release management and a clear roadmap would help, maybe a dicussion for 
another thread.




Given what Mathieu has brought forward regarding the 'depends-on'
enhancement of the OFBiz functionality, it is clear that it strengthens and
enhances understandings regarding the OFBiz product and its intricacies
(where the code is the documentation).


In this case, there seem to be plans in the minds of contributors which 
lead to the discussed changes. These plans are neither documented nor 
discussed in depth so that it is possible to make decisions. This is my 
main concern.


I also gave 

Re: What is OFBiz public API?

[Jacques Le Roux]

Thanks Pierre,

I second that, I would not write it better!

Jacques

Le 21/01/2020 à 15:56, Pierre Smits a écrit :

Hi all,

The mission of the project is, in line with the ASF, to provide software
for the greater good and offering (potential) adopters a choice. In the
projects case this is done through providing the OFBiz works (each and all
products, as well as the supporting tooling for enabling contributors like
JIRA, Github services, etc.)

The project has since its inception as a TLP made it clear, with the above
in mind, that the the project is not responsible for the choices a
(potential) adopter has made, nor that it will be held responsible for the
future choices of an adopter.
This means, when talking about the code of the main OFBiz product, the
project does not care whether an adopter (being it either an enterprise
having OFBiz implemented, or a service integrator offering services and/or
solutions for payment or not) uses, enhances or diminishes the OFBiz
functionalities in upstream activities.

The project has also made clear in communications to every kind of
community member and adopter, that for use, enhancement or diminishment of
OFBiz functionalities in his environment of development and service
delivery the adopter should always start from releases and not from the
volatile trunk repositories.
Unfortunately, there have been (and are) several prominent members of the
OFBiz community that have made assertions that it was quite ok for adopters
to start with the code in the trunk branch of the repo for their own
develop and/or implementation.
When these community members propagate this start-with-trunk paradigm in
their upstream needs-and-wants model, they tend to shoot in their own foot.
But instead of acknowledging that they took a wrong turn and remediate that
situation, they elect to bring their problem down to the OFBiz project and
demand that the other contributor removes this undesirable change from the
OFBiz code base, in order to keep the upstream deviation minimally
disrupted.

Given what Mathieu has brought forward regarding the 'depends-on'
enhancement of the OFBiz functionality, it is clear that it strengthens and
enhances understandings regarding the OFBiz product and its intricacies
(where the code is the documentation).

When looking at the process undertaken by Mathieu, he was working within
his mandate as a community member with the commit-privilege to commit the
changes to the trunk branch of the repo (as per ticket OFBIZ-11296, commit
  eeabe69813a1d9f42911dec70a912574046ef49b). Moreover, he was also working
along and in-line with precedents established by other community members
before he became a privileged contributor.

The discussion that later developed was a good thing, but with precedents
as they are it is unfortunate that the commit was later reverted.
Unfortunate, not because the discussion developed but because the commit
did not 'break the code'. Precedents have shown that questionable commits
did not lead to reversal of these commits.

If, and when, all understand the rules of engagement of the OFBiz project,
and act accordingly, there would never by such a (heated) discussion as
this one, or the ones leading up to this (whether started and followed
through in JIRA tickets, or on any of the mailing lists). Apparently many
of the community members (including those with privileges) do not
understand what these are or these provide to ambiguity which leads to
deviations when contributing.

Best regards,

Pierre Smits

*Apache Trafodion , Vice President*
*Apache Directory , PMC Member*
Apache Incubator , committer
*Apache OFBiz , contributor (without privileges)
since 2008*
Apache Steve , committer

Re: What is OFBiz public API?

[Pierre Smits]

Hi all,

The mission of the project is, in line with the ASF, to provide software
for the greater good and offering (potential) adopters a choice. In the
projects case this is done through providing the OFBiz works (each and all
products, as well as the supporting tooling for enabling contributors like
JIRA, Github services, etc.)

The project has since its inception as a TLP made it clear, with the above
in mind, that the the project is not responsible for the choices a
(potential) adopter has made, nor that it will be held responsible for the
future choices of an adopter.
This means, when talking about the code of the main OFBiz product, the
project does not care whether an adopter (being it either an enterprise
having OFBiz implemented, or a service integrator offering services and/or
solutions for payment or not) uses, enhances or diminishes the OFBiz
functionalities in upstream activities.

The project has also made clear in communications to every kind of
community member and adopter, that for use, enhancement or diminishment of
OFBiz functionalities in his environment of development and service
delivery the adopter should always start from releases and not from the
volatile trunk repositories.
Unfortunately, there have been (and are) several prominent members of the
OFBiz community that have made assertions that it was quite ok for adopters
to start with the code in the trunk branch of the repo for their own
develop and/or implementation.
When these community members propagate this start-with-trunk paradigm in
their upstream needs-and-wants model, they tend to shoot in their own foot.
But instead of acknowledging that they took a wrong turn and remediate that
situation, they elect to bring their problem down to the OFBiz project and
demand that the other contributor removes this undesirable change from the
OFBiz code base, in order to keep the upstream deviation minimally
disrupted.

Given what Mathieu has brought forward regarding the 'depends-on'
enhancement of the OFBiz functionality, it is clear that it strengthens and
enhances understandings regarding the OFBiz product and its intricacies
(where the code is the documentation).

When looking at the process undertaken by Mathieu, he was working within
his mandate as a community member with the commit-privilege to commit the
changes to the trunk branch of the repo (as per ticket OFBIZ-11296, commit
 eeabe69813a1d9f42911dec70a912574046ef49b). Moreover, he was also working
along and in-line with precedents established by other community members
before he became a privileged contributor.

The discussion that later developed was a good thing, but with precedents
as they are it is unfortunate that the commit was later reverted.
Unfortunate, not because the discussion developed but because the commit
did not 'break the code'. Precedents have shown that questionable commits
did not lead to reversal of these commits.

If, and when, all understand the rules of engagement of the OFBiz project,
and act accordingly, there would never by such a (heated) discussion as
this one, or the ones leading up to this (whether started and followed
through in JIRA tickets, or on any of the mailing lists). Apparently many
of the community members (including those with privileges) do not
understand what these are or these provide to ambiguity which leads to
deviations when contributing.

Best regards,

Pierre Smits

*Apache Trafodion , Vice President*
*Apache Directory , PMC Member*
Apache Incubator , committer
*Apache OFBiz , contributor (without privileges)
since 2008*
Apache Steve , committer

Re: What is OFBiz public API?

[Jacques Le Roux]

Le 21/01/2020 à 00:36, Mathieu Lirzin a écrit :

Hello,

Taher Alkhateeb  writes:


While working on all the above, I never faced major obstacles. But the
reason that I did not is I always made a complete, well written
argument about what I'm trying to implement. I always ask for a pair
of eyes to look at my code and give me feedback, and I always engage
with the community. In my opinion, disagreements made me do some of
the best pieces of code because I learn and grow from other people's
input.

That sounds a lot like a polite way to suggest that I did not engage
properly with the community. I have done my best to get people involved
while acknowledging the need to move relatively fast which IMO is
justified by the abyssal technical debt of OFBiz.


The boundary of a public API does not necessarily need to be agreed
upon RIGHT NOW!. I don't think the best way to move forward is to
enforce agreeing on what we should done before hand.

Instead of setting rules and boundaries on what should and should not
be done, I recommend instead collaborating on all work, one piece at a
time. As I said, I've worked on very large issues indeed and had no
problem working it with others and making it eventually to the code
base.

So whenever there is a disagreement or differing points of view, why
not start a new thread specific to that topic and work it out.
Everybody wants the best for the project, and communication is the key
to moving us forward IMHO.

I agree that communication is important as long as it enables people to
move forward, currently we are blocked.  If you look at the initial
discussion, you will see that this is precisely a deadlock in a *thread
specific* conversation between Michael and me that lead to this general
question.

This specific discussion is simply unsolvable because everybody is
basing their arguments on undocumented assumptions regarding what needs
to be preserved, what needs to be superseded, how things are supposed to
be done, etc.  This is basically a sterile “you broke my business
specific extension” vs “I want to improve the framework” debate where
both perspective can be seen as legitimate but are fundamentally
incompatible with each other in practice.

To recap the discussion, it appears that most people that have responded
to my call for defining/documenting what is part of OFBiz public API
feel this would be an unnecessary and counter-productive action and that
case by case discussions can be a more valuable substitute.

As a consequence I think we can close the subject now.

Thanks to everyone who took some of their time to participate.


Hi Mathieu,

Sorry I disagree, I for one would very like to have what you expressed at 
https://s.apache.org/5gryr in OFBiz in future

We can agree to disagree with Michael, and have it in trunk. As I said already, 
it would let several years for users (like Michael) to adjust.

Of course for that we need a vote and Michael not to veto this change on trunk 
with solid arguments.

Else, seeing yours and Samuel's recent framework efforts compared to the rest of the community, I fear the future of OFBiz is not bright. Apart if we 
consider the UI as the most important, which I don't...


I'm ready to engage a vote if you agree with me

Thanks

Jacques

Re: What is OFBiz public API?

[Mathieu Lirzin]

Hello,

Taher Alkhateeb  writes:

> While working on all the above, I never faced major obstacles. But the
> reason that I did not is I always made a complete, well written
> argument about what I'm trying to implement. I always ask for a pair
> of eyes to look at my code and give me feedback, and I always engage
> with the community. In my opinion, disagreements made me do some of
> the best pieces of code because I learn and grow from other people's
> input.

That sounds a lot like a polite way to suggest that I did not engage
properly with the community. I have done my best to get people involved
while acknowledging the need to move relatively fast which IMO is
justified by the abyssal technical debt of OFBiz.

> The boundary of a public API does not necessarily need to be agreed
> upon RIGHT NOW!. I don't think the best way to move forward is to
> enforce agreeing on what we should done before hand.
>
> Instead of setting rules and boundaries on what should and should not
> be done, I recommend instead collaborating on all work, one piece at a
> time. As I said, I've worked on very large issues indeed and had no
> problem working it with others and making it eventually to the code
> base.
>
> So whenever there is a disagreement or differing points of view, why
> not start a new thread specific to that topic and work it out.
> Everybody wants the best for the project, and communication is the key
> to moving us forward IMHO.

I agree that communication is important as long as it enables people to
move forward, currently we are blocked.  If you look at the initial
discussion, you will see that this is precisely a deadlock in a *thread
specific* conversation between Michael and me that lead to this general
question.

This specific discussion is simply unsolvable because everybody is
basing their arguments on undocumented assumptions regarding what needs
to be preserved, what needs to be superseded, how things are supposed to
be done, etc.  This is basically a sterile “you broke my business
specific extension” vs “I want to improve the framework” debate where
both perspective can be seen as legitimate but are fundamentally
incompatible with each other in practice.

To recap the discussion, it appears that most people that have responded
to my call for defining/documenting what is part of OFBiz public API
feel this would be an unnecessary and counter-productive action and that
case by case discussions can be a more valuable substitute.

As a consequence I think we can close the subject now.

Thanks to everyone who took some of their time to participate.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Re: What is OFBiz public API?

[Jacques Le Roux]

Great Message Taher!

Jacques

Le 20/01/2020 à 11:50, Taher Alkhateeb a écrit :

Hello Mathieu and all,

Thank you for this interesting and engaging thread. Before I make my
input, I would like to note that Apache OFBiz and its community from
my experience is welcoming of change and improvements. In my earlier
days in the project we went through massive changes to the system that
made tremendous differences including:

- Moving from ant to gradle
- introducing unit tests
- Redesigning the startup logic and possible flags
- Changing the distribution mechanism (remote dependencies instead of
inside the code base)
- Changing folder structures in components
- many others

While working on all the above, I never faced major obstacles. But the
reason that I did not is I always made a complete, well written
argument about what I'm trying to implement. I always ask for a pair
of eyes to look at my code and give me feedback, and I always engage
with the community. In my opinion, disagreements made me do some of
the best pieces of code because I learn and grow from other people's
input.

The boundary of a public API does not necessarily need to be agreed
upon RIGHT NOW!. I don't think the best way to move forward is to
enforce agreeing on what we should done before hand.

Instead of setting rules and boundaries on what should and should not
be done, I recommend instead collaborating on all work, one piece at a
time. As I said, I've worked on very large issues indeed and had no
problem working it with others and making it eventually to the code
base.

So whenever there is a disagreement or differing points of view, why
not start a new thread specific to that topic and work it out.
Everybody wants the best for the project, and communication is the key
to moving us forward IMHO.

My 2 cents

Cheers,
Taher Alkhateeb

On Sun, Jan 12, 2020 at 4:56 AM Mathieu Lirzin
 wrote:

Hello Paul, Jacopo and Olivier,

Paul Foxworthy  writes:


Being clearer about it would be a good thing.  There is still the
potential that a change that seems simple and straightforward to one
contributor is disruptive to others. And if it truly is disruptive, we
should collaborate on what comes next. Are the benefits worth the
disruption? Should there be some sort of deprecation, phase-out or
backwards compatibility option? I can't clearly define what "big" is,
but I know it when I see it.

IMO “We Should collaborate” is wishful thinking, because in practice
what happens most of the time is that only one person is doing the work
that he/she is proposing and the rest is doing +1 or -1. When someone
objects it often just means more burden for the person doing the work
because the person objecting will just add extra requirements.

Don't get me wrong, I find that situation normal because nobody is doing
a better job than someone who is motivated about it. Moreover people
with “ideas” but which are not responsible for implementing those tend
to advocate less practicable ideas.

It is just that I don't think we should expect the community to work
collectively better than the norm by relying on collaboration to be able
to get things done.

Jacopo Cappellato  writes:


There will always be a tension between guaranteeing backward compatibility
for the existing user base and the efforts to maintain our codebase,
enhance/refactor/innovate it.
Considering the peculiar nature of OFBiz, I don't think that trying to
define the areas that are part of the "public API" and then guarantee
backward compatibility only for them, will resolve this tension. In fact
there may be cases in which the "public API" should change in a
non-backward compatible way such as:
* the cost of maintaining a backward compatible feature is too high for our
community (or there is not enough interest in the active community in
maintaining it)
* the change is required to fix a flaw or a security vulnerability
* some fundamental (and important to the community) architectural change
may not be backward compatible
* etc...

Specifying a public API does not mean necessarily preserving it. It just
mean knowing when a border is being crossed:

- The developer changing something that is part of that API
- The user using an undocumented mechanism that happen to be here.


Apart from this, even the definition of "public API" for OFBiz is
troublesome considering for example that potentially any OFBiz service can
be used by "client" code and as a consequence in theory our community
should never change the behavior (or remove) a service.

Ideally yes because the notion of services is about providing something
to others.  To reduce the compatibility cost it should be possible to
make “convenience” services that are only meant to be used locally by
other services, be implemented as a private/package level helper methods
instead of a service.

I don't have specific examples in mind but I am pretty sure that there
exist a lot of those.


There are cases (like the one that triggered this conversation) that 

Re: What is OFBiz public API?

[Taher Alkhateeb]

Hello Mathieu and all,

Thank you for this interesting and engaging thread. Before I make my
input, I would like to note that Apache OFBiz and its community from
my experience is welcoming of change and improvements. In my earlier
days in the project we went through massive changes to the system that
made tremendous differences including:

- Moving from ant to gradle
- introducing unit tests
- Redesigning the startup logic and possible flags
- Changing the distribution mechanism (remote dependencies instead of
inside the code base)
- Changing folder structures in components
- many others

While working on all the above, I never faced major obstacles. But the
reason that I did not is I always made a complete, well written
argument about what I'm trying to implement. I always ask for a pair
of eyes to look at my code and give me feedback, and I always engage
with the community. In my opinion, disagreements made me do some of
the best pieces of code because I learn and grow from other people's
input.

The boundary of a public API does not necessarily need to be agreed
upon RIGHT NOW!. I don't think the best way to move forward is to
enforce agreeing on what we should done before hand.

Instead of setting rules and boundaries on what should and should not
be done, I recommend instead collaborating on all work, one piece at a
time. As I said, I've worked on very large issues indeed and had no
problem working it with others and making it eventually to the code
base.

So whenever there is a disagreement or differing points of view, why
not start a new thread specific to that topic and work it out.
Everybody wants the best for the project, and communication is the key
to moving us forward IMHO.

My 2 cents

Cheers,
Taher Alkhateeb

On Sun, Jan 12, 2020 at 4:56 AM Mathieu Lirzin
 wrote:
>
> Hello Paul, Jacopo and Olivier,
>
> Paul Foxworthy  writes:
>
> > Being clearer about it would be a good thing.  There is still the
> > potential that a change that seems simple and straightforward to one
> > contributor is disruptive to others. And if it truly is disruptive, we
> > should collaborate on what comes next. Are the benefits worth the
> > disruption? Should there be some sort of deprecation, phase-out or
> > backwards compatibility option? I can't clearly define what "big" is,
> > but I know it when I see it.
>
> IMO “We Should collaborate” is wishful thinking, because in practice
> what happens most of the time is that only one person is doing the work
> that he/she is proposing and the rest is doing +1 or -1. When someone
> objects it often just means more burden for the person doing the work
> because the person objecting will just add extra requirements.
>
> Don't get me wrong, I find that situation normal because nobody is doing
> a better job than someone who is motivated about it. Moreover people
> with “ideas” but which are not responsible for implementing those tend
> to advocate less practicable ideas.
>
> It is just that I don't think we should expect the community to work
> collectively better than the norm by relying on collaboration to be able
> to get things done.
>
> Jacopo Cappellato  writes:
>
> > There will always be a tension between guaranteeing backward compatibility
> > for the existing user base and the efforts to maintain our codebase,
> > enhance/refactor/innovate it.
> > Considering the peculiar nature of OFBiz, I don't think that trying to
> > define the areas that are part of the "public API" and then guarantee
> > backward compatibility only for them, will resolve this tension. In fact
> > there may be cases in which the "public API" should change in a
> > non-backward compatible way such as:
> > * the cost of maintaining a backward compatible feature is too high for our
> > community (or there is not enough interest in the active community in
> > maintaining it)
> > * the change is required to fix a flaw or a security vulnerability
> > * some fundamental (and important to the community) architectural change
> > may not be backward compatible
> > * etc...
>
> Specifying a public API does not mean necessarily preserving it. It just
> mean knowing when a border is being crossed:
>
> - The developer changing something that is part of that API
> - The user using an undocumented mechanism that happen to be here.
>
> > Apart from this, even the definition of "public API" for OFBiz is
> > troublesome considering for example that potentially any OFBiz service can
> > be used by "client" code and as a consequence in theory our community
> > should never change the behavior (or remove) a service.
>
> Ideally yes because the notion of services is about providing something
> to others.  To reduce the compatibility cost it should be possible to
> make “convenience” services that are only meant to be used locally by
> other services, be implemented as a private/package level helper methods
> instead of a service.
>
> I don't have specific examples in mind but I am pretty sure that there
> exist a 

Re: Fwd: Re: What is OFBiz public API?

[Jacques Le Roux]

Thanks Mathieu,

It's quite clear to me now

Jacques

Le 11/01/2020 à 21:52, Mathieu Lirzin a écrit :

Hello Jacques,

Jacques Le Roux  writes:


Le 05/01/2020 à 18:32, Mathieu Lirzin a écrit :

Michael Brohl  writes:


This project is not only about tech, it has a user base with serious
business running on base of OFBiz. This has always to be considered as
serious as good technical solutions should be considered. So we cannot
simply change things because single contributors like other technical
solutions better. We have to remain downwards compatible and manage
deprecation of features carefully.

First to clarify things, making evolutions in the framework is not about
developers changing arbitrary stuff, it is about structuring internals
in an understandeable way to enable correctness and the inclusion of new
features that satisfies evolving requirements.

Maybe you could clarify what you want to achieve. I have the feeling
that you have a long term view and the “component-load.xml change” is
only a step, right?

Yes I am/was working on the support of Jar distribution of OFBiz
allowing to separate the source code and compilation process of the
framework from the development of plugins. Basically it means
transforming OFBiz from a project template into a library to improve the
extensibility and reusability of both the framework and the plugins.

Moreover this work is contributing to the effort of facilitating the
deployment of OFBiz in production environments (in the continuity of
‘gradlew distTar’) by allowing the move of specific configuration out of
the source tree and deploying every resources inside the Jar.

I have described the rationale for this work in [1].


[...]

For the record, Without the ability to safely refactor a large subset of
the codebase that have the status of “implementation details”, I will
simply stop contributing to OFBiz because I don't have time for endless
discussions with people blaming my community work because their
extensions happen to rely on unspecified behavior.

For the current case, the most important question is to know if both
solutions could work at the same time, and if yes at which cost? Have
you an idea about that?

Sure we can keep both options which is the easy way to settle a
disagreement in the short term, I have allowed this in my last patch on
OFBIZ-11296 [2] by supporting both  and “component-load.xml”
at the same time with a priority on “component-load.xml”.

However to be able to continue the work of [1] it is important to remove
the usage of “component-load.xml” inside the framework.

[1] 
https://lists.apache.org/thread.html/c2612f1e296b6ea15872185871d3a9d83d6a4afc6d2a76f7a336a126%40%3Cdev.ofbiz.apache.org%3E
[2] 
https://issues.apache.org/jira/secure/attachment/12989898/12989898_OFBIZ-11296_ignore-depends-on-when-a-component-load.xml-is-prese.patch

Re: What is OFBiz public API?

[Mathieu Lirzin]

Hello Paul, Jacopo and Olivier,

Paul Foxworthy  writes:

> Being clearer about it would be a good thing.  There is still the
> potential that a change that seems simple and straightforward to one
> contributor is disruptive to others. And if it truly is disruptive, we
> should collaborate on what comes next. Are the benefits worth the
> disruption? Should there be some sort of deprecation, phase-out or
> backwards compatibility option? I can't clearly define what "big" is,
> but I know it when I see it.

IMO “We Should collaborate” is wishful thinking, because in practice
what happens most of the time is that only one person is doing the work
that he/she is proposing and the rest is doing +1 or -1. When someone
objects it often just means more burden for the person doing the work
because the person objecting will just add extra requirements.

Don't get me wrong, I find that situation normal because nobody is doing
a better job than someone who is motivated about it. Moreover people
with “ideas” but which are not responsible for implementing those tend
to advocate less practicable ideas.

It is just that I don't think we should expect the community to work
collectively better than the norm by relying on collaboration to be able
to get things done.

Jacopo Cappellato  writes:

> There will always be a tension between guaranteeing backward compatibility
> for the existing user base and the efforts to maintain our codebase,
> enhance/refactor/innovate it.
> Considering the peculiar nature of OFBiz, I don't think that trying to
> define the areas that are part of the "public API" and then guarantee
> backward compatibility only for them, will resolve this tension. In fact
> there may be cases in which the "public API" should change in a
> non-backward compatible way such as:
> * the cost of maintaining a backward compatible feature is too high for our
> community (or there is not enough interest in the active community in
> maintaining it)
> * the change is required to fix a flaw or a security vulnerability
> * some fundamental (and important to the community) architectural change
> may not be backward compatible
> * etc...

Specifying a public API does not mean necessarily preserving it. It just
mean knowing when a border is being crossed:

- The developer changing something that is part of that API
- The user using an undocumented mechanism that happen to be here.

> Apart from this, even the definition of "public API" for OFBiz is
> troublesome considering for example that potentially any OFBiz service can
> be used by "client" code and as a consequence in theory our community
> should never change the behavior (or remove) a service.

Ideally yes because the notion of services is about providing something
to others.  To reduce the compatibility cost it should be possible to
make “convenience” services that are only meant to be used locally by
other services, be implemented as a private/package level helper methods
instead of a service.

I don't have specific examples in mind but I am pretty sure that there
exist a lot of those.

> There are cases (like the one that triggered this conversation) that may
> involve some community discussion because it is difficult to figure out if
> the cost (in terms of pains for our users) is worth the gain (in terms of
> better/cleaner code): sometimes these discussions can be frustrating but if
> we all try to stay open (to the other point of view), positive, flexible,
> patient I am sure we will cope with them.

I agree that it is normal to spend some potentially long time discussing
the tradeoffs when considering a change that impact the public API
(whether it just adds a new feature or it breaks/deprecate existing
ones)

However knowing the boundaries between the public/private API is very
useful to understand each other and agree on the extent of those
tradeoffs.

For example in the current situation it happens that Michael is
considering “applications/component-load.xml” part of the implicit
public API¹ and as a consequence requires a community validation, a
deprecation period and a migration guides to be provided. When in the
same time I am considering it as an implementation detail that don't
need such extra work on my side.

Basically listening to each other in the context of an objection to a
change means putting all the burden on the framework developper which
either have to spend time convincing users that they should not use one
thing and migrate to another thing, or adding corner cases inside the
code to preverve the specific behavior expected by users.

Moreover even if the burden is all on the developer this still sucks
because it works only for users involved in the development (subscribed
to ML), but others will have to deal with their own unexpected breakage
without having the developer to help them.

Olivier Heintz  writes:

> 4) implementation detail or core choice, often, it's depend of point
> of view you use On ERP development / implementation there 

Re: Fwd: Re: What is OFBiz public API?

[Mathieu Lirzin]

Hello Jacques,

Jacques Le Roux  writes:

> Le 05/01/2020 à 18:32, Mathieu Lirzin a écrit :
>>
>> Michael Brohl  writes:
>>
>>> This project is not only about tech, it has a user base with serious
>>> business running on base of OFBiz. This has always to be considered as
>>> serious as good technical solutions should be considered. So we cannot
>>> simply change things because single contributors like other technical
>>> solutions better. We have to remain downwards compatible and manage
>>> deprecation of features carefully.
>>
>> First to clarify things, making evolutions in the framework is not about
>> developers changing arbitrary stuff, it is about structuring internals
>> in an understandeable way to enable correctness and the inclusion of new
>> features that satisfies evolving requirements.
>
> Maybe you could clarify what you want to achieve. I have the feeling
> that you have a long term view and the “component-load.xml change” is
> only a step, right?

Yes I am/was working on the support of Jar distribution of OFBiz
allowing to separate the source code and compilation process of the
framework from the development of plugins. Basically it means
transforming OFBiz from a project template into a library to improve the
extensibility and reusability of both the framework and the plugins.

Moreover this work is contributing to the effort of facilitating the
deployment of OFBiz in production environments (in the continuity of
‘gradlew distTar’) by allowing the move of specific configuration out of
the source tree and deploying every resources inside the Jar.

I have described the rationale for this work in [1].

> [...]
>> For the record, Without the ability to safely refactor a large subset of
>> the codebase that have the status of “implementation details”, I will
>> simply stop contributing to OFBiz because I don't have time for endless
>> discussions with people blaming my community work because their
>> extensions happen to rely on unspecified behavior.
>
> For the current case, the most important question is to know if both
> solutions could work at the same time, and if yes at which cost? Have
> you an idea about that?

Sure we can keep both options which is the easy way to settle a
disagreement in the short term, I have allowed this in my last patch on
OFBIZ-11296 [2] by supporting both  and “component-load.xml”
at the same time with a priority on “component-load.xml”.

However to be able to continue the work of [1] it is important to remove
the usage of “component-load.xml” inside the framework.

[1] 
https://lists.apache.org/thread.html/c2612f1e296b6ea15872185871d3a9d83d6a4afc6d2a76f7a336a126%40%3Cdev.ofbiz.apache.org%3E
[2] 
https://issues.apache.org/jira/secure/attachment/12989898/12989898_OFBIZ-11296_ignore-depends-on-when-a-component-load.xml-is-prese.patch

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Re: What is OFBiz public API?

[Olivier Heintz]

Hi, everybody,

It is not easy to add something in the discussion among the different 
arguments! (and on the different topics)

I will, however, try to give some summary of my position on the various 
subjects discussed.

1) As Jacques recall "Community over code" so :
  1.1 : It is important to trust all the members of the community in their will 
to improve OFBiz
  1.2 : the discussion cannot take place in 2 or 3 days, it is rather in a week 
or a month that it will take place.
(I don't have 2 or 3 hours to read (and answer) the mail every day, but I can 
find them during a week..
This is an advantage of mailing list discussions over chat discussions.

2) Deprecated (with documentation) before removing is a very good solution,
in a release migration process, when something has disappears (not yet move to 
the new solution), it's more easy to retrieve file when it was
deprecated to read associated documentation and know how to migrate.
  the rule is the same even it's simple util java method ;-)

3) I clearly not understand all implications/advantages/constrains ... or 
whatever about depend-on or component-load
what I only can be say is :
In a functional consultant point of view which want to configure an ofbiz with 
a lot of plugins (between 20 and 40) it's more easy to have a depend-on
on each plugin to be sure the loading order will be correct.

4) implementation detail or core choice, often, it's depend of point of view 
you use
On ERP development / implementation there are multiple type of people working 
on it with each one, specifics knowledge, usage, practice and point of view.
When someone says, it's a big change,
   start by trusting him and find out which process is affected to propose a 
solution;
   he doesn't want to block something, he wants to help you come up with a 
better solution that solves more cases.

5) patching is wrong : very often there are patch because hook or extend 
mechanism not exist so
When a plugin contains a patch, framework expert should explain
  or how to use existing extensibility mechanism to avoid the patch
  or how to improve the framework to provide a extensibility mechanism for this 
case
this point is also an example about previous point ;-)

6) what is OFBiz,
  - not only a public API ;-)
  - not only a framework
  - currently not a OOTB ERP  but I hope what, in future, there will contain 
some OOTB applications
In my long term view Apache OFBiz could be like linux, a core/kernel as small 
as possible with multiple components (the plugins) and so with some
distributions.
So, clearly, for me it is not possible to define the boundary between what is 
public (with backward compatibility) and what is only implementation.
Even on data model I can give some examples where not everyone will agree on 
what is in kernel and what is in components

just my 6 cents


Olivier

Le 06/01/2020 à 10:29, Samuel Trégouët a écrit :

> 
> I also hope others contributors will eventually join (many thanks to
> Jacques for joining!) since this discussion  seems to me to be larger
> than a particular feature (component-load.xml): it is about the process
> of contributing to OFBiz.
> 
>

Re: What is OFBiz public API?

[Jacques Le Roux]

Hi Michael,

I purposely created a new thread because then there were no answers and I 
wanted to create a new thread not inserted in the initial thread.

But because I got issue with my temporary IP address (due to my Internet 
provider) it was sent later.

OK, copying there...

Jacques

Le 08/01/2020 à 08:32, Michael Brohl a écrit :

Hi Jacques,

you've created another mail thread with exactly the same title. The first thread already has answers which would get lost if your new thread would 
be answered.


I propose that you simply answer the first thread directly.

Thanks,

Michael


Am 08.01.20 um 05:38 schrieb Jacques Le Roux:

Hi All,

Following Mathieu's question about "OFBiz public API" and the points he already 
mentioned I decided to create a new thread.
I don't know what will happen with the other tread (mostly about 
removing/replacing component-load.xml). That's not the subject here.

Mathieu wrote:

   <>

I tend to agree with this list. From the top of my head, I'd add (maybe not a 
the same level and maybe we need to define level/s)

1. Obviously, the Java API !
2. All things (why not?) supported by XSD files.
   It includes Entity and Services Engines, not all feature are 
documented/supported by XSD files though.
   It also includes component-loader.xsd and ofbiz-component.xsd which are 
somehow parts of the subject of the other thread
3. OFBiz Gradle tasks

I don't know what to say about Groovy scripts which are replacing simple-method 
services.

What else?

Jacques

Re: What is OFBiz public API?

[Jacques Le Roux]

Le 05/01/2020 à 18:32, Mathieu Lirzin a écrit :

Hello,

The arguments provided by Michael are very general and go beyond the
specific question of “component-load.xml” so I am opening a general
discussion about how to make OFBiz evolve smoothly by precising the
extent of its public API.

I urge other contributors to join this discussion which is crucial to
define our capability to work together as a community and my willing to
continue to participate.

Michael Brohl  writes:


This project is not only about tech, it has a user base with serious
business running on base of OFBiz. This has always to be considered as
serious as good technical solutions should be considered. So we cannot
simply change things because single contributors like other technical
solutions better. We have to remain downwards compatible and manage
deprecation of features carefully.

First to clarify things, making evolutions in the framework is not about
developers changing arbitrary stuff, it is about structuring internals
in an understandeable way to enable correctness and the inclusion of new
features that satisfies evolving requirements.

Backwards compability only makes sense when something has a public API
otherwise every evolution is a breaking change. In OFBiz we lack a
proper specification of what is a feature provided by the framework
subject to backward compatibility and what is an implementation detail
that can evolve/disappear between version silently. We rely on an
informal consensus to distinguish between the two.

The fact that some mechanism appears to be used in production is a valid
argument against its removal only if that mechanism is part of the
public API, otherwise it is up to the client code to adapt.

My broad understanding of what is part of OFBiz public API is:
  - the plugin mechanism
  - the data model and data access (Entity Engine)
  - The ability to call existing services and implement new ones (Service 
Engine)
  - the HTTP routing mechanism (Event Handler)
  - the various configuration files location in “{component}/config” 
directories.


[...]
If you read carefully what I previously wrote, there are several uses
for the applications component-load.xml:

  * deactivation of unused component(s) by commenting out the
load-component entry (why load marketing resources if you don't use
the component at the moment)
  * addition of components (yes, I've seen projects where this was not
done through the hot-deploy mechanism)
  * ordering these components in the right load order

While you can argument that these might be "wrong" approaches, they
are technically valid and used in customer projects. Therefore we
cannot simply switch the mechanisms without a proper deprecation
period.

The general problem here is not to know if things are wrong or
technically valid, it is to know if something is part of the public API
or is an implementation detail. This determines how to handle an
evolution on that part. Something wrong but part of the public API like
using XML for code should be handled with care (deprecation, migration
guides), but something technically valid but inappropriate like patching
framework Java source code from a plugin should be ignored.

In the case of ordering/enabling core components I consider it as an
implementation detail. If a component inside framework/applications is
effectively optional (like the marketing example you brought) it should
eventually be moved in the official plugins if we actually want to
provides the capability for users to disable it. However users should
not be entitled to think that they can freely desactivate/reorder/add
new components inside the framework/applications directory and that such
modifications will continue to work in a future release.

The larger question is about knowing if the internal organisation of the
files inside the "framework/applications" directories with the exception
of the “config” directories is considered part of OFBiz public API or
not?  What do people think?

For the record, Without the ability to safely refactor a large subset of
the codebase that have the status of “implementation details”, I will
simply stop contributing to OFBiz because I don't have time for endless
discussions with people blaming my community work because their
extensions happen to rely on unspecified behavior.

Hi All,

Following Mathieu's question about "OFBiz public API" and the points he already 
mentioned I decided to create a new thread.
I don't know what will happen with the other tread (mostly about 
removing/replacing component-load.xml). That's not the subject here.

Mathieu wrote:

   <>

I tend to agree with this list. From the top of my head, I'd add (maybe not a 
the same level and maybe we need to define level/s)

1. Obviously, the Java API !
2. All things (why not?) supported by XSD files.
   It includes Entity and Services Engines, not all feature are 
documented/supported by XSD files though.
   It also includes component-loader.xsd 

Re: What is OFBiz public API?

[Jacopo Cappellato]

There will always be a tension between guaranteeing backward compatibility
for the existing user base and the efforts to maintain our codebase,
enhance/refactor/innovate it.
Considering the peculiar nature of OFBiz, I don't think that trying to
define the areas that are part of the "public API" and then guarantee
backward compatibility only for them, will resolve this tension. In fact
there may be cases in which the "public API" should change in a
non-backward compatible way such as:
* the cost of maintaining a backward compatible feature is too high for our
community (or there is not enough interest in the active community in
maintaining it)
* the change is required to fix a flaw or a security vulnerability
* some fundamental (and important to the community) architectural change
may not be backward compatible
* etc...

Apart from this, even the definition of "public API" for OFBiz is
troublesome considering for example that potentially any OFBiz service can
be used by "client" code and as a consequence in theory our community
should never change the behavior (or remove) a service.

In an ideal World, our community would have infinite resources and would be
able to innovate and maintain our codebase while at the same time
guaranteeing backward compatibility for all the users... in our reality we
have to take the best decisions case by case. Potentially any change will
impact someone and we should accept this; our community is also here to
support and provide advices to users facing issues during migrations. I
don't think it is possible or even meaningful to ask that any
change/innovation must be backward compatible. This doesn't mean that we
should not care at all about backward compatibility because there are
changes that may have a large impact while not being so relevant for the
innovation.

There are cases (like the one that triggered this conversation) that may
involve some community discussion because it is difficult to figure out if
the cost (in terms of pains for our users) is worth the gain (in terms of
better/cleaner code): sometimes these discussions can be frustrating but if
we all try to stay open (to the other point of view), positive, flexible,
patient I am sure we will cope with them.

Just my 2 cents

Jacopo

Re: What is OFBiz public API?

[Michael Brohl]

Hi Jacques,

you've created another mail thread with exactly the same title. The 
first thread already has answers which would get lost if your new thread 
would be answered.


I propose that you simply answer the first thread directly.

Thanks,

Michael


Am 08.01.20 um 05:38 schrieb Jacques Le Roux:

Hi All,

Following Mathieu's question about "OFBiz public API" and the points 
he already mentioned I decided to create a new thread.
I don't know what will happen with the other tread (mostly about 
removing/replacing component-load.xml). That's not the subject here.


Mathieu wrote:

   < - The ability to call existing services and implement new ones 
(Service Engine)

 - the HTTP routing mechanism (Event Handler)
 - the various configuration files location in 
“{component}/config” directories.>>


I tend to agree with this list. From the top of my head, I'd add 
(maybe not a the same level and maybe we need to define level/s)


1. Obviously, the Java API !
2. All things (why not?) supported by XSD files.
   It includes Entity and Services Engines, not all feature are 
documented/supported by XSD files though.
   It also includes component-loader.xsd and ofbiz-component.xsd which 
are somehow parts of the subject of the other thread

3. OFBiz Gradle tasks

I don't know what to say about Groovy scripts which are replacing 
simple-method services.


What else?

Jacques






smime.p7s
Description: S/MIME Cryptographic Signature

Re: What is OFBiz public API?

[Paul Foxworthy]

On Mon, 6 Jan 2020 at 20:29, Samuel Trégouët 
wrote:


>   what is OFBiz public API?
>
> In my opinion we need an answer for this question otherwise we need to
> discuss every single changement! which seems to be really cumbersome!
> And even if we discuss every single changement how to decide it is good
> for our community: *one* other contributor thumb-up is enough? maybe
> *two*? do we need to wait forever if others don't care about a
> particular changement?
>

Hi Samuel,

I think this isn't a choice between having a public API and cumbersome
decision making. An API is a bits and bytes thing and won't magically
streamline decision making. A better architected, designed and understood
API will help **people** get an **initial impression** of what is easy to
refactor and re-implement.  Being clearer about it would be a good thing.
There is still the potential that a change that seems simple and
straightforward to one contributor is disruptive to others. And if it truly
is disruptive, we should collaborate on what comes next. Are the benefits
worth the disruption? Should there be some sort of deprecation, phase-out
or backwards compatibility option? I can't clearly define what "big" is,
but I know it when I see it.

And as a starting point for making these decisions, I suggest
https://www.apache.org/foundation/how-it-works.html#decision-making .

Cheers

Paul Foxworthy

-- 
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://www.coherentsoftware.com.au/
Email: i...@coherentsoftware.com.au

Re: What is OFBiz public API?

[Jacques Le Roux]

Le 07/01/2020 à 00:56, Mathieu Lirzin a écrit :

Hello,

Jacques Le Roux  writes:


Le 06/01/2020 à 10:29, Samuel Trégouët a écrit :

what is OFBiz public API?

In my opinion we need an answer for this question otherwise we need to
discuss every single changement! which seems to be really cumbersome!
And even if we discuss every single changement how to decide it is good
for our community: *one* other contributor thumb-up is enough? maybe
*two*? do we need to wait forever if others don't care about a
particular changement?

Mathieu and you make good points with the notion of "OFBiz public API".
It would be indeed good to have a such concept to officially (ie w/ a prior 
consensus) collect all parts that always need deeper discussions and consents.

But I fear it's not easy to define and this needs if not a deep discussion at 
least a long one (see the kinda recursion here?).

So before havi ng all agreed about what the "OFBiz public API" is I
think we need to cure the present issue. Except if Mathieu is pleased
to wait before this is agreed on.

I think it is an important discussion that is overdue.

Given that I need a break to both get over the frustration of the recent
heated discussions and focus on my research work, As far as I am
concerned this is a good moment for this discussion to happen.

The goal of this discussion would be to define the boundaries between
deprecated/stable/experimental/internal code by considering both the
stability requirements that are important for production environments
(that need to be able to upgrade smoothly) and the capability for
developers to refactor code that is necessary to be able to implement
the features allowing OFBiz to remain relevant in the future.

Hi Mathieu,

I started the "What is OFBiz public API?" thread

Actually I did yesterday but had to reboot my Internet box because of IP blocked issue and then ask barracudacentral.org to remove my IP from their 
blocked list (I don't thanks my Internet Provider!)


Jacques

Re: What is OFBiz public API?

[Michael Brohl]

Hi Samuel,

inline...

Am 06.01.20 um 10:29 schrieb Samuel Trégouët:

...
Michael has started to discuss because he had missed the commit which
removes component-load.xml in applications and framework and he claimed
[2] that we didn't discuss in d...@obfiz.apache.org before: completely
true! Why do we need to discuss such an implementation detail? Some


I already explained why the applications component-load.xml is not an 
implementation detail but used in real life projects.



argue that we have to discuss before intruducing any *big* changement
:confused: What is a *big* changement? In software library/framework it
is quite easy to answer: a big changement is a breaking in public api.


It's not about being a big change, it's about breaking existing 
mechanisms/configurations on the user side.



So here is the question from Mathieu:

   what is OFBiz public API?

In my opinion we need an answer for this question otherwise we need to
discuss every single changement! which seems to be really cumbersome!


It's common that you propose a change, ask for comments/review and maybe 
advice if you work in a community, yes. It saves time and energy and 
adds value to the final solution (mostly).


...


OFBiz is not just a library or core framework, it is a multi-level project:

* a web development framework

* a web based ERP system on base of this framework

* highly flexible and extendable through various mechanisms.

Like so many frameworks, OFBiz is not different according to this
points.
And like so many frameworks which are extendable we need API to ease
extension.


Yes, but it makes a diffrence in the argumentation. The argument was, 
that the load configuration is an implementation detail.


This might be true for pure ERP users but it is not users who utilize 
existing mechanisms of the web development framework.



To my understanding, if we use depends-on exclusively for framework,
applications and plugins, this would not be possible anymore.

This is where you're wrong. From the beginning using depends-on in
framework doesn't imply using it in plugins! The thing which drive
Mathieu to revert is that you cannot, in *framework* override depends-on
with a component-load.xml. And here we are with the actual discussion:

1. component-load.xml in plugin directory seems to be feature (nobody
discuss this point)


That *might* be a misunderstanding (if Mathieu agrees on this point). My 
understanding was that he first implemented and committed the change for 
framework and applications (on Nov. 25).


From his mail in dev [1] and also the issue title of [2] I understand 
that the component-load mechanism should be removed *everywhere* 
afterwards. The dev mail would not make sense otherwise because already 
committed the work at the time of writing (two weeks before) and he 
announces to go on if noone objects.


I apologize if I missed a point here, maybe Mathieu can clarify this?

I do not object against the change in framework, because I consider the 
change of component-load in framework an exceptional use case.


For applications and plugins I have explained why we should not change 
the mechanism.




2. what about component-load.xml in framework and applications
directories? is it also a feature (in other words a public API) or an
implementation detail

  

[1]
https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04

This reference is a bit old and stated as wip so I will consider it
as irrelevant for our discussion ;)


This reference was only given to document that the mechanism is meant to 
be used in the way I described it. How old it is or if it is WIP does 
not make the meaning less relevant.



cheers,

Samuel

[1]: 
https://lists.apache.org/thread.html/c2612f1e296b6ea15872185871d3a9d83d6a4afc6d2a76f7a336a126%40%3Cdev.ofbiz.apache.org%3E
[2]: 
https://lists.apache.org/thread.html/7eab3d2ae3bbeadb184b02f75f7b2b86263532485e88ecba4d4dc780%40%3Cdev.ofbiz.apache.org%3E


Thanks,

Michael


[1] 
https://lists.apache.org/thread.html/441d038d1d000429dc1f09784db4b90bdc4cdd2b0e7c9bc4ccc9e48f%40%3Cdev.ofbiz.apache.org%3E


[2] https://issues.apache.org/jira/browse/OFBIZ-11296





smime.p7s
Description: S/MIME Cryptographic Signature

Re: What is OFBiz public API?

[Mathieu Lirzin]

Hello,

Jacques Le Roux  writes:

> Le 06/01/2020 à 10:29, Samuel Trégouët a écrit :
>>
>>what is OFBiz public API?
>>
>> In my opinion we need an answer for this question otherwise we need to
>> discuss every single changement! which seems to be really cumbersome!
>> And even if we discuss every single changement how to decide it is good
>> for our community: *one* other contributor thumb-up is enough? maybe
>> *two*? do we need to wait forever if others don't care about a
>> particular changement?
>
> Mathieu and you make good points with the notion of "OFBiz public API".
> It would be indeed good to have a such concept to officially (ie w/ a prior 
> consensus) collect all parts that always need deeper discussions and consents.
>
> But I fear it's not easy to define and this needs if not a deep discussion at 
> least a long one (see the kinda recursion here?).
>
> So before havi ng all agreed about what the "OFBiz public API" is I
> think we need to cure the present issue. Except if Mathieu is pleased
> to wait before this is agreed on.

I think it is an important discussion that is overdue.

Given that I need a break to both get over the frustration of the recent
heated discussions and focus on my research work, As far as I am
concerned this is a good moment for this discussion to happen.

The goal of this discussion would be to define the boundaries between
deprecated/stable/experimental/internal code by considering both the
stability requirements that are important for production environments
(that need to be able to upgrade smoothly) and the capability for
developers to refactor code that is necessary to be able to implement
the features allowing OFBiz to remain relevant in the future.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Re: What is OFBiz public API?

[Jacques Le Roux]

Hi Samuel,

Inline too...

Le 06/01/2020 à 10:29, Samuel Trégouët a écrit :

Hi all,
[snip...]

Michael has started to discuss because he had missed the commit which
removes component-load.xml in applications and framework and he claimed
[2] that we didn't discuss in d...@obfiz.apache.org before: completely
true! Why do we need to discuss such an implementation detail? Some
argue that we have to discuss before intruducing any *big* changement
:confused: What is a *big* changement? In software library/framework it
is quite easy to answer: a big changement is a breaking in public api.
So here is the question from Mathieu:

   what is OFBiz public API?

In my opinion we need an answer for this question otherwise we need to
discuss every single changement! which seems to be really cumbersome!
And even if we discuss every single changement how to decide it is good
for our community: *one* other contributor thumb-up is enough? maybe
*two*? do we need to wait forever if others don't care about a
particular changement?


Mathieu and you make good points with the notion of "OFBiz public API".
It would be indeed good to have a such concept to officially (ie w/ a prior 
consensus) collect all parts that always need deeper discussions and consents.

But I fear it's not easy to define and this needs if not a deep discussion at 
least a long one (see the kinda recursion here?).

So before having all agreed about what the "OFBiz public API" is I think we need to cure the present issue. Except if Mathieu is pleased to wait 
before this is agreed on.


Remember at the ASF most is about discussion and agreement. As the motto, which has has 
proved is efficiency over years, says: *"Community over code".*



[snip...]


Michael's
=
To my understanding, if we use depends-on exclusively for framework,
applications and plugins, this would not be possible anymore.

This is where you're wrong. From the beginning using depends-on in
framework doesn't imply using it in plugins! The thing which drive
Mathieu to revert is that you cannot, in *framework* override depends-on
with a component-load.xml.


So you say that depends-on and component-load.xml are incompatible?
This opens a new discussion to me. Because then we don't speak about 
deprecation but about replacement...



And here we are with the actual discussion:

1. component-load.xml in plugin directory seems to be feature (nobody
discuss this point)
2. what about component-load.xml in framework and applications
directories? is it also a feature (in other words a public API) or an
implementation detail


I tend to think that it's not a feature. I'd though not say that it's an "implementation detail" ;) As often in "real-life" it's not black or white. 
It's greyed and implications may be complex...




[1]
https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04

This reference is a bit old and stated as wip so I will consider it
as irrelevant for our discussion ;)


I think we can indeed consider a 10 years old WIP reference a "bit old". I even don't see how what was wrote then still currently applies *in trunk*. 
It even me wonder what I should to about OFBIZ-3500 :/ Certainly not close it!


My 2cts

Jacques



cheers,

Samuel

[1]: 
https://lists.apache.org/thread.html/c2612f1e296b6ea15872185871d3a9d83d6a4afc6d2a76f7a336a126%40%3Cdev.ofbiz.apache.org%3E
[2]: 
https://lists.apache.org/thread.html/7eab3d2ae3bbeadb184b02f75f7b2b86263532485e88ecba4d4dc780%40%3Cdev.ofbiz.apache.org%3E

Fwd: Re: What is OFBiz public API?

[Jacques Le Roux]

(re)Forwarded at Michael's request


 Message transféré 
Sujet : Re: What is OFBiz public API?
Date :  Sun, 5 Jan 2020 20:33:03 +0100
De :Jacques Le Roux 
Organisation :  Les Arts Informatiques
Pour :  dev@ofbiz.apache.org



Hi Mathieu,

Inline too...

Le 05/01/2020 à 18:32, Mathieu Lirzin a écrit :

Hello,

The arguments provided by Michael are very general and go beyond the
specific question of “component-load.xml” so I am opening a general
discussion about how to make OFBiz evolve smoothly by precising the
extent of its public API.

I urge other contributors to join this discussion which is crucial to
define our capability to work together as a community and my willing to
continue to participate.

Michael Brohl  writes:


This project is not only about tech, it has a user base with serious
business running on base of OFBiz. This has always to be considered as
serious as good technical solutions should be considered. So we cannot
simply change things because single contributors like other technical
solutions better. We have to remain downwards compatible and manage
deprecation of features carefully.

First to clarify things, making evolutions in the framework is not about
developers changing arbitrary stuff, it is about structuring internals
in an understandeable way to enable correctness and the inclusion of new
features that satisfies evolving requirements.


Maybe you could clarify what you want to achieve. I have the feeling that you have a long term view and the “component-load.xml change” is only a 
step, right?




Backwards compability only makes sense when something has a public API
otherwise every evolution is a breaking change. In OFBiz we lack a
proper specification of what is a feature provided by the framework
subject to backward compatibility and what is an implementation detail
that can evolve/disappear between version silently. We rely on an
informal consensus to distinguish between the two.

The fact that some mechanism appears to be used in production is a valid
argument against its removal only if that mechanism is part of the
public API, otherwise it is up to the client code to adapt.


I agree, that's why I asked Michael, in answer to his last email, if he could adapt his mechanism to "generate the resulting component-load.xml at 
build time" using the new proposed mechanism.  Of course it would not longer relies on the component-load.xml file (to be eventually removed) but on 
the new mechanism.





My broad understanding of what is part of OFBiz public API is:
- the plugin mechanism
- the data model and data access (Entity Engine)
- The ability to call existing services and implement new ones (Service Engine)
- the HTTP routing mechanism (Event Handler)
- the various configuration files location in “{component}/config” directories.


I think there are more, those are part of it.



[...]

If you read carefully what I previously wrote, there are several uses
for the applications component-load.xml:

* deactivation of unused component(s) by commenting out the
load-component entry (why load marketing resources if you don't use
the component at the moment)
* addition of components (yes, I've seen projects where this was not
done through the hot-deploy mechanism)
* ordering these components in the right load order

While you can argument that these might be "wrong" approaches, they
are technically valid and used in customer projects. Therefore we
cannot simply switch the mechanisms without a proper deprecation
period.

The general problem here is not to know if things are wrong or
technically valid, it is to know if something is part of the public API
or is an implementation detail. This determines how to handle an
evolution on that part. Something wrong but part of the public API like
using XML for code should be handled with care (deprecation, migration
guides), but something technically valid but inappropriate like patching
framework Java source code from a plugin should be ignored.


Fortunately I have never seen "patched framework Java source code from a 
plugin" :), but I agree about the idea.



In the case of ordering/enabling core components I consider it as an
implementation detail. If a component inside framework/applications is
effectively optional (like the marketing example you brought) it should
eventually be moved in the official plugins if we actually want to
provides the capability for users to disable it.


+1



However users should
not be entitled to think that they can freely desactivate/reorder/add
new components inside the framework/applications directory and that such
modifications will continue to work in a future release.


+1



The larger question is about knowing if the internal organisation of the
files inside the "framework/applications" directories with the exception
of the “config” directories is considered part of OFBiz public API or
not? What do people think?


It should not be b

Re: What is OFBiz public API?

[Samuel Trégouët]

Hi all,

> Am 05.01.20 um 18:32 schrieb Mathieu Lirzin:
> >
> > I urge other contributors to join this discussion which is crucial to
> > define our capability to work together as a community and my willing to
> > continue to participate.

I also hope others contributors will eventually join (many thanks to
Jacques for joining!) since this discussion  seems to me to be larger
than a particular feature (component-load.xml): it is about the process
of contributing to OFBiz.

Michael has started to discuss because he had missed the commit which
removes component-load.xml in applications and framework and he claimed
[2] that we didn't discuss in d...@obfiz.apache.org before: completely
true! Why do we need to discuss such an implementation detail? Some
argue that we have to discuss before intruducing any *big* changement
:confused: What is a *big* changement? In software library/framework it
is quite easy to answer: a big changement is a breaking in public api.
So here is the question from Mathieu:

  what is OFBiz public API?

In my opinion we need an answer for this question otherwise we need to
discuss every single changement! which seems to be really cumbersome!
And even if we discuss every single changement how to decide it is good
for our community: *one* other contributor thumb-up is enough? maybe
*two*? do we need to wait forever if others don't care about a
particular changement?


> OFBiz is not just a library or core framework, it is a multi-level project:
> 
> * a web development framework
> 
> * a web based ERP system on base of this framework
> 
> * highly flexible and extendable through various mechanisms.

Like so many frameworks, OFBiz is not different according to this
points.
And like so many frameworks which are extendable we need API to ease
extension.

> 
> To my understanding, if we use depends-on exclusively for framework, 
> applications and plugins, this would not be possible anymore.

This is where you're wrong. From the beginning using depends-on in
framework doesn't imply using it in plugins! The thing which drive
Mathieu to revert is that you cannot, in *framework* override depends-on
with a component-load.xml. And here we are with the actual discussion:

1. component-load.xml in plugin directory seems to be feature (nobody
   discuss this point)
2. what about component-load.xml in framework and applications
   directories? is it also a feature (in other words a public API) or an
   implementation detail

 
> [1] 
> https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04

This reference is a bit old and stated as wip so I will consider it
as irrelevant for our discussion ;)

cheers,

Samuel

[1]: 
https://lists.apache.org/thread.html/c2612f1e296b6ea15872185871d3a9d83d6a4afc6d2a76f7a336a126%40%3Cdev.ofbiz.apache.org%3E
[2]: 
https://lists.apache.org/thread.html/7eab3d2ae3bbeadb184b02f75f7b2b86263532485e88ecba4d4dc780%40%3Cdev.ofbiz.apache.org%3E


signature.asc
Description: signature

Re: What is OFBiz public API?

[Michael Brohl]

Hi Mathieu,

inline...

Am 05.01.20 um 18:32 schrieb Mathieu Lirzin:

Hello,

The arguments provided by Michael are very general and go beyond the
specific question of “component-load.xml” so I am opening a general
discussion about how to make OFBiz evolve smoothly by precising the
extent of its public API.

I urge other contributors to join this discussion which is crucial to
define our capability to work together as a community and my willing to
continue to participate.

Michael Brohl  writes:


This project is not only about tech, it has a user base with serious
business running on base of OFBiz. This has always to be considered as
serious as good technical solutions should be considered. So we cannot
simply change things because single contributors like other technical
solutions better. We have to remain downwards compatible and manage
deprecation of features carefully.

First to clarify things, making evolutions in the framework is not about
developers changing arbitrary stuff, it is about structuring internals
in an understandeable way to enable correctness and the inclusion of new
features that satisfies evolving requirements.

Backwards compability only makes sense when something has a public API
otherwise every evolution is a breaking change. In OFBiz we lack a
proper specification of what is a feature provided by the framework
subject to backward compatibility and what is an implementation detail
that can evolve/disappear between version silently. We rely on an
informal consensus to distinguish between the two.

The fact that some mechanism appears to be used in production is a valid
argument against its removal only if that mechanism is part of the
public API, otherwise it is up to the client code to adapt.


OFBiz is not just a library or core framework, it is a multi-level project:

* a web development framework

* a web based ERP system on base of this framework

* highly flexible and extendable through various mechanisms.

OFBiz users are service providers, utilizing OFBiz to provide software 
solutions as well as end users who are mainly using the applications. 
There's also a mix in the case where company employees use OFBiz as a 
web development framework to provide software solutions for their own 
company.


So it cannot be simplified to a scenario where the framework is "ours" 
and the users are proivided with the applications and a public API. So 
if the project has provided a mechanism to configure how components are 
loaded, we are also responsible for taking care of this if we want to 
make changes.





My broad understanding of what is part of OFBiz public API is:
  - the plugin mechanism
  - the data model and data access (Entity Engine)
  - The ability to call existing services and implement new ones (Service 
Engine)
  - the HTTP routing mechanism (Event Handler)
  - the various configuration files location in “{component}/config” 
directories.


The component-load.xml is also a configuration option which is utilized 
in projects.


There is some documentation on how to utilize OFBiz as a core framework 
by deactivating all components (old but still valid, see [1]).






[...]
If you read carefully what I previously wrote, there are several uses
for the applications component-load.xml:

  * deactivation of unused component(s) by commenting out the
load-component entry (why load marketing resources if you don't use
the component at the moment)
  * addition of components (yes, I've seen projects where this was not
done through the hot-deploy mechanism)
  * ordering these components in the right load order

While you can argument that these might be "wrong" approaches, they
are technically valid and used in customer projects. Therefore we
cannot simply switch the mechanisms without a proper deprecation
period.

The general problem here is not to know if things are wrong or
technically valid, it is to know if something is part of the public API
or is an implementation detail. This determines how to handle an
evolution on that part. Something wrong but part of the public API like
using XML for code should be handled with care (deprecation, migration
guides), but something technically valid but inappropriate like patching
framework Java source code from a plugin should be ignored.


I don't think that patching Java code is/was part of the initial 
discussion. We should not mix up things.





In the case of ordering/enabling core components I consider it as an
implementation detail. If a component inside framework/applications is


I don't agree, see above.



effectively optional (like the marketing example you brought) it should
eventually be moved in the official plugins if we actually want to
provides the capability for users to disable it. However users should


Even it it would be a plugin, you still need a mechanism to 
enable/disable it by configuration.


To my understanding, if we use depends-on exclusively for framework, 
applications and plugins, this