Hello Folks,
I love OFBiz. I met great people and I learned a lot. Our business would
not have existed nor succeeded without being introduced to OFBiz and the
community.
In the past couple of years, we hesitantly started switching projects
out of OFBiz due to multiple issues. I think these issues cannot be
easily solved without some kind of a major re-architecture of the
framework and so the purpose of this thread is to start a discussion to
see if there is interest / ideas for moving in that direction (so it's a
Hail Mary Pass).
I'm going to list in here:
- Problems
- Solution ideas
- Keep It
- Improve It
- Add It
- Remove It
- Quality control
- Tradeoffs
- Pros
- Cons
- Summary
Problems
I've identified problems not only from our past work with OFBiz but also
by working extensively with moqui (as a point of comparison) in the past
few years. Listing them below:*
*
_1. Lack of Code Isolation_:It is unlikely to do anything substantial in
OFBiz without touching the core repositories. Compare this with moqui or
some other web frameworks like say Ruby-On-Rails or django and you will
see that you can work independently of the development path in these
frameworks. This makes upgrades quite painfulwith patches and manual
interventions. Some in the community even introduced a whole patching
framework in the past.
_2. All or nothing:_In OFBiz, if you want to use the system, either take
the whole thing or take nothing. What if I don't want any user interface
for example, and just want to utilize the entities and services?
_3. Limited Re-usability:_Because of the way current components are
designed, it's difficult to take advantage of components in other
components. Sharing artifacts is pretty much limited to entities and
services. Even then they're not very shareable because they're designed
in a non-usable way for a certain context and certain settings and
database parameters.*
*
_4. Fragmented Actions:_**Instead of having unified XML actions, you
have fragmentation across different contexts (screens, services, SECAs,
etc). Furthermore, you cannot mix XML actions with groovy or other
artifacts except by calling into external scripts and things like that.
In comparison to moqui which is more flexible in XML actions I find it
restrictive to work with OFBiz actions and it's hard to review, debug,
or modify. I'm listing something below as an example for comparison [1]*
*
_5. Limited Routing capability:_**Routing is very minimal
(controller.xml). It cannot contain business logic and is limited either
to chaining requests or rendering screens. Sophisticated routing logic
is important for making rich systems that react to differing circumstances.
**
_6. Core framework difficult to change:_**OFBiz core was written
a/long/time ago, and back then you didn't have many libraries for
caching, transactions, DB pooling, Concurrency stuff, and others, so a
lot was hand crafted, and we have a lots of interconnected API that is
difficult to change, and most of these APIs depend on static util files
that just kept piling and piling over the years. Where is the service
engine implementation exactly? What about the entity engine? What about
widgets? They are all over the place! Pieces for XML parsing, others for
DOM models, and so on. The code is scattered and fixing it means going
through a maze of many classes that are interconnected and all
referencing each other. It's also hard to identify the signature or
figure out the "big picture" since you have essentially minimal
interfaces and no clear public API that you can refactor against. Pretty
much the whole thing is public API**[2].
**
_7. Web-apps design problem__:_ A big problem (in my opinion) in the
design is around having multiple web-apps, one for each component (or
more). This interrupts the flow of the app and makes it difficult to
come up with smooth solutions that work across the various screens and
instead we need to pass a key between the components (and face other
problems around session management). This restricts you in what you can
offer as a user experience.*
*
_8. Problematic UI design__:_ Another/big/problem (in my opinion) is
having the UI show everything (reference implementation). I never
understood the value of showing/everything/! If I want to understand the
data model, I will just look at the data model. The purpose of the UI is
not to show everything, but to implement specific useful use-cases. The
reference implementation we have is not doing that, and not offering
something very useful, instead it is showing everything! To improve
this, we need to switch from data-model based interface (party, order,
etc ...) to use-case based interface (project management, e-commerce,
ERP, CRM, etc ...)*
*
_9. Limited client rendering:_Because it is difficult to refactor the
framework, it's hard to pin-point where to go exactly to refactor all
the code needed to support more client-rendering technology (react, vue,
angular, etc ...) and make it interact smoothly with the back-end
(drop-downs, complex forms, etc ...). A lot of work needs to happen both
to the /themes and to /framework to allow and provide more flexibility
(also moving things to DB). If this step is not done you will be stuck
with your UI capabilities and won't catch up (unless you develop your
own UI which makes the UI of the community redundant and you let go of
the whole widget system anyway).*
*
_10. Security is difficult to implement:_One of my favorite features in
moqui is having security baked in the artifacts themselves to avoid
sprinkling all screens / services with security logic. I never even
worry about writing security related stuff until way close to the end of
delivering the project. So there needs to be a whole security layer that
sits below all the artifacts such that we don't even need to write
security code in the various screens and services and around certain
entities.*
*
_11. Limited Integration Solutions__:_ I know we have now a REST
component, but this is not the same as being a core feature of the
framework (essential for any modern web framework). Why should it be
tightly integrated? Many reasons: security, localization, mapping to
service engine, mapping to entities, etc ...*
*
_12. No Specialization:_Because of the big repository, and having
everything point to everything, it's hard to specialize. Now compare
this to the approach we have in our work which is easier to specialize
in. For example, I have a component for payment gateway integration,
another one for twilio (SMS integration), and I have application
components for each solution in its own repository (HR, insurance stuff,
many others) and each is living in its own git repo yet they work
together just fine and without touching one line of code in the
framework or even core components (despite depending on them and using
pieces of them). I can jump into new versions of the framework everyday
without breaking a sweat. When I work on component X, I'm only thinking
of that and nothing else. I don't have to worry about changing something
that would crash another component or bring the system down (happened to
me many times in the past).
Solutions
Whatever solutions we come up with I think they're going to be difficult
and would require community buy-in and support. I'm breaking this down
into things to keep, update, add and remove as follows:
Keep it
- XSD signatures (so entity definitions, service definitions, XML
actions though should expand)
- Basic constructs of what is a component (with basic directory structure)
- Data model
- Existing services (core data-model services)
- Java / groovy code that doesn't use core framework constructs or goes
too deep into the internals.
- Most used and useful APIs (Making queries, calling services, etc ...)
although it should all be converted to interfaces with separate
implementations where it's lacking.
Improve It
- Break the project to smaller git repositories.
- framework
- data model
- services
- common UI
- use-case applications (ERP, CRM, e-commerce, etc ...)
- plugins
- entire framework directory needs refurbishment and big chunks probably
require a delete / rewrite
- the web architecture (get rid of web-apps)
- the tests (lots of them need to be deleted anyway)
- Routing needs to be programmatic with ability to inject code.
Add It
- Configuration override mechanism to remove the need to touch the core.
Overrides include everything including routing, component registration,
etc ...
- REST integration into the core
- Artifact based security that maps users / groups to artifacts without
the need to inject security code into every screen
- A super-set of XML actions that can be applied in all contexts and can
embed groovy code
- New, simple use-case based user interface. It needs to be designed
such that it is easy to extend to avoid the all-or-nothing UI that
currently exists.
Remove It / Deprecate It / Overwrite It
- Most of the existing UI (needs to be replaced with simpler use-case
based interface)
- Lots of the core framework code (needs major rewrites, needs
interfaces, import libraries for things like transactions, security,
concurrency, etc ...)
- Probably some of the existing plugins might need to be deprecated.
Quality Control
Investing in such a huge effort requires that we change the way we view
the project after / while work is being completed. Code should be
approached with greater care and a lot of emphasis should be put on high
quality, readability, avoiding anti-patterns, enforcing good patterns
and best-practices. Some of the daily workflows need to improve /
change, more thorough auditing and peer-reviews, and so on.
Tradeoffs
It goes without saying that the above is a _lot_ of work and commitment
and requires a lot of effort. That's why I hesitated in starting this
thread, but I thought maybe showing the tradeoffs would make it easier
to weigh things out and decide on a direction to push.
Pros
- Pretty much most of the above mentioned problems will be resolved.
- The framework would be resilient and future proof.
- We will have more options and capabilities in the future, it would be
easier to incorporate new technologies as they emerge
- Development would go faster
- Technical Debt will be reduced
Cons
- Huge (maybe too huge) upfront costs in time and effort to make the switch
- Extra effort needed to ensure a smooth upgrade path for existing users
/ adopters / system integrators. A whole layer of code needs to be
written to ensure the ability to upgrade without massive rewrites.
- Risk of failure, exhaustion, inability to push through to the end
- Inconvenience to those with existing systems / solutions
Summary
Change is the only constant. Inability to adapt to the future means not
surviving. I'm not confident what is the best change possible or at what
pace, but I do know that lack of change is a death sentence to any
project. Maybe it's time to reflect and think of where we'd like to go
and this thread is sort of a brain-dump of possible ideas.
I look forward to hearing your thoughts.
Cheers,
Taher Alkhateeb
[1] Example mixed XML actions and groovy from moqui
<if condition="settlementTerm?.orderPmtServiceRegisterId">
<then>
<service-call name="update#mantle.order.OrderPart"
in-map="[orderId:orderId, orderPartSeqId:orderPartSeqId,
settlementTermId:settlementTermId]"/>
<entity-find-one entity-name="moqui.service.ServiceRegister"
value-field="serviceRegister">
<field-map field-name="serviceRegisterId"
from="settlementTerm.orderPmtServiceRegisterId"/>
</entity-find-one>
<script><![CDATA[
try {
ec.service.sync().name(serviceRegister.serviceName).parameters(context).softValidate(true).disableAuthz().call()
if (ec.message.hasError()) return
} catch (Throwable t) {
ec.logger.log(300, "Error calling auto order payments service for
SettlementTerm ${settlementTermId}", t)
}
]]></script>
</then>
<else>
<service-call name="mantle.account.PaymentServices.create#Payment"
out-map="context"
in-map="context + [amount:amount, amountUomId:amountUomId,
fromPartyId:fromPartyId, toPartyId:toPartyId]"/>
</else>
</if>
[2] https://ci.apache.org/projects/ofbiz/site/trunk/javadocs/
