wrote:
I understand that Sharan brought up the framework rewrite
subject at
ApacheCon, and some attendees felt that the framework is fine
and no
action
needs to be taken.
In this message, I will try to give a detailed explanation of
why a
framework rewrite is necessary. I don't plan to take any further
action on
this subject, because I've brought it up before without
success, and
I'm
tired of discussing it. It is my hope that the light bulb will
click
on in
someone's head and they will take action.
My Background
-------------
I became a member of the OFBiz community in 2004. I immediately
started
making contributions to the project by supplying patches to
the issue
tracker. In 2007, I became a committer. Most of my initial
work was
on the
UI and some work in the applications (mainly Asset Maintenance
and
Work
Effort). I stayed away from touching the framework code
because it was
deep, dark, and scary.
Eventually, I started to understand how the framework code
works and I
made some minor modifications. As my understanding grew, I
progressed
to
rewriting large swaths of framework code - making it thread-safe,
fault
tolerant, efficient, and easier to use.
I will list some of my contributions here, so everyone can have a
clear
understanding of my experience with the framework code:
New Features
User Preferences
Visual Themes
Custom UI Label XML File Format
Temporal Expressions
Data Type Conversion Framework
Screen Widget Boundary Comments
Metrics
Integrations
UEL
iCalendar
JSR 223
WebDAV
LDAP
Refactorings/Improvements
FlexibleStringExpander
FlexibleMapExpander
FOP Integration
FreeMarkerWorker
Date-Time Handling
Mini-language
Job Scheduler
In addition, I have performed innumerable framework bug fixes.
So, the contents of this message come from years of experience
mucking
about in the framework code.
Okay, let's get started...
Initial Problem Statement
-------------------------
In 2009, David Jones started a framework rewrite in a branch:
https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716
At the time, there was some agreement that a rewrite was
necessary,
but
there was disagreement as to how the rewrite should be
incorporated
into
the project:
https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3c455601.62605...@web63102.mail.re1.yahoo.com%3E
There were concerns that a rewrite would break backward
compatibility.
Work on the rewrite branch stopped. Eventually, Jacopo
suggested the
community be more accepting of backward-incompatible changes:
https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cd24f129d-4f9f-444e-84af-aca46f499...@hotwaxmedia.com%3e
Despite an effort to convince David to proceed with the framework
rewrite,
he ended up doing it in a separate project:
http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3c07565c88-4023-4d24-93a3-a4906e86f...@me.com%3E
This page describes differences between OFBiz and Moqui, and
within
it you
can extract information on the problems David was trying to
solve:
http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/
There was an email he sent out on the OFBiz dev list where he
listed
the
problems he saw in the framework, but I can't find it. The
rest of
this
message will include the issues he mentioned (the ones I
remember). I
was
in agreement with him at the time, and I still agree that a
framework
rewrite is necessary.
The Problems
------------
Code is scattered everywhere - due to an initial effort to
make the
framework modular. This causes serious problems. The mere fact
that
components like entityext and securityext EXIST makes it clear
that
there
are problems - those components should not be there. Also, we run
into the
recurring problem of circular dependencies (component A will
not build
unless component B is built, and component B will not build
unless
component A is built).
Bad separation of concerns. There are far too many examples of
classes
that try to be everything to everyone. This makes debugging
difficult, and
it makes maintenance/improvements a nightmare. [Using an analogy,
consider
an automobile design where a spark plug is not separate from a
transmission. Instead, the automobile uses a
spark-plug-transmission.
So
when the engine is running rough because the spark plug is
bad, you
have to
replace the spark plug AND the transmission.] A good framework
example can
be found in my rewrite of the mini-language code. Originally, the
models
AND the script execution context both contained script
behaviors -
making
debugging/improvements difficult. I changed it so only the models
contain
script behavior and the script execution context contains only
the
script
execution state.
Lack of good OO design. There are many places where a bit of
framework
functionality is contained in a single method that is hundreds or
thousands
of lines long. There is a term for that: Brittle Code. Code isn't
reused.
Instead, it is copy-and-pasted all over - so when a problem is
found
in the
C&P code, it has to be fixed in many places instead of one.
Fail-slow design. There are a lot of places in low-level code
where an
error condition is encountered, but instead of throwing an
exception,
the
error is ignored and maybe it is logged, or the code tries to
"guess"
at a
solution and then provide an arbitrary default behavior. I've
seen
many
developers struggle with debugging a problem because they
didn't look
at
the logs, or because the error was not logged and there is no
way of
knowing what caused it. They end up spending hours
single-stepping
through
code until it reaches the error.
Out-of-date code. A good example is the use of Javolution. That
library
was beneficial in the Java 1.4 days, but it is not necessary
today
because
of improved garbage collection. Another good example is DCL
code. DCL
was
used extensively in OFBiz, but it is clearly documented to be an
unreliable
design (I can get it to fail 90% of the time). Some DCL code
has been
replaced, but a lot of it is still there.
Portions of the API are overly complicated. Some methods
require a
collection of user-specified artifacts/arguments, which makes
client
code
complicated and verbose. (David solved that problem with his
Execution
Context.) Portions of the API are cluttered with unnecessary
"convenience
methods" - making the API harder to learn and memorize. In some
places, a
domain-specific API is spread across instance methods and static
methods
and across different classes - making the API hard to
understand and
use.
Yes, there can be good designs that require something like
that, but
in the
OFBiz framework, it exists because of a bad design, not a good
one.
Use of thread-local variables. This makes multi-threaded design
impossible. The J2EE specification and the Servlet API require
one
thread
per request (and most J2EE libraries depend on that behavior),
so the
current design makes sense from a J2EE perspective, but what if I
don't
want to run the framework in a J2EE container? Which leads to
the next
problem...
Dependence on J2EE designs/APIs/libraries. There are
developers in the
Java community (myself included) who are beginning to question if
J2EE is
really necessary to run web applications. The folks at
Atomikos are a
good
example. OFBiz does not use EJBs, so tying the framework to
J2EE does
not
make sense. It would be better if the framework was designed
to run
outside
a J2EE container, and then have container integration as an
option.
Configuration files are scattered everywhere. Anyone who has
deployed
OFBiz in a production environment will agree this is a
problem. Try
changing the HTTP/HTTPS and port settings - it is a nightmare.
Some
configuration settings are in nonsensical places.
An abysmal lack of unit testing. I don't have an exact figure
for code
coverage, but my gut feeling is coverage is less than 10%.
Basically,
we
all have our fingers crossed - hoping that the framework code
works as
expected. This was made painfully obvious a while back when I was
looking
at some entity caching code and thought to myself "this code
can't
work."
So I wrote some entity cache unit tests and confirmed that the
entity
cache
had serious problems. Think about that - years passed with no
entity
cache
unit tests and consequently we had no idea it wasn't working.
Fix Versus Rewrite
------------------
Jira issues could be created for these problems and teams of
developers
could work to fix them.
Or, we could create a branch and start over from scratch. This
time
around, there should be less push-back from people concerned
about
backwards compatibility. A rewrite offers the advantage of
reconsidering
everything - like API design, general problem solving, and new
features.
I created a Wiki page for a framework design:
https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision
but there hasn't been much interest in it. If the community
decides
to go
ahead with a rewrite, then please feel free to use the Wiki
pages as a
guide.
Sandglass Foundation
--------------------
Like David, I came to the conclusion that a framework rewrite
would be
easier outside the OFBiz community. So, I created my own library
called
Foundation:
http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf
(PDF)
and I only mention it here to stress how wonderful it can be
to start
with
a clean slate and design an API that is concise yet powerful.
(Please
do
not discuss Foundation here, contact me privately if you want
more
information.)
Some examples of what can be done with a rewrite:
A single configuration file
Use ANSI/ISO SQL SELECT statement strings instead of
constructing
complicated Java structures
Simultaneous asynchronous queries
Relational integrity across multiple datasources
Multi-table SELECT across multiple datasources
Automatic and transparent row version control
Automatic and transparent multi-language datasource support
Abstract entities (similar to SQL user types)
Service engine throttling (protects against server
over-utilization)
Simplified security (authorization) (
https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign)
Pure interface-based API - so developers are free to modify
framework
behavior by using decorators
Thorough unit tests
Benefits of a rewrite:
Reduced resource requirements (lower hosting fees)
Reduce application development time - due to a
simplified API
Easier framework code maintenance
Better reliability
Conclusion
----------
Like I said at the start, this is all I will say about the
subject.
I'm
done trying to convince everyone. I hope someone agrees with
me and
they
are able to build support for the idea.
--
Adrian Crum
Sandglass Software
www.sandglass-software.com
