About
the ratio of writing data, I usually tend to look more at the read/write ratio.
It allows me to write better caching gadgets and to plan network-trips more
carefully. I say plan, because on many ocassions, obtaining proper metrics is
usually beyond my reach(I have many, many problems simulating realistically the
actual load of some of these systems, as some have to process millions of not so
simple transactions within a few minutes, and I just don't have access to
hundreds of PCs in my dev environment).
The
actual read/write ratio for some of the examples you provide is in the
25-50 range. I have many times checked for books/DVDs in Amazon, yet I have
embarked in the actual process of purchasing only a few times. Logic executed
most of the time is merely presentation logic; hence, the RDBMS manager only
needs to produce reads, and only needs to manage little TX contexts(the level of
isolation needed is minimal). Hence this logic is best outside the RDBMS, and I
think we'd agree on that.
What
happens when I purchase a book? Here's when I _must_ agree with you, but only
for the most simple systems(and I think that the model used in Amazon is one of
these, for reasons I'll state below). A fetch/calculate/store procedure, if
simple and without the need to lock important resources(table/page locks) is the
way to go. But at the most this approach yields faster execution times, not
greater scalability. In these cases, moving the business logic out of the RDBMS
yields longer processing times, but these are usually LAN latencies, which may
not produce a noticeable distinction for the end user, and in turn, allow the
same RDBMS(both hardware and software), to process more concurrent querys, hence
scaling more.
If the
assumption that the splitting is too expensive (in response time), then by all
means use SPs.
I
would also like to add that we're talking about really big systems here. If you
can determine load and it is easily reproducible in the dev environment,
then you're probably in the best position to decide wether SPs are enough for
you or not. I only tend to walk away from them when the load(or it's
consecuences) is unknown at the time of building the system, which is most of
the time for me, and that's why I get the big bucks ;-).
Now,
what if we add data-warehousing? You know, Amazon not selling you(and charging
for) items that it cannot deliver as they're not available in
stock?
Usually, these systems require to implement either a
higher isolation level or a transactional saga(both are very scary to me as I've
had to do some such things in the past, and you shouldn't start without a lot of
aspirins and sending a lot of flowers to your soulmate ;-) ). BTW, Amazon
doesn't do this, in fact they have a ranking of availability for each product
instead of handling stock more directly; some logistics systems cannot afford
this, particulary in tracking orders from contractors in the automobile and
construction industries; if cars lights do not reach the assembly line on
time, the whole assembly line must stop; in construction the problem worsens, as
it is always project oriented. Many companies are now selling mixed concrete
right on time, and making lots of profit from it(the dynamics are pretty much
similar to those of IT projects: the project gets budgeted, then it costs more
money and lasts longer, etc.).
Whenever these impediments are present, my first
instinct is to marshall the business logic out of the RDBMS as much as
possible.
Reading data and sending it thru the wire is really
very inexpensive if you have set-up your LAN properly; regrettably this doesn't
happen as often as I'd like to(damn NATs). Also, this is particulary slow in
COM+, as you're always using ADO to fetch and as the underlying protocol is
usually DCOM itself, first going in and out of MS DTC, which altogether is a
performance hog(yes, even worse than J2EE comparables, mostly because of the
location transparency magic COM and COM+ have to perform).
So,
the bottom line for me is to try to avoid SPs if there's a choice and it's not
the best for the project. Many many systems don't need to scale at all, as
they're deployed on a controlled environment. Many just cannot afford to handle
BL outside the RDBMS because at the time of writing no app server technology was
present, not to mention I'm not keen to trash out systems that work and are
stable after years of continued service.
I'd
like to add that it's always a pleasure to discuss these subjects with so many
bright, experienced and polite engineers. I owe lots to this forum and the
brilliant gentleman that so frecuently visit it.
My
2c,
Juan Pablo Lorandi
Chief Software
Architect
Code Foundry Ltd.
Barberstown, Straffan, Co. Kildare, Ireland.
Tel: +353-1-6012050 Fax: +353-1-6012051
Mobile: +353-86-2157900
www.codefoundry.com
Disclaimer:
Opinions expressed are entirely
personal and bear no relevance to opinions held by my employer.
Code Foundry Ltd.'s opinion is that I
should get back to work.
-----Original Message-----
From: A mailing list for Enterprise JavaBeans development [mailto:[EMAIL PROTECTED]] On Behalf Of Craig McMurtry
Sent: Wednesday, July 17, 2002 7:06 PM
To: [EMAIL PROTECTED]
Subject: Re: Why Ejb?
Your response is certainly not flame-bait, Juan, for it is as careful and well-informed as your posts usually are.
It seems that the nub of the issue is the "weight" of the business logic. May I propose that the ratio of the work involved in executing the business logic versus the work involved in reading in writing the data varies between systems, and that in many systems that ratio is high, whereas in many others, it is relatively low. If that ratio is relatively low, then would you agree that if the DBMS is approaching the point of choking in cases where it is executing the business logic in the form of stored procedures, then it would be very close to choking even if it was merely reading and writing data at the behest of component-based middleware that was doing the "thinking" ? Then is it possible that how appropriate it would be to increase the amount of data shipping (moving data back and forth to the component-based middleware so that it can apply the processing) would depend on how sophisticated the ! business logic is?
The inclination that I have developed based on my own experience is that typical business systems, as opposed to scientific applications, do not tend to manipulate data in very sophisticated ways (which is why COBOL was the perfect tool for so many years). So, if one is the paradigmatic modern e-commerce applications, Amazon.COM, eBay, etc., except where one is going off to validate credit cards, for example, I cannot see what one would gain by data shipping.
--Craig McMurtry
Juan Pablo Lorandi <[EMAIL PROTECTED]>
Sent by: A mailing list for Enterprise JavaBeans development <[EMAIL PROTECTED]>07/17/2002 11:12 AM
Please respond to Juan Pablo Lorandi
To: [EMAIL PROTECTED]
cc:
Subject: Re: Why Ejb?
I appreciate your comments, but I've had first hand experience on this. On large deployment setups, the DB eventually chokes. The DB hardware is more expensive than the hardware in which we deploy our app server. Scaling a database properly is a painful experience(many _only_ replicate). Devising the architecture to make out the most of network trips is of course, essential.
I have read Transactional COM+, and I have found it insightful in regards to the COM+ platform. I had been working with MTS/COM+ for two years before moving into EJBs. Since there is no equivalent of Entity Beans (not even BMP) in MS TP, books on the subject of Microsoft TP tend to overrate the scalability and performance of SPs (almost always running against SQL Server). Then eventually the RDBMS server chokes under heavy load. Then the Microsoft(or Oracle or Sybase or...) representatives come around, and try to sell you their RDBMS scalability solution, which usually involves one or more of the following: more CPUs per machine, a really expensive RAID, and a Gigabit network (because they don't really care much about network round-trips), and software upgrades starting at the OS and working it way up. And it usually doesn't work that well.
Also, and I see this quite often, it's not always about RDBMSs; there are many components that merely fetch from the DB and calculate, but never store. You may intelligently cache data either on the app-server or the presentation layer. There are different persistence options than RDBMS (and they don't perform as well, yet they're as expensive to scale). Many business components operate on a different paradigm that doesn't involve fetching data (such as Message Queues).
Of course, there's many many reasons to use SPs. And of course it also scales. But it is more _difficult and expensive_; and usually this knowledge is acquired _way too late_. From my experience, I try to avoid them. You like them. We both have a choice in J2EE.
My 2c (hope it's not flame-bait),
Juan Pablo Lorandi
Chief Software Architect
Code Foundry Ltd.
[EMAIL PROTECTED]
Barberstown, Straffan, Co. Kildare, Ireland.
Tel: +353-1-6012050 Fax: +353-1-6012051
Mobile: +353-86-2157900
www.codefoundry.com
Disclaimer:
Opinions expressed are entirely personal and bear no relevance to opinions held by my employer.
Code Foundry Ltd.'s opinion is that I should get back to work.
-----Original Message-----
From: A mailing list for Enterprise JavaBeans development [mailto:[EMAIL PROTECTED]] On Behalf Of Craig McMurtry
Sent: Wednesday, July 17, 2002 3:32 PM
To: [EMAIL PROTECTED]
Subject: Re: Why Ejb?
The notion that having business logic in component-based middleware rather than in stored procedures provides for better scalability is falacious. The middleware components have to retrieve data to which they apply their logic and they have to update the database with data to which they have applied their logic. Each call those components make to the database creates traffic over the finite resources of database and network connections. In practice, those resources prove to be far more of a botteneck than the processing capabilities of the database server. Hence, better scalability is achieved by simply having the component-based middleware apply security, transactional boundaries and connection management to calls from the presentation tier, and pass incoming data to stored procedures which implement the business logic, update the database and retrieve the data the presentation tier needs subsequently.
I've tried the business logic in component-based middleware solution and was surprised by how poorly it performed and scaled. Read Transactional COM+, (which is as pertinent to the EJB platform as it is to COM+), and since tried the business logic in stored procedures solution and have been very impressed with how it performs and scales.
Juan Pablo Lorandi <[EMAIL PROTECTED]>
Sent by: A mailing list for Enterprise JavaBeans development <[EMAIL PROTECTED]>07/17/2002 09:52 AM
Please respond to Juan Pablo Lorandi
To: [EMAIL PROTECTED]
cc:
Subject: Re: Why Ejb?
Nicholas, tough I agree that in most cases this mixed combination works
wonders, it presents three problems:
1) Business logic is fragmented. Hence, when changing DBs you're in for
some trouble. Granted, if you're part of the IT staff on a company, that
won't be much of a problem, but if you're building re-usable
components(with the purpose of selling them afterwards in a component
marketplace such as Flashline) then you're in for a lot of trouble.
2) The strain on the database is higher. That compromises your ability
to scale both up and out, one of the goals in every TP engine built so
far. Tuxedo[CORBA] (built by BEA), MTS[COM+] (Microsoft), and the miriad
of J2EE servers allow for the application assembler to redefine the
physical architecture of a system that uses this component, leveraging
ease of scalability, not to mention cost.
3) Not all of the persistence managers are RDBMS. Many RDBMSs of choice
do not allow SPs and/or some do not compile SPs.
On the other hand, if you're just using SPs to perform CREATE, INSERT,
DELETE, UPDATE statements with no business logic on them, _most_
appservers won't take a big hit in performance since they will use
PreparedStatement. Most important RBDMS will compile the statement, and
also store it temporarily as long as the connection is alive. Since
connections are pooled, the differences in speed of execution between
using SPs and using PreparedStatements is negligible.
In the near future, you could also expect that the appservers compile
many of the auto-generated SQL statements into SPs as well.
Bottom line is, while in your particular situation usage of SPs means no
hassle but actual speed gains, then go ahead (and knock yourself out ;-)
) but from a service/component provider POV, like us here at
CodeFoundry, the choice must be done carefully for each case.
My 2c,
Juan Pablo Lorandi
Chief Software Architect
Code Foundry Ltd.
[EMAIL PROTECTED]
Barberstown, Straffan, Co. Kildare, Ireland.
Tel: +353-1-6012050 Fax: +353-1-6012051
Mobile: +353-86-2157900
www.codefoundry.com
> -----Original Message-----
> From: A mailing list for Enterprise JavaBeans development
> [mailto:[EMAIL PROTECTED]] On Behalf Of Nicholas
> Sent: Wednesday, July 17, 2002 11:42 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Why Ejb?
>
>
> 100% portability is an illusion. Consider CMP and one
> must conclude that porting from one app server to
> another will come with a series of head aches
> regenerating all the proprietary extended descriptors.
>
> In this event, it could be argued that BMP offers a
> higher degree of portability, if not ease of use,
> since you have a higher degree of control over the
> actual SQL that is issued. Portability is in the eye
> of the porter.
>
> My point about SPs is that the Java code itself is
> better abstracted from the underlying database
> structure when it simply calls a CallableStatement and
> passes parameters. Admitedly, the downside is that the
> SPs themselves have to be regeneratred upon switching
> databases, but if you have anything like a serious
> application, and you are switching databases, that will the
> least of you worries.
>
> Anyways, I believe the trend is that medium to large
> sized companies (if not all companies) tend to have
> one standardized database (generally) for J2EE back
> ending, but they switch J2EE engines like they change
> their underwear.J2EE licenses are quite cheap compared
> to DB licenses.
>
>
> //Nicholas
>
>
> --- John Harby <[EMAIL PROTECTED]> wrote:
> > But they aren't portable across database vendors so
> > if this
> > is an issue you should avoid stored procedures.
> >
> > I'm also not sure exactly what "ISOMORPHIC" means in
> > this
> > context. Actually the functionality of stored
> > procedures
> > would comprise a superset of the functionality of
> > plain SQL.
> > For example in Oracle take a look at DBMS_PIPE.
> >
> >
> > >From: Nicholas <[EMAIL PROTECTED]>
> > >Reply-To: [EMAIL PROTECTED]
> > >To: [EMAIL PROTECTED]
> > >Subject: Re: Why Ejb?
> > >Date: Tue, 16 Jul 2002 14:02:44 -0700
> > >
> > >No No No !
> > >
> > >Does anyone on this list actually use stored
> > >procedures ?
> > >
> > >When you invoke a stored procedure, it participates
> > in
> > >the current transaction identically to a dynamic or
> > >embedded SQL statement. (Yes, you can mess it up by
> > >issuing a commit or a rollback in the SP, but that
> > >would be silly.) So, when you start a JTA managed
> transaction, calls
> > >to stored procedures are still within the context of that
> > >transaction. They are
> > not
> > >subject to different rules.
> > >
> > >Moreover, the same goes for security (in most
> > >databases). SPs do not have any magic security
> > >override, except that users may have permission to
> > >issues SP requests, but not have access to the
> > >underlying tables/views/other SPs. This does not
> > >affect the security model in J2EE at all.
> > >
> > >My new mantra: SQL ISOMORPHIC TO SP
> > >
> > >//Nicholas
> > >--- Ashwani Kalra <[EMAIL PROTECTED]> wrote:
> > > > You will have to use wrapper/facade in SLSB/SFSB
> > > > from which you can take
> > > > advanatage of the transactions and security. You
> > can
> > > > manage the transaction
> > > > in SPs also and propagate errors to the session
> > bean
> > > > methods.
> > > > Scalability issue will then shift to session
> > beans.
> > > >
> > > > ----- Original Message -----
> > > > From: "Vikram Naik" <[EMAIL PROTECTED]>
> > > > To: <[EMAIL PROTECTED]>
> > > > Sent: Tuesday, July 16, 2002 3:46 PM
> > > > Subject: Re: Why Ejb?
> > > >
> > > >
> > > > > Hi,
> > > > > Thanks for the quick reply...
> > > > > Apart from getting generated SQL's
> > what
> > > > about the transactions
> > > > and
> > > > > scalability ????
> > > > >
> > > > > Vikram Naik
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: A mailing list for Enterprise JavaBeans
> > > > development
> > > > > [mailto:[EMAIL PROTECTED]]On Behalf Of
> > > > Ashwani Kalra
> > > > > Sent: Tuesday, July 16, 2002 3:19 PM
> > > > > To: [EMAIL PROTECTED]
> > > > > Subject: Re: Why Ejb?
> > > > >
> > > > > Two reasons :
> > > > > 1.EJB(cmp) makes code independent of the
> > Database.
> > > > > 2. You dont have to code sql queries for
> > > > inserting/updating etc.
> > > > >
> > > > > If you already have Sps then you can
> > completely
> > > > avoid Entity EJBs and use
> > > > > DAO to call SPs.
> > > > >
> > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > Cheers
> > > > > Ashwani Kalra
> > > > > http://www.geocities.com/ashwani_kalra/
> > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > From: "Vikram Naik" <[EMAIL PROTECTED]>
> > > > > To: <[EMAIL PROTECTED]>
> > > > > Sent: Tuesday, July 16, 2002 3:22 PM
> > > > > Subject: Why Ejb?
> > > > >
> > > > >
> > > > > > Hello All,
> > > > > >
> > > > > > Why should we opt of EjBs when stored
> > procedures
> > > > can give us better
> > > > > > performance?
> > > > > >
> > > > > > Your opinions will be highly appreciated.
> > > > > >
> > > > > > Thanks & Regards,
> > > > > > Vikram Naik
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> >=============================================================
> ==========
> >====
> > > > > > To unsubscribe, send email to
> > > > [EMAIL PROTECTED] and include in the
> > > > > body
> > > > > > of the message "signoff EJB-INTEREST". For
> > > > general help, send email to
> > > > > > [EMAIL PROTECTED] and include in the
> > body of
> > > > the message "help".
> > > > > >
> > > > >
> > > > >
> > > >
> >
> >=============================================================
> ==========
> >====
> > > > > To unsubscribe, send email to
> > > > [EMAIL PROTECTED] and include in the
> > > > body
> > > > > of the message "signoff EJB-INTEREST". For
> > > > general help, send email to
> > > > > [EMAIL PROTECTED] and include in the body
> > of
> > > > the message "help".
> > > > >
> > > > >
> > > >
> >
> >=============================================================
> ==========
> >====
> > > > > To unsubscribe, send email to
> > > > [EMAIL PROTECTED] and include in the
> > > > body
> > > > > of the message "signoff EJB-INTEREST". For
> > > > general help, send email to
> > > > > [EMAIL PROTECTED] and include in the body
> > of
> > > > the message "help".
> > > > >
> > > >
> > > >
> >
> >=============================================================
> ==========
> >====
> > > > To unsubscribe, send email to
> > [EMAIL PROTECTED]
> > > > and include in the body
> > > > of the message "signoff EJB-INTEREST". For
> > general
> > > > help, send email to
> > > > [EMAIL PROTECTED] and include in the body of
> > the
> > > > message "help".
> > > >
> > >
> > >
> > >=====
> > >Nicholas Whitehead
> > >Home: (973) 377 9335
> > >Cell: (201) 615 2716
> > >Work: (212) 622 5639
> > >[EMAIL PROTECTED]
> > >
> > >__________________________________________________
> > >Do You Yahoo!?
> > >Yahoo! Autos - Get free new car price quotes http://autos.yahoo.com
> > >
> >
> >=============================================================
> ==========
> >====
> > >To unsubscribe, send email to [EMAIL PROTECTED]
> > and include in the body
> > >of the message "signoff EJB-INTEREST". For general
> > help, send email to
> > >[EMAIL PROTECTED] and include in the body of
> > the message "help".
> >
> >
> >
> >
> >
> _________________________________________________________________
> > Send and receive Hotmail on your mobile device:
http://mobile.msn.com
>
>
=== message truncated ===
=====
Nicholas Whitehead
Home: (973) 377 9335
Cell: (201) 615 2716
Work: (212) 622 5639
[EMAIL PROTECTED]
__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com
========================================================================
===
To unsubscribe, send email to [EMAIL PROTECTED] and include in the
body
of the message "signoff EJB-INTEREST". For general help, send email to
[EMAIL PROTECTED] and include in the body of the message "help".
===========================================================================
To unsubscribe, send email to [EMAIL PROTECTED] and include in the body
of the message "signoff EJB-INTEREST". For general help, send email to
[EMAIL PROTECTED] and include in the body of the message "help".
_________________________________________________________________
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
_________________________________________________________________
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
