Hi,
> > > With Temenos we have found there is a problem at the Jbase level as
> > we
> > > have patched the F.READU program and there were no NULL records being
> > > delivered here.
Could you please elaborate kindly? Do you know what was the reason? What was
fixed? Was it problem strictly related to HP-UX?
Jim suggested that these could be group/binary locks and I would agree with him.
I think that binary locks are listed like that under SHOW-ITEM-LOCKS (at least
on jBASE 4.1).
You have also said that:
> > > > > > > We are not using the jbase locking but the UNIX locking (set
> > > > > > > at
> > > > > > > 24,000) on a HP box as the jDLS was causing problems. The
> > unix
> > > > > > > locking has been running fine for over a month now.
Our LIVE system is P5, AIX, jBASE 4.1 and we are also sucessfully running UNIX
locking. However we have observed performance and scalability problems on P7.
There are 2 facts about P7, AIX 6.1 and jBASE 4.1:
a) jRLA (pthreads) locks are bit faster than UNIX locks on P7/AIX6.1/jBASE 4.1
- proven with jBASE test programs
b) group locks in 4.1 are always taken as UNIX locks disregarding wheter you
run jRLA or not. We have got information that this changed (was improved) in
jBASE 5+/TAFC. Can jBASE 5+/TAFC use pthreads for group/binary locks?
In our case high 'lock collision' ratio on group locks is killing performance.
JOB.LIST.x files suffer from multiple, concurrent 'SELECTs SAMPLE' (~ full
scan) and multiple, concurrent READUs/WRITEs. There should be an option to
jBASE added - to traverse file with no locking on groups ('dirty traversal').
It seems that this new option should be used on JOB.LIST.x files and would
improve performance a lot. Each process (agent) could do 'dirty READNEXT
traversal', try to lock record id and move
forward if locked / not existing.
Can somebody answer me below question? Scenario:
1. Assume you have file with 3 groups - 8192 bytes each group (2 blocks in
jBASE 4.1)
2. Assume there are 35 records in total - 10, 10, 15
3. Assume there are 15 concurrent processes doing: SELECT, READNEXT (up to 1000
iterations), READU followed by READNEXT (when record locked) and SLEEP 1 or
doing processing (if lock sucessfully aquired)
4. Assume response time is no limitation (say file put to RAM disk, striped SSD
with multiple physical disks, OS/RAID caching enabled, etc.)
5. Assume transaction is simple / lightweight and is quickly commited.
6. Asumme all processes start exactly in the same moment.
Can (theoretically) other processes be starving? (due to greedy process #1
which start, will constantly lock group #1 preventing other processes to move
forward - it will be obtaing id, processing transaction and quickly commiting)
Of course in realistic world no process will be starving, but may be
significantly delayed? Am I right?
By the way: we observe very higher System:User ratio (eg. System=90, User=10).
Simplifying System means - 'I am not doing a (user) job for you my friend'.
Kind regards
Pawel
Dnia 15-08-2011 o godz. 20:27 Jim Idle napisał(a):
> Are you sure you are not just seeing group/binary locks?
>
> Jim
>
> > -----Original Message-----
> > From: jbase@googlegroups.com [mailto:jbase@googlegroups.com] On Behalf
> > Of VK
> > Sent: Monday, August 15, 2011 7:35 AM
> > To: jBASE
> > Subject: Re: jbase problems show-items-locks
> >
> > Hi,
> >
> > > have patched the F.READU program and there were no NULL records being
> > > delivered here.
> >
> > What about READU? A lot of routines in core T24 still use it.
> >
> > VK
> >
> > On Aug 14, 3:46 am, adrian <ar_atkin...@yahoo.com> wrote:
> > > With Temenos we have found there is a problem at the Jbase level as
> > we
> > > have patched the F.READU program and there were no NULL records being
> > > delivered here.
> > > The null problem is seen more when there are a lots of locks
> > happening
> > > for a application, and AA and AZ have this so it is easier to spot.
> > >
> > > The F.OFS.REQUEST.DETAIL on some of out loads takes this file to 15gb
> > > and bigger.
> > >
> > > Regards and Thanks
> > > AA
> > >
> > > On Aug 11, 6:58 am, VK <kzm...@yahoo.com> wrote:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Hi,
> > >
> > > > F_JOB_LIST looks ok... 2 files with no IDs are F_LANGUAGE for one
> > > > session and FBNK_CUSTOMER_ROLE for another (though it looks not
> > > > being DM-related - maybe the problem is here? both use CUSTOMER
> > file)...
> > >
> > > > Might be that some core code issued F.READU without supplying ID
> > (or
> > > > supplying a COMMON variable that contains an empty string). Put
> > that
> > > > query to Temenos helpdesk.
> > >
> > > > What's the task is BTW? Update of which fields in CUSTOMER? Might
> > be
> > > > an easier solution to that...
> > >
> > > > Another note - non-secure is really not secure :(( Consider using
> > > > secure mode (and pay something in performance) or J4.
> > >
> > > > Yet another note... is DM something called "migration tool"?
> > >
> > > > And - your OFS.REQUEST.DETAIL is distributed? Just curious why.
> > > > Accumulating over 2Gb a day?
> > >
> > > > VK
> > >
> > > > On Aug 10, 7:45 pm, adrian <ar_atkin...@yahoo.com> wrote:
> > >
> > > > > We are using 10 threads at the moments
> > >
> > > > > we are using JR type out of the box from Temenos. (non-secure)
> > >
> > > > > we are using DM.SERVICE.DATA.FILE which we are using on a RAM
> > drive.
> > >
> > > > > as you can see below in the show-item-locks you can see blank
> > keys.
> > >
> > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER_CHARGE
> > > > > 5383596
> > > > > 0x57a7c41c,W ---
> > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER_ROLE
> > > > > 0x02300000,W ---
> > > > > 31 9358 ../bnk.data/st/ FBNK_RELATION_CUSTOMER
> > > > > 5383596
> > > > > 0x57a7c41c,W ---
> > > > > 31 9358 ../bnk.data/st/ FBNK_RELATION_CUSTOMER
> > > > > 11561555
> > > > > 0x5e90b3b0,W ---
> > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER#NAU
> > > > > 5383596
> > > > > 0x57a7c41c,W
> > >
> > > > > 33 9367 ../bnk.data/of/
> > > > > F_OFS_REQUEST_DETAIL_06
> > > > > MBDM112090115838227.05
> > > > > 0x31fc7d36,W ---
> > > > > 33 9367 ../bnk.data/eb/ F_DM_SERVICE_DATA_FILE
> > > > > CCS.DM.CUSTOMER.MEMA-20110810037483638476
> > > > > 0x59b0e064,W ---
> > > > > 33 9367 ../bnk.data/eb/
> > > > > F_JOB_LIST_3
> > > > > 1139
> > > > > 0x6883ccaf,W ---
> > > > > 34 9371 ../bnk.data/eb/ F_LANGUAGE
> > 0x0000001c,R
> > > > > ---
> > > > > 34 9371 ../bnk.data/st/ FBNK_CUSTOMER#NAU
> > > > > 4872036
> > > > > 0x04b9c9ed,W ---
> > > > > 34 9371 ../bnk.data/of/
> > > > > F_OFS_REQUEST_DETAIL_08
> > > > > MBDM112090521738237.01
> > > > > 0x5c4e0da1,W ---
> > > > > 34 9371 ../bnk.data/eb/ F_DM_SERVICE_DATA_FILE
> > > > > CCS.DM.CUSTOMER.MEMA-20110810037483537290
> > > > > 0x0f29a5e6,W ---
> > > > > 34 9371 ../bnk.data/eb/
> > > > > F_JOB_LIST_3
> > > > > 250
> > > > > 0x7b754110,W ---
> > >
> > > > > AA
> > >
> > > > > yep , onlive it will be a one off but now its run once every two
> > > > > weeks.
> > >
> > > > > On Aug 10, 7:16 am, VK <kzm...@yahoo.com> wrote:
> > >
> > > > > > Hi,
> > >
> > > > > > Clarify several things please:
> > >
> > > > > > > many threads say to update customer
> > >
> > > > > > How many? Which method is used? Service OFS.MESSAGE.SERVICE I
> > presume?
> > >
> > > > > > > RECORDKEY is blank in the locking tables
> > >
> > > > > > Which tables exactly? F.LOCKING or F.JOB.LIST.n?
> > >
> > > > > > > JR type files database
> > >
> > > > > > Secure updates or non-secure?
> > >
> > > > > > Last but not least - updating 1m customers looks like a one-
> > time
> > > > > > task, not a regular one, isn't it?
> > >
> > > > > > VK
> > >
> > > > > > On Aug 6, 5:39 am, adrian <ar_atkin...@yahoo.com> wrote:
> > >
> > > > > > > When running many threads say to update customer (1,000.000)
> > > > > > > some of the tsa's show SLEEP in the mw42 view.
> > >
> > > > > > > Looking at the show-items-locks we are see many times lately
> > > > > > > the RECORDKEY is blank in the locking tables, when this is
> > > > > > > happening the agents are only processing 120 per minute while
> > > > > > > if we see no blank keys then we can process 20,000 + a
> > minute.
> > >
> > > > > > > Have we a problem with our Jbase version or is this normal to
> > > > > > > see records in the lock table with blanks , I must say i have
> > > > > > > never seen this over the pass years.
> > >
> > > > > > > We are not using the jbase locking but the UNIX locking (set
> > > > > > > at
> > > > > > > 24,000) on a HP box as the jDLS was causing problems. The
> > unix
> > > > > > > locking has been running fine for over a month now.
> > >
> > > > > > > Could somebody knowing Jbase explain many thanks.
> > >
> > > > > > > Sorry we are using JR type files database and R10 with many
> > patches.
> > >
> > > > > > > System Information
> > > > > > > ==================
> > >
> > > > > > > System : HP-UX vic-samt B.11.31.U ia64
> > > > > > > UNIX User : aatkinso (uid 187, euid 187)
> > Tty
> > > > > > > name : /dev/pts/0
> > Time
> > > > > > > : Fri Aug 5 20:31:32 2011
> > >
> > > > > > > Environment
> > > > > > > ===========
> > >
> > > > > > > JBCPORTNO : Not Set
> > TAFC_HOME
> > > > > > > : '/app/tafc/r10SP8/R10'
> > > > > > > JBCGLOBALDIR : '/app/tafc/r10SP8/R10'
> > > > > > > WARNING: JBCDATADIR is not set, Default
> > '/app/tafc/r10SP8/R10/
> > > > > > > jbase_data'
> > > > > > > WARNING: JBCDATADIR is subdirectory of JBCGLOBALDIR
> > HOME
> > > > > > > : '/app/t24/dm5/bnk.run'
> > > > > > > JEDIFILEPATH : '/app/t24/dm5/bnk.run'
> > > > > > > JEDIFILENAME_MD : '/app/t24/dm5/bnk.run/VOC'
> > > > > > > JEDIFILENAME_SYSTEM : '/app/t24/dm5/bnk.run/SYSTEM'
> > > > > > > RELEASE Information : Major 10.0 , Minor 0.8 ,
> > Patch
> > > > > > > (Change
> > > > > > > 92702)
> > > > > > > Spooler dir (JBCSPOOLERDIR) : '/usr/spool/jspooler'
> > > > > > > JBCEMULATE : 'prime'
> > > > > > > Object path (JBCOBJECTLIST) :
> > > > > > > '/app/t24/dm5/bnk.run/ccslib:/app/t24/
> > > > > > >
> > dm5/bnk.run/GR0800004lib:/app/t24/dm5/bnk.run/t24lib:/app/t24/
> > > > > > > dm5/
> > > > > > >
> > bnk.run/GR0800005lib:/app/t24/dm5/bnk.run/GR0800006lib:/app/t2
> > > > > > > 4/dm5/
> > > > > > >
> > bnk.run/radlib:/app/t24/dm5/bnk.run/cardlib:/app/t24/dm5/bnk.r
> > > > > > > un/
> > > > > > >
> > usrdlib:/app/t24/dm5/bnk.run/GR0800012lib:/app/t24/dm5/bnk.run/atmlib'
> > > > > > > jBASE Compiler Run-time : '/app/tafc/r10SP8/R10/config/
> > > > > > > system.properties'
> > > > > > > Program dir (JBCDEV_BIN) : '/app/t24/dm5/bnk.run/ccsbin'
> > > > > > > Subroutine dir (JBCDEV_LIB) : '/app/t24/dm5/bnk.run/ccslib'
> > > > > > > Max open files : 8192 jsh aatkinso ~ -->- Hide
> > > > > > > quoted text -
--
Please read the posting guidelines at:
http://groups.google.com/group/jBASE/web/Posting%20Guidelines
IMPORTANT: Type T24: at the start of the subject line for questions specific to
Globus/T24
To post, send email to jBASE@googlegroups.com
To unsubscribe, send email to jbase-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/jBASE?hl=en
