There is in fact an option to traverse files without taking locks if you can 'know' that there are no updates. It has to be supported by the target file type (J4, etc).
http://jbase.markmail.org/search/?q=fastscan Jim > -----Original Message----- > From: jbase@googlegroups.com [mailto:jbase@googlegroups.com] On Behalf > Of Pawel (privately) > Sent: Monday, August 15, 2011 1:47 PM > To: jbase@googlegroups.com > Subject: RE: RE: jbase problems show-items-locks > > Hi, > > > > > With Temenos we have found there is a problem at the Jbase level > > > > as > > > we > > > > have patched the F.READU program and there were no NULL records > > > > being delivered here. > Could you please elaborate kindly? Do you know what was the reason? > What was fixed? Was it problem strictly related to HP-UX? > > Jim suggested that these could be group/binary locks and I would agree > with him. > I think that binary locks are listed like that under SHOW-ITEM-LOCKS > (at least on jBASE 4.1). > > You have also said that: > > > > > > > > We are not using the jbase locking but the UNIX locking > > > > > > > > (set at > > > > > > > > 24,000) on a HP box as the jDLS was causing problems. The > > > unix > > > > > > > > locking has been running fine for over a month now. > Our LIVE system is P5, AIX, jBASE 4.1 and we are also sucessfully > running UNIX locking. However we have observed performance and > scalability problems on P7. There are 2 facts about P7, AIX 6.1 and > jBASE 4.1: > a) jRLA (pthreads) locks are bit faster than UNIX locks on > P7/AIX6.1/jBASE 4.1 - proven with jBASE test programs > b) group locks in 4.1 are always taken as UNIX locks disregarding > wheter you run jRLA or not. We have got information that this changed > (was improved) in jBASE 5+/TAFC. Can jBASE 5+/TAFC use pthreads for > group/binary locks? > > In our case high 'lock collision' ratio on group locks is killing > performance. JOB.LIST.x files suffer from multiple, concurrent 'SELECTs > SAMPLE' (~ full scan) and multiple, concurrent READUs/WRITEs. There > should be an option to jBASE added - to traverse file with no locking > on groups ('dirty traversal'). It seems that this new option should be > used on JOB.LIST.x files and would improve performance a lot. Each > process (agent) could do 'dirty READNEXT traversal', try to lock record > id and move forward if locked / not existing. > > Can somebody answer me below question? Scenario: > 1. Assume you have file with 3 groups - 8192 bytes each group (2 blocks > in jBASE 4.1) 2. Assume there are 35 records in total - 10, 10, 15 3. > Assume there are 15 concurrent processes doing: SELECT, READNEXT (up to > 1000 iterations), READU followed by READNEXT (when record locked) and > SLEEP 1 or doing processing (if lock sucessfully aquired) 4. Assume > response time is no limitation (say file put to RAM disk, striped SSD > with multiple physical disks, OS/RAID caching enabled, etc.) 5. Assume > transaction is simple / lightweight and is quickly commited. > 6. Asumme all processes start exactly in the same moment. > > Can (theoretically) other processes be starving? (due to greedy process > #1 which start, will constantly lock group #1 preventing other > processes to move forward - it will be obtaing id, processing > transaction and quickly commiting) > > Of course in realistic world no process will be starving, but may be > significantly delayed? Am I right? > > By the way: we observe very higher System:User ratio (eg. System=90, > User=10). Simplifying System means - 'I am not doing a (user) job for > you my friend'. > > Kind regards > Pawel > > > Dnia 15-08-2011 o godz. 20:27 Jim Idle napisał(a): > > Are you sure you are not just seeing group/binary locks? > > > > Jim > > > > > -----Original Message----- > > > From: jbase@googlegroups.com [mailto:jbase@googlegroups.com] On > > > Behalf Of VK > > > Sent: Monday, August 15, 2011 7:35 AM > > > To: jBASE > > > Subject: Re: jbase problems show-items-locks > > > > > > Hi, > > > > > > > have patched the F.READU program and there were no NULL records > > > > being delivered here. > > > > > > What about READU? A lot of routines in core T24 still use it. > > > > > > VK > > > > > > On Aug 14, 3:46 am, adrian <ar_atkin...@yahoo.com> wrote: > > > > With Temenos we have found there is a problem at the Jbase level > > > > as > > > we > > > > have patched the F.READU program and there were no NULL records > > > > being delivered here. > > > > The null problem is seen more when there are a lots of locks > > > happening > > > > for a application, and AA and AZ have this so it is easier to > spot. > > > > > > > > The F.OFS.REQUEST.DETAIL on some of out loads takes this file to > > > > 15gb and bigger. > > > > > > > > Regards and Thanks > > > > AA > > > > > > > > On Aug 11, 6:58 am, VK <kzm...@yahoo.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > F_JOB_LIST looks ok... 2 files with no IDs are F_LANGUAGE for > > > > > one session and FBNK_CUSTOMER_ROLE for another (though it looks > > > > > not being DM-related - maybe the problem is here? both use > > > > > CUSTOMER > > > file)... > > > > > > > > > Might be that some core code issued F.READU without supplying > ID > > > (or > > > > > supplying a COMMON variable that contains an empty string). Put > > > that > > > > > query to Temenos helpdesk. > > > > > > > > > What's the task is BTW? Update of which fields in CUSTOMER? > > > > > Might > > > be > > > > > an easier solution to that... > > > > > > > > > Another note - non-secure is really not secure :(( Consider > > > > > using secure mode (and pay something in performance) or J4. > > > > > > > > > Yet another note... is DM something called "migration tool"? > > > > > > > > > And - your OFS.REQUEST.DETAIL is distributed? Just curious why. > > > > > Accumulating over 2Gb a day? > > > > > > > > > VK > > > > > > > > > On Aug 10, 7:45 pm, adrian <ar_atkin...@yahoo.com> wrote: > > > > > > > > > > We are using 10 threads at the moments > > > > > > > > > > we are using JR type out of the box from Temenos. (non- > secure) > > > > > > > > > > we are using DM.SERVICE.DATA.FILE which we are using on a RAM > > > drive. > > > > > > > > > > as you can see below in the show-item-locks you can see blank > > > keys. > > > > > > > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER_CHARGE > > > > > > 5383596 > > > > > > 0x57a7c41c,W --- > > > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER_ROLE > > > > > > 0x02300000,W --- > > > > > > 31 9358 ../bnk.data/st/ FBNK_RELATION_CUSTOMER > > > > > > 5383596 > > > > > > 0x57a7c41c,W --- > > > > > > 31 9358 ../bnk.data/st/ FBNK_RELATION_CUSTOMER > > > > > > 11561555 > > > > > > 0x5e90b3b0,W --- > > > > > > 31 9358 ../bnk.data/st/ FBNK_CUSTOMER#NAU > > > > > > 5383596 > > > > > > 0x57a7c41c,W > > > > > > > > > > 33 9367 ../bnk.data/of/ > > > > > > F_OFS_REQUEST_DETAIL_06 > > > > > > MBDM112090115838227.05 > > > > > > 0x31fc7d36,W --- > > > > > > 33 9367 ../bnk.data/eb/ F_DM_SERVICE_DATA_FILE > > > > > > CCS.DM.CUSTOMER.MEMA-20110810037483638476 > > > > > > 0x59b0e064,W --- > > > > > > 33 9367 ../bnk.data/eb/ > > > > > > F_JOB_LIST_3 > > > > > > 1139 > > > > > > 0x6883ccaf,W --- > > > > > > 34 9371 ../bnk.data/eb/ F_LANGUAGE > > > 0x0000001c,R > > > > > > --- > > > > > > 34 9371 ../bnk.data/st/ FBNK_CUSTOMER#NAU > > > > > > 4872036 > > > > > > 0x04b9c9ed,W --- > > > > > > 34 9371 ../bnk.data/of/ > > > > > > F_OFS_REQUEST_DETAIL_08 > > > > > > MBDM112090521738237.01 > > > > > > 0x5c4e0da1,W --- > > > > > > 34 9371 ../bnk.data/eb/ F_DM_SERVICE_DATA_FILE > > > > > > CCS.DM.CUSTOMER.MEMA-20110810037483537290 > > > > > > 0x0f29a5e6,W --- > > > > > > 34 9371 ../bnk.data/eb/ > > > > > > F_JOB_LIST_3 > > > > > > 250 > > > > > > 0x7b754110,W --- > > > > > > > > > > AA > > > > > > > > > > yep , onlive it will be a one off but now its run once every > > > > > > two weeks. > > > > > > > > > > On Aug 10, 7:16 am, VK <kzm...@yahoo.com> wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > Clarify several things please: > > > > > > > > > > > > many threads say to update customer > > > > > > > > > > > How many? Which method is used? Service OFS.MESSAGE.SERVICE > > > > > > > I > > > presume? > > > > > > > > > > > > RECORDKEY is blank in the locking tables > > > > > > > > > > > Which tables exactly? F.LOCKING or F.JOB.LIST.n? > > > > > > > > > > > > JR type files database > > > > > > > > > > > Secure updates or non-secure? > > > > > > > > > > > Last but not least - updating 1m customers looks like a > one- > > > time > > > > > > > task, not a regular one, isn't it? > > > > > > > > > > > VK > > > > > > > > > > > On Aug 6, 5:39 am, adrian <ar_atkin...@yahoo.com> wrote: > > > > > > > > > > > > When running many threads say to update customer > > > > > > > > (1,000.000) some of the tsa's show SLEEP in the mw42 > view. > > > > > > > > > > > > Looking at the show-items-locks we are see many times > > > > > > > > lately the RECORDKEY is blank in the locking tables, when > > > > > > > > this is happening the agents are only processing 120 per > > > > > > > > minute while if we see no blank keys then we can process > > > > > > > > 20,000 + a > > > minute. > > > > > > > > > > > > Have we a problem with our Jbase version or is this > normal > > > > > > > > to see records in the lock table with blanks , I must say > > > > > > > > i have never seen this over the pass years. > > > > > > > > > > > > We are not using the jbase locking but the UNIX locking > > > > > > > > (set at > > > > > > > > 24,000) on a HP box as the jDLS was causing problems. The > > > unix > > > > > > > > locking has been running fine for over a month now. > > > > > > > > > > > > Could somebody knowing Jbase explain many thanks. > > > > > > > > > > > > Sorry we are using JR type files database and R10 with > > > > > > > > many > > > patches. > > > > > > > > > > > > System Information > > > > > > > > ================== > > > > > > > > > > > > System : HP-UX vic-samt B.11.31.U > > > > > > > > ia64 UNIX User : aatkinso (uid 187, > euid > > > > > > > > 187) > > > Tty > > > > > > > > name : /dev/pts/0 > > > Time > > > > > > > > : Fri Aug 5 20:31:32 2011 > > > > > > > > > > > > Environment > > > > > > > > =========== > > > > > > > > > > > > JBCPORTNO : Not Set > > > TAFC_HOME > > > > > > > > : '/app/tafc/r10SP8/R10' > > > > > > > > JBCGLOBALDIR : '/app/tafc/r10SP8/R10' > > > > > > > > WARNING: JBCDATADIR is not set, Default > > > '/app/tafc/r10SP8/R10/ > > > > > > > > jbase_data' > > > > > > > > WARNING: JBCDATADIR is subdirectory of JBCGLOBALDIR > > > HOME > > > > > > > > : '/app/t24/dm5/bnk.run' > > > > > > > > JEDIFILEPATH : '/app/t24/dm5/bnk.run' > > > > > > > > JEDIFILENAME_MD : '/app/t24/dm5/bnk.run/VOC' > > > > > > > > JEDIFILENAME_SYSTEM : > '/app/t24/dm5/bnk.run/SYSTEM' > > > > > > > > RELEASE Information : Major 10.0 , Minor 0.8 , > > > Patch > > > > > > > > (Change > > > > > > > > 92702) > > > > > > > > Spooler dir (JBCSPOOLERDIR) : '/usr/spool/jspooler' > > > > > > > > JBCEMULATE : 'prime' > > > > > > > > Object path (JBCOBJECTLIST) : > > > > > > > > '/app/t24/dm5/bnk.run/ccslib:/app/t24/ > > > > > > > > > > > dm5/bnk.run/GR0800004lib:/app/t24/dm5/bnk.run/t24lib:/app/t24/ > > > > > > > > dm5/ > > > > > > > > > > > bnk.run/GR0800005lib:/app/t24/dm5/bnk.run/GR0800006lib:/app/t2 > > > > > > > > 4/dm5/ > > > > > > > > > > > bnk.run/radlib:/app/t24/dm5/bnk.run/cardlib:/app/t24/dm5/bnk.r > > > > > > > > un/ > > > > > > > > > > > > usrdlib:/app/t24/dm5/bnk.run/GR0800012lib:/app/t24/dm5/bnk.run/atmlib' > > > > > > > > jBASE Compiler Run-time : > > > > > > > > '/app/tafc/r10SP8/R10/config/ system.properties' > > > > > > > > Program dir (JBCDEV_BIN) : > '/app/t24/dm5/bnk.run/ccsbin' > > > > > > > > Subroutine dir (JBCDEV_LIB) : > '/app/t24/dm5/bnk.run/ccslib' > > > > > > > > Max open files : 8192 jsh aatkinso ~ -->- > > > > > > > > Hide quoted text - > > > > -- > Please read the posting guidelines at: > http://groups.google.com/group/jBASE/web/Posting%20Guidelines > > IMPORTANT: Type T24: at the start of the subject line for questions > specific to Globus/T24 > > To post, send email to jBASE@googlegroups.com To unsubscribe, send > email to jbase-unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/jBASE?hl=en -- Please read the posting guidelines at: http://groups.google.com/group/jBASE/web/Posting%20Guidelines IMPORTANT: Type T24: at the start of the subject line for questions specific to Globus/T24 To post, send email to jBASE@googlegroups.com To unsubscribe, send email to jbase-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/jBASE?hl=en
