Re: [U2] Guaranteed unique sequential keys
In message [EMAIL PROTECTED], Glen Batchelor [EMAIL PROTECTED] writes -Original Message- From: [EMAIL PROTECTED] [mailto:owner-u2- [EMAIL PROTECTED] On Behalf Of Anthony W. Youngman Sent: Saturday, April 26, 2008 9:14 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys In message [EMAIL PROTECTED], Glen B [EMAIL PROTECTED] writes It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Ummm My first reaction to this was linux has a guaranteed true random number generator - the only problem is that if it runs out of randomness it simply stops generating and anything asking for a number has to wait. Oh - and what do you mean by high resolution? I think that, if you have a true random number generator, the chances of having a collision approach 50% when you've used only 10% of the number space. (The classic question about this is how many people do you need in a room to have even odds that two of them will share a birthday?. I think the answer is about 20.) That is true and what I mean by high resolution is extremely frequent requests. The scale of that is dependant on the hardware and the application making the requests. When the entropy pool runs out, you get a function block until more hardware data is available to generate a new value. That is what I mean by flawed. If you have a tiny box with a small entropy pool, you could exhaust the entropy pool often which could lead to performance issues. Of course, the new pseudo generators don't block at all. That is what you are referring to below. The problem there is that entropy pool is reused to generate pseudo bits to regenerate new keys from, which increases the chance of repeated keys. Except that there, the whole point is you DON'T WANT the entropy pool used at all, because then you can't guarantee unique keys ... :-) But then, if you want unique, pseudo-random keys, I think there are generators that are guaranteed to return every possible number, only once, each cycle through the number space. None of this, however, solves the OP's original problem, I don't think... Cheers, Wol That's true. Random keys can not be sorted by order of creation. Nor can they guarantee uniqueness. Cheers, Wol -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
-Original Message- From: [EMAIL PROTECTED] [mailto:owner-u2- [EMAIL PROTECTED] On Behalf Of Anthony W. Youngman Sent: Saturday, April 26, 2008 9:14 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys In message [EMAIL PROTECTED], Glen B [EMAIL PROTECTED] writes It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Ummm My first reaction to this was linux has a guaranteed true random number generator - the only problem is that if it runs out of randomness it simply stops generating and anything asking for a number has to wait. Oh - and what do you mean by high resolution? I think that, if you have a true random number generator, the chances of having a collision approach 50% when you've used only 10% of the number space. (The classic question about this is how many people do you need in a room to have even odds that two of them will share a birthday?. I think the answer is about 20.) That is true and what I mean by high resolution is extremely frequent requests. The scale of that is dependant on the hardware and the application making the requests. When the entropy pool runs out, you get a function block until more hardware data is available to generate a new value. That is what I mean by flawed. If you have a tiny box with a small entropy pool, you could exhaust the entropy pool often which could lead to performance issues. Of course, the new pseudo generators don't block at all. That is what you are referring to below. The problem there is that entropy pool is reused to generate pseudo bits to regenerate new keys from, which increases the chance of repeated keys. But then, if you want unique, pseudo-random keys, I think there are generators that are guaranteed to return every possible number, only once, each cycle through the number space. None of this, however, solves the OP's original problem, I don't think... Cheers, Wol That's true. Random keys can not be sorted by order of creation. Glen --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? Sounds like you want to call a UUID generator. The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. Most operating systems have a UUID generator. It will generate a 128 bit hex number guaranteed unique (at least until the year 3400) and should be sequential; but you would possibly need to convert it to sort. Under AIX this is uuid_gen and can be a VOC verb or could be called in a CGI routine if you don't want to 'execute' . I'm currently approximating this using a concatenation of date and time with milliseconds but I'm worried about the possibility of two phantoms generating exactly the same key. Using a UUID (or GUID) has an advantage over something like 'time()' or even system(99) because it also guarantees that time shifts or hardware events, like crashes, leap seconds, daylight savings changes and ntp events don't interfere with unique key generation. Stuart ** This email message and any files transmitted with it are confidential and intended solely for the use of addressed recipient(s). If you have received this communication in error, please reply to this e-mail to notify the sender of its incorrect delivery and then delete it and your reply. It is your responsibility to check this email and any attachments for viruses and defects before opening or sending them on. Spotless collects information about you to provide and market our services. For information about use, disclosure and access, see our privacy policy at http://www.spotless.com.au Please consider our environment before printing this email. ** --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
In message [EMAIL PROTECTED], Glen B [EMAIL PROTECTED] writes It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Ummm My first reaction to this was linux has a guaranteed true random number generator - the only problem is that if it runs out of randomness it simply stops generating and anything asking for a number has to wait. Oh - and what do you mean by high resolution? I think that, if you have a true random number generator, the chances of having a collision approach 50% when you've used only 10% of the number space. (The classic question about this is how many people do you need in a room to have even odds that two of them will share a birthday?. I think the answer is about 20.) But then, if you want unique, pseudo-random keys, I think there are generators that are guaranteed to return every possible number, only once, each cycle through the number space. None of this, however, solves the OP's original problem, I don't think... Cheers, Wol -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Thank you all for all the responses. I was hoping that somewhere within the operating system's scheduling mechanism there is something like a timeslice serial number which could be read with some system function. No 2 processes could ever return the same timeslice serial number within a day for example. My unique id would then be formed as follows: snip BASEKEY = DATE():*:time slice serial number IF BASEKEY EQ PREVBASEKEY THEN SEQ.NUM += 1 ELSE PREVBASEKEY = BASEKEY SEQ.NUM = 1 END UNIQUE.KEY = BASEKEY:*:SEQ.NUM /snip This way there would be no need for extra IOs and no bottlenecks on a syncronisation key. --- On Fri, 25/4/08, Glen B [EMAIL PROTECTED] wrote: From: Glen B [EMAIL PROTECTED] Subject: RE: [U2] Guaranteed unique sequential keys To: u2-users@listserver.u2ug.org Date: Friday, 25 April, 2008, 5:30 AM You'll need a central key generator to manage high resolution sortable sequential keys. You can use whatever connection medium is feasible and let a single process/phantom generate the keys in numerical order. The problem with using a key generator like this is that you could easily produce a bottleneck. On the other hand, the benefit of doing it this way is that the generator can be a single phantom. It can keep track of the last used key in memory and can pregenerate keys for near-future or parallel usage. If the connection medium you choose allows for multiple requests at a time, then your management code must be able to manage and pregenerate keys for each thread concurrently. A wide solution could be a socket service that serves unique keys to clients. I use base-16 for a lot of sequential keys so that I have many unique iterations per key length. I always use them as direct pointers and I never sort them, though. Hex sortability from LIST/SORT/SELECT could be questionable. Glen -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ _ Yahoo! For Good. Give and get cool things for free, reduce waste and help our planet. Plus find hidden Yahoo! treasure --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
Apologies if this has already been said but I have not been following this thread too closely. D3 has a SYSTEM(19) function that does what you are wanting, returning the date/time with a suffix added if any user has already generated the same key. Unfortunately, UV implements SYSTEM(19) as something different. It should not be too difficult to do this using the GCI mechanism. The actual code to do this is simple (we recently implemented SYSTEM(19) in QM and it took only a few minutes to do). The code needs to maintain a shared memory record of the last timestamp value for which it was called and the suffix if one was added. Then, using a semaphore to ensure that it is single threaded, a call to the GCI function would create the new key, updating the shared memory. I admit to never having written a UV GCI function but the documention seems to suggest that it is easy. Martin Phillips Ladybridge Systems Ltd 17b Coldstream Lane, Hardingstone, Northampton, NN4 6DB +44-(0)1604-709200 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
Since we're talking OOB (out of box) why not try the user semaphore locking that is built into the system. Check the BASIC LOCK and UNLOCK , and LIST.LOCKS in tcl. Your process cant write until it can lock. It writes, then unlocks. and so on. j On 4/25/08, Marco Manyevere [EMAIL PROTECTED] wrote: Thank you all for all the responses. I was hoping that somewhere within the operating system's scheduling mechanism there is something like a timeslice serial number which could be read with some system function. No 2 processes could ever return the same timeslice serial number within a day for example. My unique id would then be formed as follows: snip BASEKEY = DATE():*:time slice serial number IF BASEKEY EQ PREVBASEKEY THEN SEQ.NUM += 1 ELSE PREVBASEKEY = BASEKEY SEQ.NUM = 1 END UNIQUE.KEY = BASEKEY:*:SEQ.NUM /snip This way there would be no need for extra IOs and no bottlenecks on a syncronisation key. --- On Fri, 25/4/08, Glen B [EMAIL PROTECTED] wrote: From: Glen B [EMAIL PROTECTED] Subject: RE: [U2] Guaranteed unique sequential keys To: u2-users@listserver.u2ug.org Date: Friday, 25 April, 2008, 5:30 AM You'll need a central key generator to manage high resolution sortable sequential keys. You can use whatever connection medium is feasible and let a single process/phantom generate the keys in numerical order. The problem with using a key generator like this is that you could easily produce a bottleneck. On the other hand, the benefit of doing it this way is that the generator can be a single phantom. It can keep track of the last used key in memory and can pregenerate keys for near-future or parallel usage. If the connection medium you choose allows for multiple requests at a time, then your management code must be able to manage and pregenerate keys for each thread concurrently. A wide solution could be a socket service that serves unique keys to clients. I use base-16 for a lot of sequential keys so that I have many unique iterations per key length. I always use them as direct pointers and I never sort them, though. Hex sortability from LIST/SORT/SELECT could be questionable. Glen -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ _ Yahoo! For Good. Give and get cool things for free, reduce waste and help our planet. Plus find hidden Yahoo! treasure --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ -- john --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
On Fri, Apr 25, 2008 at 10:19 PM, Marco Manyevere [EMAIL PROTECTED] wrote: Thank you all for all the responses. Marco, One last thought looking at your OP for a non-disk IO locking function, I don't see how you can avoid disk IO unless there is some kind of of in-memory locking process. There may be a kernel option you could use but I'm not a c programmer so can't say. In bash I have seen a lot of the classic test/touch but this can lead to race conditions and a symlink is a more robust solution. However, I'm not sure running a ln -s command at the OS level is going to be faster than a readu (and I can't see from your posts if you are on a *nix platform and even have that option). So... in the best traditions of using a pile-driver to crack a nut you could roll your own . Of the top of my head, a locking service would be a U2 subroutine listening on a particular network port that is started/stopped along with the DB. Your client routines would use the socket functions to connect. The nice thing is you get the duplicate connection handling for free. U2 socket listeners are not multi-threaded so the first client in gets the socket. As long as you hold it for 1 clock tick before closing and have the other clients on a time-out longer than 1 clock-tick, you should get your unique keys. I did mention this was over-kill right? Although the code itself is pretty simple and, in my experience on Unidata, the U2 sockets routines work very well, I would spend some time benchmarking to make sure this kind of effort is worthwhile. Assuming you have some spare server capacity, if the phantoms are using a lock file on a frequent basis, the file will likely be in cache anyway. HTH but probably not. Adrian I was hoping that somewhere within the operating system's scheduling mechanism there is something like a timeslice serial number which could be read with some system function. No 2 processes could ever return the same timeslice serial number within a day for example. My unique id would then be formed as follows: snip BASEKEY = DATE():*:time slice serial number IF BASEKEY EQ PREVBASEKEY THEN SEQ.NUM += 1 ELSE PREVBASEKEY = BASEKEY SEQ.NUM = 1 END UNIQUE.KEY = BASEKEY:*:SEQ.NUM /snip This way there would be no need for extra IOs and no bottlenecks on a syncronisation key. --- On Fri, 25/4/08, Glen B [EMAIL PROTECTED] wrote: From: Glen B [EMAIL PROTECTED] Subject: RE: [U2] Guaranteed unique sequential keys To: u2-users@listserver.u2ug.org Date: Friday, 25 April, 2008, 5:30 AM You'll need a central key generator to manage high resolution sortable sequential keys. You can use whatever connection medium is feasible and let a single process/phantom generate the keys in numerical order. The problem with using a key generator like this is that you could easily produce a bottleneck. On the other hand, the benefit of doing it this way is that the generator can be a single phantom. It can keep track of the last used key in memory and can pregenerate keys for near-future or parallel usage. If the connection medium you choose allows for multiple requests at a time, then your management code must be able to manage and pregenerate keys for each thread concurrently. A wide solution could be a socket service that serves unique keys to clients. I use base-16 for a lot of sequential keys so that I have many unique iterations per key length. I always use them as direct pointers and I never sort them, though. Hex sortability from LIST/SORT/SELECT could be questionable. Glen -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org
RE: [U2] Guaranteed unique sequential keys
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Adrian Merrall Sent: Friday, April 25, 2008 8:07 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys On Fri, Apr 25, 2008 at 10:19 PM, Marco Manyevere [EMAIL PROTECTED] wrote: So... in the best traditions of using a pile-driver to crack a nut you could roll your own . Of the top of my head, a locking service would be a U2 subroutine listening on a particular network port that is started/stopped along with the DB. Your client routines would use the socket functions to connect. The nice thing is you get the duplicate connection handling for free. U2 socket listeners are not multi-threaded so the first client in gets the socket. As long as you hold it for 1 clock tick before closing and have the other clients on a time-out longer than 1 clock-tick, you should get your unique keys. There are other options and they all have been tested. A master service phantom responds to all initial requests with a port# to connect back on. The client reconnects from the master over to the client phantom on that port. You can have X client phantoms handling the actual processes. That provides multi-client service, but it provides it at the cost of reconnections. I utilize a file-based spooling architecture in conjunction with inet/Winetd sockets in MVWWW to avoid this reconnection scenario, but that does have performance limitations. I did mention this was over-kill right? It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Although the code itself is pretty simple and, in my experience on Unidata, the U2 sockets routines work very well, I would spend some time benchmarking to make sure this kind of effort is worthwhile. Assuming you have some spare server capacity, if the phantoms are using a lock file on a frequent basis, the file will likely be in cache anyway. HTH but probably not. Adrian GlenB --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Has anyone considered submitting the transactions to said phantom, and have one process control the writes? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Glen B Sent: Friday, April 25, 2008 10:53 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Adrian Merrall Sent: Friday, April 25, 2008 8:07 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys On Fri, Apr 25, 2008 at 10:19 PM, Marco Manyevere [EMAIL PROTECTED] wrote: So... in the best traditions of using a pile-driver to crack a nut you could roll your own . Of the top of my head, a locking service would be a U2 subroutine listening on a particular network port that is started/stopped along with the DB. Your client routines would use the socket functions to connect. The nice thing is you get the duplicate connection handling for free. U2 socket listeners are not multi-threaded so the first client in gets the socket. As long as you hold it for 1 clock tick before closing and have the other clients on a time-out longer than 1 clock-tick, you should get your unique keys. There are other options and they all have been tested. A master service phantom responds to all initial requests with a port# to connect back on. The client reconnects from the master over to the client phantom on that port. You can have X client phantoms handling the actual processes. That provides multi-client service, but it provides it at the cost of reconnections. I utilize a file-based spooling architecture in conjunction with inet/Winetd sockets in MVWWW to avoid this reconnection scenario, but that does have performance limitations. I did mention this was over-kill right? It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Although the code itself is pretty simple and, in my experience on Unidata, the U2 sockets routines work very well, I would spend some time benchmarking to make sure this kind of effort is worthwhile. Assuming you have some spare server capacity, if the phantoms are using a lock file on a frequent basis, the file will likely be in cache anyway. HTH but probably not. Adrian GlenB --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
I believe we use a large pseudo random number as the key and put the time/date stamp in a log file. Your keys won't have the information in them about the sequence (although some may find that a better solution - let's not go there); however, a simply join/translate will get that for you. You can even do this across files hth Colin Alfke Calgary, Canada -Original Message- From: Glen B -Original Message- From: Adrian Merrall On Fri, Apr 25, 2008 at 10:19 PM, Marco Manyevere wrote: So... in the best traditions of using a pile-driver to crack a nut you could roll your own . Of the top of my head, a locking service would be a U2 subroutine listening on a particular network port that is started/stopped along with the DB. Your client routines would use the socket functions to connect. The nice thing is you get the duplicate connection handling for free. U2 socket listeners are not multi-threaded so the first client in gets the socket. As long as you hold it for 1 clock tick before closing and have the other clients on a time-out longer than 1 clock-tick, you should get your unique keys. There are other options and they all have been tested. A master service phantom responds to all initial requests with a port# to connect back on. The client reconnects from the master over to the client phantom on that port. You can have X client phantoms handling the actual processes. That provides multi-client service, but it provides it at the cost of reconnections. I utilize a file-based spooling architecture in conjunction with inet/Winetd sockets in MVWWW to avoid this reconnection scenario, but that does have performance limitations. I did mention this was over-kill right? It's not overkill if you need a multi-app/multi-process service that offers a single source for sequential keys. The other option is disk file locking. Any way you look at it, you have to go to one place for the key. Don't relay on O/S random numbers either. Even those are flawed at high resolution. Although the code itself is pretty simple and, in my experience on Unidata, the U2 sockets routines work very well, I would spend some time benchmarking to make sure this kind of effort is worthwhile. Assuming you have some spare server capacity, if the phantoms are using a lock file on a frequent basis, the file will likely be in cache anyway. HTH but probably not. Adrian GlenB --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
Marco, Have all the phantoms use a function or subroutine to get the next id based on the current system milliseconds. Use a semaphore lock to keep it singleton. Add a NAP statement so the next time it's run you're sure to get a different millisecond count -- note that many systems won't give 1ms granularity (IIRC Windows is about 6ms): FUNCTION GET.NEXT.ID * EQU MY$SEMAPHORE TO 55 EQU TD$TICKS LIT '(SYSTEM(99):((SYSTEM(12) * 1000) R%3))' * LOCK MY$SEMAPHORE MY.ID = TD$TICKS NAP 15 ;* must be big enough to get unique id's UNLOCK MY$SEMAPHORE RETURN (MY.ID) END rex Marco Manyevere wrote: What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
[U2] Guaranteed unique sequential keys
What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray * Learn and do * Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
Marco Manyevere wrote: What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. Assuming unix, add the process id to your key. So if you're currently using internal_date*internal*time, add *pid. There's a GCI interlude to get the process id, I *think* it's GETPID, but you'll have to look that up. If you're on windows, this won't help you much. -- Allen Egerton aegerton at pobox dot com --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray * Learn and do * Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? --- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring. Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. --- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
If you use @USERNO, instead of the pid, it will be OS independent. Tom Whitmore -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Allen Egerton Sent: Thursday, April 24, 2008 6:58 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys Marco Manyevere wrote: What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. Assuming unix, add the process id to your key. So if you're currently using internal_date*internal*time, add *pid. There's a GCI interlude to get the process id, I *think* it's GETPID, but you'll have to look that up. If you're on windows, this won't help you much. -- Allen Egerton aegerton at pobox dot com --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Setup a loop This assumes you've already set up ID with the date:time I=0 LOOP I += 1 WRITE.ID = ID : '*' : I READV JUST.CHECKING FROM FILE.NAME, WRITE.ID, 0 ELSE WRITE REC ON FILE.NAME, WRITE.ID EXIT END REPEAT This will always append *1 to the end of the ID unless it already exists. If it does the loop will engage and will try *2, *3, ... until it hits an unused ID. I've used this several times in the past with total success. However, the above code is untested so there could be a typo, etc., but I hope you get the idea! hth, Allen -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 02:55 To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. - -- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
You could use SETENV / GETENV but I wouldn't think this is any faster / safer than READU/WRITE. Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: 24 April 2008 13:47 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. - -- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe
RE: [U2] Guaranteed unique sequential keys
Wow - what kind of application needs THAT kind of accuracy?!?... old man voice Why I remember when two items writing in the same SECOND was a feat! /old man voice In this case, I don't know how you escape having to do a READVU on the item, and only writing on the ELSE clause, and changing the key if a THEN or LOCKED happens -- someone else showed using a READV, but you really have to do a READVU to ensure the items are not hitting at EXACTLY the same time. BASE.KEY = KEY SEQCNT = 1 DO.WRITE = 0 LOOP UNTIL DO.WRITE READVU TESTINGIT FROM FILENAME, KEY, 0 LOCKED * No need to wait, Drop To Key Update END THEN * Need a new key, Drop to Key Update END ELSE * Go! EXIT END * Once here, start adding sequence data... KEY = BASE.KEY:*:SEQCNT SEQCNT += 1 REPEAT WRITE REC ON FILENAME, KEY D3 has a 'System-Wide' Unique ID generator in their SYSTEM() selectiosn... U2 does not. Darn it. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 7:47 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
No, fast systems can generate records faster than SYSTEM(12) - so you need something to check the timestamp on SYSTEM(12) and handle accordingly. We've used a common block to do this. As far as two users writing the same time/date/mili - well you're going to need a port ID or a system to lock a next number array. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 8:47 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. - -- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ __ Sent from Yahoo! Mail. A Smarter Email http
RE: [U2] Guaranteed unique sequential keys
And on this part ... potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). I was only partially kidding about the 'two items a second' thing -- even if this were NOT possible today, in the lifetime of your software, you must plan on this being a reality with the next piece of hardware you get... I know of several programs we had to 'touch' once we started having 'overwrites' of data due to using a Date/TimeInSeconds stamp *way* back in the 80s. DW --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
But common wouldn't help here as it can't be shared between processes? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marc Harbeson Sent: 24 April 2008 14:42 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys No, fast systems can generate records faster than SYSTEM(12) - so you need something to check the timestamp on SYSTEM(12) and handle accordingly. We've used a common block to do this. As far as two users writing the same time/date/mili - well you're going to need a port ID or a system to lock a next number array. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 8:47 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR
Re: [U2] Guaranteed unique sequential keys
I have already found records that are only 28 milliseconds apart and the server isnt really a big one. Its not so much the clock accuracy thats important but the records must be processed in exactly the same order in which they were written to disk. Thanks for the confirmation. It seems there is no other way to do this. Regards, Marco. Wow - what kind of application needs THAT kind of accuracy?!?...B old man voice Why I remember when two items writing in the same SECOND was a feat! /old man voice In this case, I don't know how you escape having to do a READVU on the item, and only writing on the ELSE clause, and changing the key if a THEN or LOCKED happens -- someone else showed using a READV, but you really have to do a READVU to ensure the items are not hitting at EXACTLY the same time. BASE.KEY = KEY SEQCNT = 1 DO.WRITE = 0 LOOP UNTIL DO.WRITE B READVU TESTINGIT FROM FILENAME, KEY, 0 LOCKED B B * No need to wait, Drop To Key Update B END THEN B B * Need a new key, Drop to Key Update B END ELSE B B * Go! B B EXIT B END B * Once here, start adding sequence data... B KEY = BASE.KEY:*:SEQCNT B SEQCNT += 1 REPEAT WRITE REC ON FILENAME, KEY D3 has a 'System-Wide' Unique ID generator in their SYSTEM() selectiosn... U2 does not.B Darn it. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 7:47 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ ___ Yahoo! For Good. Give and get cool things for free, reduce waste and help our planet. Plus find hidden Yahoo! treasure http://green.yahoo.com/uk/earth-day/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
D3 has a 'System-Wide' Unique ID generator in their SYSTEM() selectiosn... U2 does not. Darn it. Ah, but nothing's free. To ensure uniqueness, there needs to be some sort of negotiation. Years ago I worked on a Sequoia system running Pick O/A. We were happy to see SYSTEM(19), which provided a unique ID. We used it in some processes that were used by many users and discovered that performance suffered dramatically. After talking with the system engineers, we discovered that there was a semaphore that negotiated the requests to ensure that exactly one user was served at a time, then assigned the unique ID to that user. The problem was that, if you have dozens of users requesting an ID simultaneously, you have a bottleneck. Because we didn't care what the ID contained, we did something similar to what others here have suggested - a combination of port, date and time the program started, and a counter unique to the process. Performance system-wide improved dramatically and immediately. If it's important to have an ID that guarantees that you can sort the file in the order the transactions occurred, the READU scenario is probably the best. You could probably work out something with linked C routines and/or a socket server, but there will still be a bottleneck if there are many simultaneous requests. Tim Snyder Consulting I/T Specialist U2 Lab Services Information Management, IBM Software Group --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Is the start time tied in with the phantom starting up? If so the process number is generally allocated in an incremental fashion, so date:milliseconds:process number would give you an approximation of what you're looking for? Some background information on what the phantoms are doing might help others come up with ideas. Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: 24 April 2008 15:44 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Not that I know all that much about it, but if outside of the box is needed. What would the possibility be of a callc routine to access a common memory location? Bruce M Neylon Health Care Management Group Edward Brown [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 04/24/2008 10:06 AM Please respond to u2-users@listserver.u2ug.org To u2-users@listserver.u2ug.org cc Subject RE: [U2] Guaranteed unique sequential keys But common wouldn't help here as it can't be shared between processes? --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? --- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring. Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. --- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
True. :-) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Edward Brown Sent: Thursday, April 24, 2008 10:07 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys But common wouldn't help here as it can't be shared between processes? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marc Harbeson Sent: 24 April 2008 14:42 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys No, fast systems can generate records faster than SYSTEM(12) - so you need something to check the timestamp on SYSTEM(12) and handle accordingly. We've used a common block to do this. As far as two users writing the same time/date/mili - well you're going to need a port ID or a system to lock a next number array. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 8:47 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B
RE: [U2] Guaranteed unique sequential keys
You need to re-read my solution, it will do exactly what you want. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 05:47 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Guaranteed unique sequential keys David, Allen, Edward, thanks for the suggestions but I want to be able to sort theB records in the exact physical order in which they occured.B Using the port or pid numbers could potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). Regards, Marco. - Original Message From: Edward Brown [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Thursday, 24 April, 2008 1:07:13 PM Subject: RE: [U2] Guaranteed unique sequential keys Of course if the port number is added to the end of the key then you can still sort by date/time. In the past I've done an R%5 oconv on both the (internal) date and time before concatenation - this gives you a 10 digit number regardless of the date and time used, so no need for separators... Edward -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Murray Sent: 24 April 2008 11:53 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Marco, I have used similar processes in the past, but also have appended port number to the beginning of the key. Separating the fields with '*' so the date and time can also be used as a time stamp rather than duplicate the same info in the record. Cheers, David Murray *B B B B B B Learn and do *B B B Excel and share http://u2blog.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. B B B __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ - -- Please remember to recycle wherever possible. Reduce, reuse, recycle, think do you need to print this e-mail? - -- This e-mail and any attachment(s), is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the addressee, dissemination, copying or use of this e-mail or any of its content is prohibited and may be unlawful. If you are not the intended recipient please inform the sender immediately and destroy the e-mail, any attachment(s) and any copies. All liability for viruses is excluded to the fullest extent permitted by law. It is your responsibility to scan or otherwise check this email and any attachment(s). Unless otherwise stated (i) views expressed in this message are those of the individual sender (ii) no contract may be construed by this e-mail. Emails may be monitored and you are taken to consent to this monitoring.B Civica Services Limited, Company No. 02374268; Civica UK Limited, Company No. 01628868 Both companies are registered in England and Wales and each has its registered office at 2 Burston Road, Putney, London, SW15 6AR. - -- --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Hi Marco - this is in fact the way I update files on very busy systems - we are getting say 30 hits per second and for logging with unique keys I have in the past used the date():system(12):endbit where endbit is incremented after doing a readvu to check if it exists or not. (as per your example), and system(12) is time in milleseconds If you do not need to have absolute order then the @userno is a useful extra to the key and will give guaranteed uniqueness. Symeon. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere Sent: 24 April 2008 10:55 To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
OOPS!! I forgot the RELEASE on the THEN!! Bad David!! BASE.KEY = KEY SEQCNT = 1 DO.WRITE = 0 LOOP UNTIL DO.WRITE READVU TESTINGIT FROM FILENAME, KEY, 0 LOCKED * No need to wait, Drop To Key Update END THEN RELEASE FILENAME, KEY ; VERY NEEDED! * Need a new key, Drop to Key Update END ELSE * Go! EXIT END * Once here, start adding sequence data... KEY = BASE.KEY:*:SEQCNT SEQCNT += 1 REPEAT WRITE REC ON FILENAME, KEY --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Use the suggestion about a read to verify the non-existence of the record and then loop around until the time() function changes to a unique id. From: [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Date: Thu, 24 Apr 2008 11:02:26 -0500 Consider using a lock-and-write thing... --Bill SUBROUTINE SUB.LOCK.AND.WRITE.R2 ( R.This, This.File, Record.ID ) prompt '' open This.File to F.This.File else gosub Error.Opening.File gosub Lock.And.Write goThe.End *--- -- *--- -- Lock.And.Write: Lock.Test = recordlocked (F.This.File, Record.ID) *crt 'This.File___ ' : F.This.File : ' ___ ' : ' [] ' : *input Ans begin case case Lock.Test = 0 ;recordlocku F.This.File, Record.ID write R.This on F.This.File, Record.ID release F.This.File, Record.ID case 1; gosub Error.Record.Locking end case return *--- --- Error.Opening.File: crt @(-1) crt @(-5) crt crt crt crt crt crt crt crt crt crt crt crt ' Big Problem...' : @(-6) crt crt ' _ ' crt ' \\ ' : This.File crt ' \ Error Opening File \' crt ' \\Contact HK.IT ' crt ' [X] ' crt '' : input Ans, 1 Ans = upcase(Ans) begin case case Ans = 'X' ; null case 1 ; go Error.Opening.File end case return to The.End *--- --- Error.Record.Locking: crt @(-1) crt @(-5) crt crt crt crt crt crt crt crt crt crt ' Big Problem...' : @(-6) crt crt ' _ ' crt ' \\ ' : This.File crt ' \ Error, Record Lock \' crt ' \\Contact HK.IT ' crt crt ' Open New Gull Session, Try UNLOCK.ME ' crt ' [X] ' crt ' ' : input Ans, 1 Ans = upcase(Ans) begin case case Ans = 'X' ; null case 1 ; go Error.Record.Locking end case return to The.End *--- --- The.End: RETURN END -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of David Wolverton Sent: Thursday, April 24, 2008 9:59 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys And on this part ... potentially give me the wrong sequence should two events occur within the same millisecond (unless it is ruled out that this is not possible). I was only partially kidding about the 'two items a second' thing -- even if this were NOT possible today, in the lifetime of your software, you must plan on this being a reality with the next piece of hardware you get... I know of several programs we had to 'touch' once we started having 'overwrites' of data due to using a Date/TimeInSeconds stamp *way* back in the 80s. DW --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ _ Spell a grand slam in this game where word skill meets World Series. Get in the game. http://club.live.com/word_slugger.aspx?icid=word_slugger_wlhm_admod_april08 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Humor But then you've limited your performance to one write a millisecond at best... Who could *stand* that kind of limitation?!? /Humor -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ron Hutchings Sent: Thursday, April 24, 2008 10:55 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys Use the suggestion about a read to verify the non-existence of the record and then loop around until the time() function changes to a unique id. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Lol
Now you've opened up the GOTO or LOOP REPEAT debate. :-)
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of DAVID WADEMAN
Sent: Thursday, April 24, 2008 10:58 AM
To: u2-users@listserver.u2ug.org
Cc: u2-users@listserver.u2ug.org
Subject: RE: [U2] Guaranteed unique sequential keys
I had this issue where I was overwriting files and found out that I need
to used @ rather than the (). I know there a difference in using
time/date where :
@DATE [EMAIL PROTECTED] is when the user/phantom was logged in
DATE() {TIME()} is the actually time and date
I got around this by using this to write to the file:
XKEY = @LOGNAME:*:DATE():*:TIME()
Then I can create VF like
X.DATE I FIELD(@ID,@VM,2) D2/ Date 20L S
X.TIME I FIELD(@ID,@VM,3) MTH Time 20L S
Also you can avoid 2 processes not writing to the same file by inserting
this right before you write to the file and/or to make sure also you
can:
X.DATA = Something
DO_AGAIN:
XKEY = @LOGNAME:*:DATE():*:TIME()
READ XFILE.REC FROM F.XFILE, X.KEY THEN
SLEEP(1)
GOTO DO_AGAIN
END ELSE
WRITE X.DATA TO F.XFILE, X.KEY
END
This is Unix and UniData (not sure with the rest)
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Marco Manyevere
Sent: Thursday, April 24, 2008 5:55 AM
To: u2-users@listserver.u2ug.org
Subject: [U2] Guaranteed unique sequential keys
What is the most reliable way to generate unique sequential keys without
having to resort to a record on disk updated through readu/write? The
keys
don't have to be contiguous but only be sortable in the order in which
they
were generated by several phantom processes running concurrently. I'm
currently approximating this using a concatenation of date and time with
millisecondsB but I'm worried about the possibility of two phantoms
generating
exactly the same key.
B
Although no collision has been detected so far, I
have added an extra check where after generating the key I first test if
a
record with that key exists. If so IB increment and append aB serial
number
and repeat the test until aB unique key is found. ItB seems to be
working well
but I still think there is a better way to do this.
B
Thanks for any help.
B
Marco.
__
Sent
from Yahoo! Mail.
A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
Only needed if you care about eventually running out of locks. LOL It will run a few thousand transactions before it blows up the system. :-D -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Wolverton Sent: Thursday, April 24, 2008 11:51 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Guaranteed unique sequential keys OOPS!! I forgot the RELEASE on the THEN!! Bad David!! BASE.KEY = KEY SEQCNT = 1 DO.WRITE = 0 LOOP UNTIL DO.WRITE READVU TESTINGIT FROM FILENAME, KEY, 0 LOCKED * No need to wait, Drop To Key Update END THEN RELEASE FILENAME, KEY ; VERY NEEDED! * Need a new key, Drop to Key Update END ELSE * Go! EXIT END * Once here, start adding sequence data... KEY = BASE.KEY:*:SEQCNT SEQCNT += 1 REPEAT WRITE REC ON FILENAME, KEY --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
I forgot to mention in my previous post is that a WRITE of a new record into a UniVerse/SQL table is regarded as an INSERT. So the new key generation will occur. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
Here is a UniVerse/SQL answer. Create the primary key with a default value of NEXT AVAILABLE. CREATE TABLE MyTable ( MyKey INTEGER NOT NULL PRIMARY KEY DEFAULT NEXT AVAILABLE, Column1 VARCHAR, Column2 VARCHAR, Column3 VARCHAR NOT NULL MULTIVALUED, Column4 VARCHAR MULTIVALUED, ASSOCIATION MyASSOC(Column3 KEY, Column4) ); Whenever you insert a row without supplying a key value, the next available key value will be used. This uses the same mechanism as REVISE, namely an item called NEXT.AVAILABLE in the file dictionary. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
That's very good, but DICT NEXT.AVAILABLE sure is ugly. It means you can't make a dictionary read-only. Dictionaries should be under source control. And a data file can have multiple dictionaries. A better answer would be to embed the next-id in the file header. Then give a utility to edit it similar to how SET.INDEX can edit the file header's path to it's index dir. If one were to set up an SQL table like that, what would you consider a good UV/Basic coding practice for writing to it? Explicit, low-level readu increment of the next id record? -Original Message- From: Ray Wurlod Here is a UniVerse/SQL answer. Create the primary key with a default value of NEXT AVAILABLE. CREATE TABLE MyTable ( MyKey INTEGER NOT NULL PRIMARY KEY DEFAULT NEXT AVAILABLE, Column1 VARCHAR, Column2 VARCHAR, Column3 VARCHAR NOT NULL MULTIVALUED, Column4 VARCHAR MULTIVALUED, ASSOCIATION MyASSOC(Column3 KEY, Column4) ); Whenever you insert a row without supplying a key value, the next available key value will be used. This uses the same mechanism as REVISE, namely an item called NEXT.AVAILABLE in the file dictionary. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
I'm surprised to see most people suggesting a date time stamp for the record key. Isn't a sequential number the ideal? Which hashes better? I like Mr. Stevenson's suggestion of storing a next-available key in the file header. It could be configurable by file to be an integer, or some of the other suggestions, like a date*time*millisecond stamp, or date*time*sequential integer. Does the U2 User's Group make suggestions to IBM? -- Louie in Seattle --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Guaranteed unique sequential keys
How to write? I'd probably go for INSERT statements via the pre-defined @HSTMT statement handle and the BCI function SQLExecDirect. That's because the WRITE statement requires a key value. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Guaranteed unique sequential keys
You'll need a central key generator to manage high resolution sortable sequential keys. You can use whatever connection medium is feasible and let a single process/phantom generate the keys in numerical order. The problem with using a key generator like this is that you could easily produce a bottleneck. On the other hand, the benefit of doing it this way is that the generator can be a single phantom. It can keep track of the last used key in memory and can pregenerate keys for near-future or parallel usage. If the connection medium you choose allows for multiple requests at a time, then your management code must be able to manage and pregenerate keys for each thread concurrently. A wide solution could be a socket service that serves unique keys to clients. I use base-16 for a lot of sequential keys so that I have many unique iterations per key length. I always use them as direct pointers and I never sort them, though. Hex sortability from LIST/SORT/SELECT could be questionable. Glen -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Marco Manyevere Sent: Thursday, April 24, 2008 5:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] Guaranteed unique sequential keys What is the most reliable way to generate unique sequential keys without having to resort to a record on disk updated through readu/write? The keys don't have to be contiguous but only be sortable in the order in which they were generated by several phantom processes running concurrently. I'm currently approximating this using a concatenation of date and time with millisecondsB but I'm worried about the possibility of two phantoms generating exactly the same key. B Although no collision has been detected so far, I have added an extra check where after generating the key I first test if a record with that key exists. If so IB increment and append aB serial number and repeat the test until aB unique key is found. ItB seems to be working well but I still think there is a better way to do this. B Thanks for any help. B Marco. __ Sent from Yahoo! Mail. A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
