Initially data volume will be small - probably about 20 customers per day and about 6 records per customer but it is likely to grow very quickly up to some unknown amount. We'll have to see I guess. The records per customer are likely to be the high volume stuff.
There is transaction logging on the remote system that I can use and am relying on for data integrity but I did not know about the SQL stuff until this week. Not sure if it would work until I can find time to play with it and see how it might suit us. Some of the other constraints were: no access allowed to the remote unirpc port, the remote UV is below 10.3 and different from ours, neither site has a UV/Net licence in prod. So I think I'm pretty much stuck with file queuing and multiple phantoms for now at least. I also was able to get a successful remote file process going on our test system while waiting for confirmation of UV/Net accessibility. It is very nice and might have been a nice solution too. Then I did some experimenting with the performance requirements on the file triggers and found there to be too much of a lag for the triggers to fire directly to the other end. This would've caused unacceptable problems in testing not to mention going live so a file queuing process had to be devised. In all the file queuing works quite well and can be coded for rock solid reliability so it has a lot of pluses. I can also fine tune the phantom loop times and socket timeouts on the fly at both ends once the data starts flowing and we get some idea of what needs changing. Fingers, toes, arms, legs (and eyes!) all currently crossed. Cheers Peter Peter Cheney Ultracs Developer t 07 3017 8837 | f 07 3002 8400 e peter.che...@firstmac.com.au w firstmac.com.au -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Hona, David Sent: Friday, 7 March 2014 13:41 To: U2 Users List Subject: Re: [U2] Multi-threaded phantom processing Hi Peter Sounds like an interested piece of work, just a couple of questions for you: - what sort of data volumes and update frequencies are involved? - do you need to know if the data has been successfully committed to the remote UV database? - did you consider SQL BCI or even UV/Net (read the different UV versions is an issue and so this may be a constraint) Gregor mentioned SQL BCI as an alternative interface - I've used that before to exchange data between systems and it works well. If you had the same UV version on both hosts - then UV/Net is simple effect way of transmitting data between UV systems with only licenses required on the remote system. With BCI you could utilise the Stored Procedure facility via an ExecDirect() call to push the data to the remote system in your UV file trigger. With UV/Net you could "push" the items to be copied the by your UV file trigger - thus the updates being written directly onto the remote host. BCI is free/included in UV and of course will utilise a license on the remote host. You need to write your own custom code. Probably a bit easier to maintain than any TCP/IP sockets based interface. Definitely a lot easier to troubleshoot ;-) ! YMMV. UV/Net is licensed on the number user connections on the remote host. Hence the above alternative as designed to eliminate the file-based queue - which in some instances by be desirable to maintain - depending on business, technical or operational needs/objections. Cheers, David -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Peter Cheney Sent: Wednesday, 5 March 2014 12:18 PM To: U2 Users List (u2-users@listserver.u2ug.org) Subject: [U2] Multi-threaded phantom processing Hi Everyone, A project that I’m currently working on requires transmission of transactional and ancillary data from one UV site to site to another. I have the communications all set up and it is working very well via the UV socket functions with a client at one end and a server at the other. Both the client and the server both run as a UV phantom processes written in UV basic and the data being transferred (also encrypted) is more or less real time with various UV file triggers on the client end creating queue items for transmission based on the data content / transaction type. This set up is a single threaded affair and I am wondering how it will cope and what options I’ll have as the volume of data increases. Multi-threading my server process to dish out some of the load might be one possibility and I’ve seen examples in the past where the last digit of a numeric key (i.e. 0-9) is used to determine which one of 10 phantoms to call thus providing an effective distribution of the work. But even then, reading/writing processing data to a file for each child process seems to me to be the only way to achieve parent <-> child phantom communications at either end. Is there another way perhaps? It may be that I will need 2 sets of phantoms, 1 for the transactions and one for the ancillary data. Although we run UV on AIX the only other requirement is that I must be able to do it within UV Basic. Does anyone out there on the list know of a better way or any alternatives/advice so I can get a better appreciation of what else might be possible please? Many thanks in advance, Peter Peter Cheney Ultracs Developer t 07 3017 8837 | f 07 3002 8400 e peter.che...@firstmac.com.au w firstmac.com.au ------------------------------------------------------------------------------- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600) ------------------------------------------------------------------------------- _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ************** IMPORTANT MESSAGE ***************************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. We can be contacted through our web site: commbank.com.au. If you no longer wish to receive commercial electronic messages from us, please reply to this e-mail by typing Unsubscribe in the subject line. ************************************************************** _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ------------------------------------------------------------------------------- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600) ------------------------------------------------------------------------------- _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
