Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
PCAnywhere works really well on windows servers, good security Robert - Original Message - From: Peter Cheney peter.che...@firstmac.com.au To: U2 Users List u2-users@listserver.u2ug.org Sent: Sunday, January 20, 2013 4:47 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Distance should not be an obstacle. You may not be able to physically access the console but console access should still be possible either by getting someone else to physically eyeball it or a launch remote console session yourself. How about using something like a Cyclades terminal server to access one or more host consoles remotely via ssh? Even if a Cyclades is not an option then what about a network connected management facility such as an ALOM/ILOM or similar? Again via ssh, login and launch the console process from there. As long as the chassis had power then you should have LOM access if the WAN is available. Voila! Serial console heaven from the other side of the world even. Sure talking someone on the other end of a phone through a system recovery is not the easiest of tasks and it may not be as quick as direct connect in the same room or even an ssh session over your LAN but it sure would beat an acoustic coupled 300 baud half duplex modem that we used to use at AWA to dial in and fix GFE's etc. So how do you access your system console? Cheers Peter -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Saturday, 19 January 2013 3:33 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users --- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600) --- ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
To use that tool, he would have to be on the same subnet/network as the server (i.e. he would have to vpn into their system, or remote desktop/vnc/etc into a system they have over there that had the admin tool installed). Having said that I think that's a great idea, the customer could setup a windows system with the admin tool installed and logmein/gotomypc/etc (which would NOT require vpn access) so that he could access the system remotely and run uniadmin Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 11:52 AM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Hi Doug, We use UniVerse instead of UniData, but I understand the tools are the same. You do not need to be on the same subnet or network, but you do need access to the subnet/network for the RPC port. I have successfully connected to external servers without a problem, you may need to use an IP address rather than the host name but that depends on how you have things configured. I would strongly recommend that you use a VPN or a gateway to access your server, this keeps your sever and network secure. I think using VPN is easier to use with the admin tools than setting up a gateway. Tom Whitmore RATEX Business Solutions -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of doug chanco Sent: Monday, January 21, 2013 12:02 PM To: 'U2 Users List' Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly To use that tool, he would have to be on the same subnet/network as the server (i.e. he would have to vpn into their system, or remote desktop/vnc/etc into a system they have over there that had the admin tool installed). Having said that I think that's a great idea, the customer could setup a windows system with the admin tool installed and logmein/gotomypc/etc (which would NOT require vpn access) so that he could access the system remotely and run uniadmin Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 11:52 AM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Your right that if the server have an external IP then certainly you could access it but I doubt many companies actually do that (put an external IP on an internal sever but it can certainly be done). Then there is always firewall port forwarding that could allow an internal server to be accessed externally, but in all these cases you need a good system/network admin to ensure its setup and secure. Anyway we are getting off topic, so if we want to continue this discussion we should probably move it to its own thread. But good points Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tom Whitmore Sent: Monday, January 21, 2013 12:13 PM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Hi Doug, We use UniVerse instead of UniData, but I understand the tools are the same. You do not need to be on the same subnet or network, but you do need access to the subnet/network for the RPC port. I have successfully connected to external servers without a problem, you may need to use an IP address rather than the host name but that depends on how you have things configured. I would strongly recommend that you use a VPN or a gateway to access your server, this keeps your sever and network secure. I think using VPN is easier to use with the admin tools than setting up a gateway. Tom Whitmore RATEX Business Solutions -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of doug chanco Sent: Monday, January 21, 2013 12:02 PM To: 'U2 Users List' Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly To use that tool, he would have to be on the same subnet/network as the server (i.e. he would have to vpn into their system, or remote desktop/vnc/etc into a system they have over there that had the admin tool installed). Having said that I think that's a great idea, the customer could setup a windows system with the admin tool installed and logmein/gotomypc/etc (which would NOT require vpn access) so that he could access the system remotely and run uniadmin Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 11:52 AM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Heres a copy of my udtconf, were running about 100 users. Its unix and it's been very stable for 18 some odd years. I have a calculation worksheet somewhere for setting these variables if your interested. Robert | | | NUSERS.:130SHM_LPINENTS:20 UDT_LANGGRP.:255/192/129 | | SHM_GNTBLS.:100SHM_LMINENTS:48 N_FILESYS...:200 | | SHM_GNPAGES:32 SHM_LCINENTS:100 NULL_FLAG...:0 | | SHM_GPAGESZ:512SHM_LPAGESZ.:8 NVLMARK.: | | SHM_FREEPCT:25 AVG_TUPLE_LEN...:4 N_GLM_GLOBAL_BUCKET:101 | | SHM_NFREES.:1 EXPBLKSIZE..:32 N_GLM_SELF_BUCKET..:23 | | MIN_MEMORY_TEMP.:64 GLM_MEM_SEGSZ:4194304 | | SHMMAX..:1073741824COMPACTOR_POLICY:1 | | SHMMIN..:1 VARMEM_PCT..:50 MAX_NEXT_HOLD_DIGITS:4 | | SHM_ATT_ADD.:0 CHECK_HOLD_EXIST:0 | | SHM_LBA.:4096 | | LOCK FIFO: 1 System PV: 3 | | | | NFILES..:1019 WRITE_TO_CONSOLE:0 MAX_FLENGTH:1073741824 | | SHM_MIN_NATT:4 FCNTL_ON:0 SBCS_SHM_SIZE..:4194304 | | NSEM_PSET...:8 TOGGLE_NAP_TIME.:21 MAX_OBJ_SIZE...:307200 | | GRP_FREE_BLK:5 JRNL_MAX_PROCS..:1 EFS_LCKTIME:0 | | SHM_FIL_CNT.:2048 JRNL_MAX_FILES..:400 MAX_CAPT_LEVEL.:2 | | SPLIT_LOAD:60 MAX_OPEN_FILE...:2000MAX_RETN_LEVEL.:2 | | MERGE_LOAD:40 MAX_OPEN_SEQF...:150 MGLM_BUCKET_SIZE.:50 | | KEYDATA_SPLIT_LOAD:95 MAX_OPEN_OSF:100 SETINDEX_BUFFER_KEYS.:0 | | KEYDATA_MERGE_LOAD:40 MAX_DSFILES.:1000SETINDEX_VALIDATE_KEY:0 | | PART_TBL..:/usr/ud/parttbl | | TMP...:/dbms2/tmp/ | - Original Message - From: Tom Whitmore tewhitm...@ratex.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Monday, January 21, 2013 10:13 AM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Hi Doug, We use UniVerse instead of UniData, but I understand the tools are the same. You do not need to be on the same subnet or network, but you do need access to the subnet/network for the RPC port. I have successfully connected to external servers without a problem, you may need to use an IP address rather than the host name but that depends on how you have things configured. I would strongly recommend that you use a VPN or a gateway to access your server, this keeps your sever and network secure. I think using VPN is easier to use with the admin tools than setting up a gateway. Tom Whitmore RATEX Business Solutions -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of doug chanco Sent: Monday, January 21, 2013 12:02 PM To: 'U2 Users List' Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly To use that tool, he would have to be on the same subnet/network as the server (i.e. he would have to vpn into their system, or remote desktop/vnc/etc into a system they have over there that had the admin tool installed). Having said that I think that's a great idea, the customer could setup a windows system with the admin tool installed and logmein/gotomypc/etc (which would NOT require vpn access) so that he could access the system remotely and run uniadmin Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 11:52 AM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Can you email me your worksheet, I would be very interested in seeing it Thanks Dougc dougATchancofamilyDOTcom -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 1:13 PM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Heres a copy of my udtconf, were running about 100 users. Its unix and it's been very stable for 18 some odd years. I have a calculation worksheet somewhere for setting these variables if your interested. Robert | | | NUSERS.:130SHM_LPINENTS:20 UDT_LANGGRP.:255/192/129 | | SHM_GNTBLS.:100SHM_LMINENTS:48 N_FILESYS...:200 | | SHM_GNPAGES:32 SHM_LCINENTS:100 NULL_FLAG...:0 | | SHM_GPAGESZ:512SHM_LPAGESZ.:8 NVLMARK.: | | SHM_FREEPCT:25 AVG_TUPLE_LEN...:4 N_GLM_GLOBAL_BUCKET:101 | | SHM_NFREES.:1 EXPBLKSIZE..:32 N_GLM_SELF_BUCKET..:23 | | MIN_MEMORY_TEMP.:64 GLM_MEM_SEGSZ:4194304 | | SHMMAX..:1073741824COMPACTOR_POLICY:1 | | SHMMIN..:1 VARMEM_PCT..:50 MAX_NEXT_HOLD_DIGITS:4 | | SHM_ATT_ADD.:0 CHECK_HOLD_EXIST:0 | | SHM_LBA.:4096 | | LOCK FIFO: 1 System PV: 3 | | | | NFILES..:1019 WRITE_TO_CONSOLE:0 MAX_FLENGTH:1073741824 | | SHM_MIN_NATT:4 FCNTL_ON:0 SBCS_SHM_SIZE..:4194304 | | NSEM_PSET...:8 TOGGLE_NAP_TIME.:21 MAX_OBJ_SIZE...:307200 | | GRP_FREE_BLK:5 JRNL_MAX_PROCS..:1 EFS_LCKTIME:0 | | SHM_FIL_CNT.:2048 JRNL_MAX_FILES..:400 MAX_CAPT_LEVEL.:2 | | SPLIT_LOAD:60 MAX_OPEN_FILE...:2000MAX_RETN_LEVEL.:2 | | MERGE_LOAD:40 MAX_OPEN_SEQF...:150 MGLM_BUCKET_SIZE.:50 | | KEYDATA_SPLIT_LOAD:95 MAX_OPEN_OSF:100 SETINDEX_BUFFER_KEYS.:0 | | KEYDATA_MERGE_LOAD:40 MAX_DSFILES.:1000SETINDEX_VALIDATE_KEY:0 | | PART_TBL..:/usr/ud/parttbl | | TMP...:/dbms2/tmp/ | - Original Message - From: Tom Whitmore tewhitm...@ratex.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Monday, January 21, 2013 10:13 AM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Hi Doug, We use UniVerse instead of UniData, but I understand the tools are the same. You do not need to be on the same subnet or network, but you do need access to the subnet/network for the RPC port. I have successfully connected to external servers without a problem, you may need to use an IP address rather than the host name but that depends on how you have things configured. I would strongly recommend that you use a VPN or a gateway to access your server, this keeps your sever and network secure. I think using VPN is easier to use with the admin tools than setting up a gateway. Tom Whitmore RATEX Business Solutions -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of doug chanco Sent: Monday, January 21, 2013 12:02 PM To: 'U2 Users List' Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly To use that tool, he would have to be on the same subnet/network as the server (i.e. he would have to vpn into their system, or remote desktop/vnc/etc into a system they have over there that had the admin tool installed). Having said that I think that's a great idea, the customer could setup a windows system with the admin tool installed and logmein/gotomypc/etc (which would NOT require vpn access) so that he could access the system remotely and run uniadmin Dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Robert Frailey Sent: Monday, January 21, 2013 11:52 AM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Didn't you get Unidata admin tools? You can connect to the server with it and look at all the locks, files and configs with the one tool. Thank You Robert Frailey Manager Information Systems Utah Medical Products Inc. Femcare-Nikomed Ltd. rfrai...@utahmed.com 801-569-4016 - Original Message - From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Saturday, January 19, 2013 12:16 PM Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Thanks Wally. You so rock. This customer does have a support contract with an awful provider that does very little, charges very much, and is the antithesis of responsive. That's why they come to me. I don't have any official connections but can usually get them some help within minutes vs. days. Unfortunately, the way the VAR agreements are structured with U2, there's no way they can ever escape the abuse they get from this vendor. On Sat, Jan 19, 2013 at 4:01 PM, Wally Terhune wterh...@rocketsoftware.comwrote: I doubt that your system non-responsiveness is related to UniData record locks. Not sure what you even mean by an 'overflowed lock table'. However, you can look at record locks with a non-documented engineering tool: glm_tool. First menu choice: 1: Show the Map of Locks might be useful. If you just want to see the number of locks held by each UniData process, select 4: More Options for Developer and then 1: transaction table (in this case 'transaction' refers to each udt process - listing udtnos counting from 0 instead of 1 (as you see in listuser output). Does this customer not have a support contract? Have they submitted a udtdiag dump to their support provider while the problem was occuring? regards, Wally Terhune U2 Support Architect Rocket Software Tel: (720) 475-8055 Mobile: (303) 807-6222 wterh...@rocketsoftware.com u2supp...@rocketsoftware.com From: u2-users-boun...@listserver.u2ug.org [ u2-users-boun...@listserver.u2ug.org] on behalf of Kevin King [ke...@precisonline.com] Sent: Saturday, January 19, 2013 12:16 PM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Distance should not be an obstacle. You may not be able to physically access the console but console access should still be possible either by getting someone else to physically eyeball it or a launch remote console session yourself. How about using something like a Cyclades terminal server to access one or more host consoles remotely via ssh? Even if a Cyclades is not an option then what about a network connected management facility such as an ALOM/ILOM or similar? Again via ssh, login and launch the console process from there. As long as the chassis had power then you should have LOM access if the WAN is available. Voila! Serial console heaven from the other side of the world even. Sure talking someone on the other end of a phone through a system recovery is not the easiest of tasks and it may not be as quick as direct connect in the same room or even an ssh session over your LAN but it sure would beat an acoustic coupled 300 baud half duplex modem that we used to use at AWA to dial in and fix GFE's etc. So how do you access your system console? Cheers Peter -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Saturday, 19 January 2013 3:33 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users --- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600) --- ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Write a program which captures and writes the lock table to an O/S level text file, then sleeps for 15 minutes, then does it again. Kick it off as a phantom job, running forever. -Original Message- From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Sat, Jan 19, 2013 11:16 am Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
I doubt that your system non-responsiveness is related to UniData record locks. Not sure what you even mean by an 'overflowed lock table'. However, you can look at record locks with a non-documented engineering tool: glm_tool. First menu choice: 1: Show the Map of Locks might be useful. If you just want to see the number of locks held by each UniData process, select 4: More Options for Developer and then 1: transaction table (in this case 'transaction' refers to each udt process - listing udtnos counting from 0 instead of 1 (as you see in listuser output). Does this customer not have a support contract? Have they submitted a udtdiag dump to their support provider while the problem was occuring? regards, Wally Terhune U2 Support Architect Rocket Software Tel: (720) 475-8055 Mobile: (303) 807-6222 wterh...@rocketsoftware.com u2supp...@rocketsoftware.com From: u2-users-boun...@listserver.u2ug.org [u2-users-boun...@listserver.u2ug.org] on behalf of Kevin King [ke...@precisonline.com] Sent: Saturday, January 19, 2013 12:16 PM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Usually we can still telnet into AIX. That's how I'm running the stopud/startud. This errpt command is new to me but thanks to everyone for that input; that should be very valuable. I still wish there was some way we could do a LIST.READU from AIX without having to login to udt, like listuser but for locks. I don't know if an overflowed lock table is even remotely related to the problem, but it would be nice to be able to check it without having to get to TCL. -K On Fri, Jan 18, 2013 at 10:49 AM, doug chanco d...@chancofamily.com wrote: That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
You need an onsite expert user Kevin. Just for such things. -Original Message- From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Thu, Jan 17, 2013 7:18 pm Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
That was my question as well, do you telnet or ssh into the box? Once you login su over to root and do errpt -aD this will give you short listing of all the errors aix has logged and consolidates duplicate errors (see below for an example) LABEL: STOK_RCVRY_EXIT Date/Time: Tue Dec 14 15:25:33 Type:TEMP Resource Name: tok0 Description PROBLEM RESOLVED Detail Data FILE NAME line: 273 file: stok_wdt.c SENSE DATA DEVICE ADDRESS 0004 AC62 25F1 My aix sysadmin days are way behind me but I am willing to bet that the company has a aix support contract and you could contact them to help with any errors found. dougc -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, January 18, 2013 12:33 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly On 18/01/13 03:18, Kevin King wrote: Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. How do you get to the box to do a stopud/startud? This implies AIX is up and responding. So it shouldn't be too hard to open a remote session at the AIX level to find out what is going wrong. Not that I can advise how to do it ... Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Holy crap, that would have been good information to have 10 years ago. :-) Thanks Jeff for the errpt mention; I always wondering where that stuff was on AIX. On Wed, Jan 16, 2013 at 1:36 PM, Jeff Schasny jscha...@gmail.com wrote: The AIX errpt command reads the system error log. See man errpt for details. That would be a good place to start Kevin King wrote: We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K __**_ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/**mailman/listinfo/u2-usershttp://listserver.u2ug.org/mailman/listinfo/u2-users -- --**--** Jeff Schasny - Denver, Co, USA jschasny at gmail dot com --**--** __**_ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/**mailman/listinfo/u2-usershttp://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Peter, I'm 1200+ miles away from the box. Getting to the console is not an option. To restart, I've been forcing Unidata down with stopud -f and starting it again with startud. Yeah, drastic, I know. But that's why I'm looking for better ways. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Other than telnet. *Go* to the server, and open a local session, *not telnet* Do you get a login prompt? Or do they I mean? Next time have them do that. It may simply be that the telnet itself died. -Original Message- From: Kevin King ke...@precisonline.com To: U2 Users List u2-users@listserver.u2ug.org Sent: Wed, Jan 16, 2013 10:40 am Subject: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Hi Kevin, I can't help you with what you're asking for, directly, but as a suggestion on something to try, if you make a small script that dumps the currently running processes into a date/time stamped text file, new for each time it runs, then you could set it up in the system scheduler to run every x number of minutes. You could schedule it to start running at something like 1:00am. It might help to isolate exactly what processes are running just before the lock-up and give you some more information to work with. I'm guessing that it's a process and not something like the /tmp location getting full for some reason. Good luck. BobW -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Kevin King Sent: Wednesday, January 16, 2013 10:40 AM To: U2 Users List Subject: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
I have had to troubleshoot many strange things like that over a 26 year period for different companies. It isn't always the operating environment or OS. - In one case we traced it back to an arc welder producing RFI (radio frequency interference). We had to use insulated wiring (low capacitance, twisted pair, shielded wiring) to solve it. We had to shorten the cable to the shortest length possible and make sure that the ground connections on the cable were connected. There are usually limits to how long a cable can be before you go over the limit. The signal gets weaker and is therefore vulnerable to RFI at longer and longer distances. That is why you have to lower the speed of line to go over certain distances. Cables act like an antenna and introduce RFI into a system. That is why many people will disconnect unused cables from their computers. - At one company they were using a coax cable from RADIO SHACK that was NOT the exact cable required even though the cable plugged in just fine. It didn't have the electrical characteristics that was necessary to maintain a high quality signal and caused network crashes once a day. - In another case it was a failing UPS producing flaky power. - In another case, bad memory chips (determined by using a memory testing program that runs overnight). - One time, a company had power fluctuations due to large equipment on the same line. We had to connect the computer to a dedicated line (straight out to the power pole) to solve the problem. - I don't know what area you are in, but some power companies balance their loads at night, and this sometimes creates brief, momentary, power outages that can effect equipment if you don't have a UPS. - If the power is wired incorrectly, it can behave strangely. A toaster oven will work just fine, but a sensitive computer will not. You have to get one of those power outlet testing devices ($10) and plug it in and look at the light readout. It will tell you if negative and positive are reversed, or there is no ground, etc. - A PM (preventative maintenance) might be in order. Sometimes dust will build up inside of a case on the motherboard and short connections out once in a while when there is enough ambient electricity to come through it. In other words it receives and picks up energy like an antenna. - Grounding the computer with your mind works in cases where there are spiritual disturbances. I have had customers with spiritual problems that cause my computer to crash at the exact moment they call. Or a light bulb blows out at the exact moment. Grounding it with your mind can dissipate negative energy if that is what is causing it. This happens a lot when there is negative/angry/hostile people throwing negative energy at the computer. Putting a carbon-filled anti-static mat underneath the computer can help as well. - I have seen problems due to overheating computer also. We have had to add CPU heat sinks/fans, replace power supply fans, blow the dust out of them, etc. - Sometimes a line can pick up a control characters to pause the output. In the old days it was only CTRL-S, but UNIX adds some additional signals to the mix. Clearing the line and resetting it can work in some cases. - One good thing to do is keep a log of the crash each and every time. You put the port, date, time, event, user, and any special process running at that time. Over a period of time, you might see a pattern. For example, something late at night suggests power problems. Something first thing Monday morning, might be due to a brownout caused by the powering up of large factory equipment. Find out when the weekly cleaning people come in. I had a problem once with client that kept having problems with their computer on Friday morning. It would crash Thursday nights. I happened to be working late one evening for them, and I watch a cleaning person walk up to the computer, unplug it, and then plug in their vacuum cleaner and start vacuuming! They would finish vacuuming, and then plug the computer back in. If you still can't resolve it after trying/researching these things, please post the problem again. Also, please let us know what it took to fix it for our FAQ. ROBERT NORMAN AND ASSOCIATES 23441 Golden Springs Dr., #289, Diamond Bar, CA 91765 (951) 541-1668 i...@keyway.net mailto:i...@keyway.net http://users.keyway.net/~ice/ http://users.keyway.net/%7Eice/ Affordable UNIDATA programming services for PICK/BASIC, DATA/BASIC, UniVerse Basic, UniBasic, R/BASIC, jBC. On 1/16/2013 10:39 AM, Kevin King wrote: We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
The AIX errpt command reads the system error log. See man errpt for details. That would be a good place to start Kevin King wrote: We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
I'd download nmon have it running in 5 minute increments, then look at it in the morning. I'd also look at errpt to see if it's logging any errors. Date: Wed, 16 Jan 2013 11:09:19 -0800 From: bob_woodw...@k2sports.com To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly Hi Kevin, I can't help you with what you're asking for, directly, but as a suggestion on something to try, if you make a small script that dumps the currently running processes into a date/time stamped text file, new for each time it runs, then you could set it up in the system scheduler to run every x number of minutes. You could schedule it to start running at something like 1:00am. It might help to isolate exactly what processes are running just before the lock-up and give you some more information to work with. I'm guessing that it's a process and not something like the /tmp location getting full for some reason. Good luck. BobW -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Kevin King Sent: Wednesday, January 16, 2013 10:40 AM To: U2 Users List Subject: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
As another alternative I would see if syslogging is turned on. Look at /etc/syslog.conf . Check for any lines that do not have a # in front. These lines would be the paths to the logs. Dan Goble | Senior Systems Engineer Interline Brands, Inc. 804 East Gate Drive Suite 100, Mount Laurel, NJ 08054 Office: 856.533.3110 | Mobile: 609.792.6855 E-mail: dan.go...@interlinebrands.com | Website: www.interlinebrands.com This e-mail and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. Please notify the sender immediately by e-mail if you have received this e-mail in error and delete all copies of this message. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Wednesday, January 16, 2013 3:36 PM To: U2 Users List Subject: Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly The AIX errpt command reads the system error log. See man errpt for details. That would be a good place to start Kevin King wrote: We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly
Not sure what it's called in AIX/IBM speak but if it was a Solaris machine I'd be finding the system console and looking at that to see if anything interesting is showing. At least then you have a direct serial connection (and/or K/B + monitor) facility to get directly to the host without relying on telnet or any other communications daemon to be running. Are you sure it's completely dead? What do you do to get back up and running again? Does it come alive again suddenly? The aix server we have here sometimes slows down so much during the nightly backup that it seems completely dead. Apparently this is/was the disks being thrashed during the save and nothing else gets a look in for ages. As others have said - writing a sript to dump out current list of running processes is a good idea. You might want to also include some other stuff like the load averages and io activities on the system at the same time. See the uptime, iostat, and vmstat commands. ThIssuing a sngle iostat or vmstat may not always yield accurate results - my experience on Solaris is the first line of output is trash and you need the next few lines to get a clearer picture. So an 'iostat 1 5' might get you a better idea as to where a potential bottleneck might be. I'd also be running a similar script during the day or at least a few times during. This will give you a 'normal' point of reference to compare to the overnight happenings. Cheers Peter -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Kevin King Sent: Thursday, 17 January 2013 4:40 To: U2 Users List Subject: [U2] Unidata 6.1.13/AIX 5.3 Stops Unexpectedly We have a customer with a box as described in the subject that has recently been experiencing something odd. Overnight, the system will just lock up and stop responding. Normally there isn't anyone doing anything at night; the backup runs, that sort of thing, but then some days when they come in early in the morning the entire machine is unresponsive. Connecting with telnet there is no login prompt, just a blank screen. It happened last night and I've confirmed from the logs that the backup (which is using a split mirror backup strategy) ran successfully and finished at 2:02am. Sometime between then and 4am, when there really should not have been anything going on, it just locked up. Are there any logs that can be enabled on AIX to record some evidence for these kinds of failures so we can figure out what's going on with this machine? -K ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users --- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600) --- ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users