Memo: RE: [U2] Universe file size limit
Hi Aleksejs, thanks for that info. It does seem to fit in with what we have encountered. We are restoring the file from backup as fixtool isnt able to fix it. Once that is done, we will convert it to a distributed file and then rebuild the missing data. thanks again to everyone... Aleksejs Truhans <[EMAIL PROTECTED] on 01 Sep 2004 17:51 Please respond to [EMAIL PROTECTED] Sent by:[EMAIL PROTECTED] To:[EMAIL PROTECTED] cc:Asvin DATTANI/IBEU/[EMAIL PROTECTED] bcc: Subject: RE: [U2] Universe file size limit Hello! We have had problems with big UV 64-bit files on HPUX11i PA-RISC. As far as I remember, it is UV version 9.6.2.1 or 9.6.1.smth, where a static hashed file bigger than 4GB could be killed by modifying a record (beyond the 4GB border?). That is, if one is just adding records to a file it is not a problem, we've seen them reaching 13GB smoothly. But modifying already existing record (so that its length changes?) may (it does not happen every time) mess up entire group. From filepeek it looked like record headers were getting overwritten by record data. Since then we are on 9.6.2.1, distributed every big file getting record updates so that none of them exceeds 4GB. Hope, this helps. Aleksejs. HSBC Bank plc Registered Office: 8 Canada Square, London E14 5HQ Registered in England - Number 14259 Authorised and regulated by the Financial Services Authority Member of the HSBC Bank marketing group. We sell life assurance, pensions and collective investment schemes and advise only on our own range of these products. _ This transmission has been issued by a member of the HSBC Group "HSBC" for the information of the addressee only and should not be reproduced and / or distributed to any other person. Each page attached hereto must be read in conjunction with any disclaimer which forms part of it. Unless otherwise stated, this transmission is neither an offer nor the solicitation of an offer to sell or purchase any investment. Its contents are based on information obtained from sources believed to be reliable but HSBC makes no representation and accepts no responsibility or liability as to its completeness or accuracy. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Universe file size limit
Hello! We have had problems with big UV 64-bit files on HPUX11i PA-RISC. As far as I remember, it is UV version 9.6.2.1 or 9.6.1.smth, where a static hashed file bigger than 4GB could be killed by modifying a record (beyond the 4GB border?). That is, if one is just adding records to a file it is not a problem, we've seen them reaching 13GB smoothly. But modifying already existing record (so that its length changes?) may (it does not happen every time) mess up entire group. From filepeek it looked like record headers were getting overwritten by record data. Since then we are on 9.6.2.1, distributed every big file getting record updates so that none of them exceeds 4GB. Hope, this helps. Aleksejs. Date: Wed, 1 Sep 2004 14:47:30 +0100 From: [EMAIL PROTECTED] Subject: Memo: RE: [U2] Universe file size limit Hi, Thanks for all the responses. Ulimit, is set to unlimited so that shouldn't be the problem. Our unix support people cant find any other HPUX kernel parameter that would affect the maximum file size. Glen's recollection is correct - HPUX support file sizes up to 128 terabytes. It is looking increasingly as if this is a Universe 9.6 issue or a 9.6 on HPUX issue. is anyone running UV 9.6 with files over 4 GB? thanks again.. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
Re: Memo: Re: RE: [U2] Universe file size limit
Well, I do know that the Internal data error message (at 9.6) is generated within the file handling subsystem when blink values are mucked up; interesting that no information pertaining to this type of error is displayed so one might postulate this isn't the issue. This message can also be generated when the INMAT function isn't used against an array or in the (undocumented) ASSIGN statement referencing an invalid action. All other generators of this message are within utilities, such as create.file, file.stat, hash.help, resize, etc, which I assume isn't the case here. It does sound like this file may be beyond the repair of a tool and could require manual intervention or restoration. One thing I just thought of: are there indices on this file? Best of luck! At 10:49 AM 9/1/2004, you wrote: Hi Glen, This is the output when we first ran across the problem: 0008 Program "FIS00200": Line 551, Internal data error. 0009 Program "FIS00200": Line 555, Internal data error. 0010 Program "FIS00200": Line 555, WRITE failure. Line 551 is a MATREADU from the file, and line 555 is a MATWRITE to the file. This file is a an archive of another file, so there cannot be any invalid characters in the key. We just ran fixtool on the file, which claimed to have found 657 errors and fixed 649 : Start Diagnostics. 0 Errors found in physical structure. 0 Errors found in file header. 0 Errors found in primary groups. 654 Errors found in data. 3 Other errors found in file. 657 Errors on file. Start Fixing. 649 Errors fixed. Fixtool Completed. but when I did a COUNT of the file there were still problems: WARNING: Internal file corruption detected during SCAN operation! Invalid pad length detected within record block! Filename: JOBHIST Group offset: 3575808 (0x0) Key: Abnormal termination of UniVerse. Fault type is 11. Layer type is Query. I am getting the sysadmin to run the fixtool command again thanks for your help, Glenn Herbert <[EMAIL PROTECTED] on 01 Sep 2004 15:24 Please respond to [EMAIL PROTECTED] Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: bcc: Subject:Re: RE: [U2] Universe file size limit I have a version of 9.6 (and beyond!) running here on an hp 11.11i with a few large files that exceed 4 gig (one is around 20 gig) with no problems; note that these files aren't exercised all that much as I don't really play around with this stuff much anymore. Additionally, we have a number of DS end users who have files that regularly exceed this size and don't have issues, so I'm sure it's something specific to your system. I don't recall you mentioning what the exact error was other than it being an internal data error, so maybe posting the entire message might jiggle some latent memory somewhere. HSBC Bank plc Registered Office: 8 Canada Square, London E14 5HQ Registered in England - Number 14259 Authorised and regulated by the Financial Services Authority Member of the HSBC Bank marketing group. We sell life assurance, pensions and collective investment schemes and advise only on our own range of these products. _ This transmission has been issued by a member of the HSBC Group "HSBC" for the information of the addressee only and should not be reproduced and / or distributed to any other person. Each page attached hereto must be read in conjunction with any disclaimer which forms part of it. Unless otherwise stated, this transmission is neither an offer nor the solicitation of an offer to sell or purchase any investment. Its contents are based on information obtained from sources believed to be reliable but HSBC makes no representation and accepts no responsibility or liability as to its completeness or accuracy. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
Memo: Re: RE: [U2] Universe file size limit
Hi Glen, This is the output when we first ran across the problem: 0008 Program "FIS00200": Line 551, Internal data error. 0009 Program "FIS00200": Line 555, Internal data error. 0010 Program "FIS00200": Line 555, WRITE failure. Line 551 is a MATREADU from the file, and line 555 is a MATWRITE to the file. This file is a an archive of another file, so there cannot be any invalid characters in the key. We just ran fixtool on the file, which claimed to have found 657 errors and fixed 649 : Start Diagnostics. 0 Errors found in physical structure. 0 Errors found in file header. 0 Errors found in primary groups. 654 Errors found in data. 3 Other errors found in file. 657 Errors on file. Start Fixing. 649 Errors fixed. Fixtool Completed. but when I did a COUNT of the file there were still problems: WARNING: Internal file corruption detected during SCAN operation! Invalid pad length detected within record block! Filename: JOBHIST Group offset: 3575808 (0x0) Key: Abnormal termination of UniVerse. Fault type is 11. Layer type is Query. I am getting the sysadmin to run the fixtool command again thanks for your help, Glenn Herbert <[EMAIL PROTECTED] on 01 Sep 2004 15:24 Please respond to [EMAIL PROTECTED] Sent by:[EMAIL PROTECTED] To:[EMAIL PROTECTED] cc: bcc: Subject: Re: RE: [U2] Universe file size limit I have a version of 9.6 (and beyond!) running here on an hp 11.11i with a few large files that exceed 4 gig (one is around 20 gig) with no problems; note that these files aren't exercised all that much as I don't really play around with this stuff much anymore. Additionally, we have a number of DS end users who have files that regularly exceed this size and don't have issues, so I'm sure it's something specific to your system. I don't recall you mentioning what the exact error was other than it being an internal data error, so maybe posting the entire message might jiggle some latent memory somewhere. HSBC Bank plc Registered Office: 8 Canada Square, London E14 5HQ Registered in England - Number 14259 Authorised and regulated by the Financial Services Authority Member of the HSBC Bank marketing group. We sell life assurance, pensions and collective investment schemes and advise only on our own range of these products. _ This transmission has been issued by a member of the HSBC Group "HSBC" for the information of the addressee only and should not be reproduced and / or distributed to any other person. Each page attached hereto must be read in conjunction with any disclaimer which forms part of it. Unless otherwise stated, this transmission is neither an offer nor the solicitation of an offer to sell or purchase any investment. Its contents are based on information obtained from sources believed to be reliable but HSBC makes no representation and accepts no responsibility or liability as to its completeness or accuracy. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
Re: Memo: RE: [U2] Universe file size limit
I have a version of 9.6 (and beyond!) running here on an hp 11.11i with a few large files that exceed 4 gig (one is around 20 gig) with no problems; note that these files aren't exercised all that much as I don't really play around with this stuff much anymore. Additionally, we have a number of DS end users who have files that regularly exceed this size and don't have issues, so I'm sure it's something specific to your system. I don't recall you mentioning what the exact error was other than it being an internal data error, so maybe posting the entire message might jiggle some latent memory somewhere. At 09:47 AM 9/1/2004, you wrote: Hi, Thanks for all the responses. Ulimit, is set to unlimited so that shouldn't be the problem. Our unix support people cant find any other HPUX kernel parameter that would affect the maximum file size. Glen's recollection is correct - HPUX support file sizes up to 128 terabytes. It is looking increasingly as if this is a Universe 9.6 issue or a 9.6 on HPUX issue. is anyone running UV 9.6 with files over 4 GB? thanks again.. HSBC Bank plc Registered Office: 8 Canada Square, London E14 5HQ Registered in England - Number 14259 Authorised and regulated by the Financial Services Authority Member of the HSBC Bank marketing group. We sell life assurance, pensions and collective investment schemes and advise only on our own range of these products. _ This transmission has been issued by a member of the HSBC Group "HSBC" for the information of the addressee only and should not be reproduced and / or distributed to any other person. Each page attached hereto must be read in conjunction with any disclaimer which forms part of it. Unless otherwise stated, this transmission is neither an offer nor the solicitation of an offer to sell or purchase any investment. Its contents are based on information obtained from sources believed to be reliable but HSBC makes no representation and accepts no responsibility or liability as to its completeness or accuracy. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
Memo: RE: [U2] Universe file size limit
Hi, Thanks for all the responses. Ulimit, is set to unlimited so that shouldn't be the problem. Our unix support people cant find any other HPUX kernel parameter that would affect the maximum file size. Glen's recollection is correct - HPUX support file sizes up to 128 terabytes. It is looking increasingly as if this is a Universe 9.6 issue or a 9.6 on HPUX issue. is anyone running UV 9.6 with files over 4 GB? thanks again.. HSBC Bank plc Registered Office: 8 Canada Square, London E14 5HQ Registered in England - Number 14259 Authorised and regulated by the Financial Services Authority Member of the HSBC Bank marketing group. We sell life assurance, pensions and collective investment schemes and advise only on our own range of these products. _ This transmission has been issued by a member of the HSBC Group "HSBC" for the information of the addressee only and should not be reproduced and / or distributed to any other person. Each page attached hereto must be read in conjunction with any disclaimer which forms part of it. Unless otherwise stated, this transmission is neither an offer nor the solicitation of an offer to sell or purchase any investment. Its contents are based on information obtained from sources believed to be reliable but HSBC makes no representation and accepts no responsibility or liability as to its completeness or accuracy. --- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
