Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding
Sent from Yahoo Mail for iPad On Thursday, May 23, 2024, 2:29 AM, ITschak Mugzach <05a7ced721d8-dmarc-requ...@listserv.ua.edu> wrote: All major relational databases support variable length data load (incl. DB2) . This is not a problem at all. ITschak Mugzach *|** IronSphere Platform* *|* *Information Security Continuous Monitoring for z/OS, x/Linux & IBM I **| z/VM coming soon * On Wed, May 22, 2024 at 11:41 PM Tom Ross wrote: > >On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" > > wrote: > > > >:>3. Depends on the language. IMS can have variable length segments but > not= > > variable length fields, so any variable length fields are defined by the > a= > >pplication. If the application is in COBOL, there's no native variable > leng= > >th field format. Applications can simulate variable length fields but how > t= > >hey do it is in application logic. > > > >Well, I don't know about that. > > > > > >01 var-field-length pic s9(4) comp. > > > >01 var-field. > > 05 var-field-character pic x occurs 1 to 1000 times depending on > >var-field-length. > > > >When a MOVE is performed to or from VAR-FIELD the length from > VAR-FIELD-LEN > >GTH is used. > > > And now COBOL on z/OS has dynamic-length data items! > > 1 DL-A PIC X DYNAMIC VALUE 'abcd '. > > > > Cheers, > Tom Ross >> COBOL is the Language of the Future! << > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding
All major relational databases support variable length data load (incl. DB2) . This is not a problem at all. ITschak Mugzach *|** IronSphere Platform* *|* *Information Security Continuous Monitoring for z/OS, x/Linux & IBM I **| z/VM coming soon * On Wed, May 22, 2024 at 11:41 PM Tom Ross wrote: > >On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" > > wrote: > > > >:>3. Depends on the language. IMS can have variable length segments but > not= > > variable length fields, so any variable length fields are defined by the > a= > >pplication. If the application is in COBOL, there's no native variable > leng= > >th field format. Applications can simulate variable length fields but how > t= > >hey do it is in application logic. > > > >Well, I don't know about that. > > > > > >01 var-field-length pic s9(4) comp. > > > >01 var-field. > > 05 var-field-character pic x occurs 1 to 1000 times depending on > >var-field-length. > > > >When a MOVE is performed to or from VAR-FIELD the length from > VAR-FIELD-LEN > >GTH is used. > > > And now COBOL on z/OS has dynamic-length data items! > > 1 DL-A PIC X DYNAMIC VALUE 'abcd '. > > > > Cheers, > Tom Ross >> COBOL is the Language of the Future! << > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding
>On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" > wrote: > >:>3. Depends on the language. IMS can have variable length segments but not= > variable length fields, so any variable length fields are defined by the a= >pplication. If the application is in COBOL, there's no native variable leng= >th field format. Applications can simulate variable length fields but how t= >hey do it is in application logic. > >Well, I don't know about that. > > >01 var-field-length pic s9(4) comp. > >01 var-field. > 05 var-field-character pic x occurs 1 to 1000 times depending on >var-field-length. >When a MOVE is performed to or from VAR-FIELD the length from VAR-FIELD-LEN >GTH is used. And now COBOL on z/OS has dynamic-length data items! 1 DL-A PIC X DYNAMIC VALUE 'abcd '. Cheers, Tom Ross >> COBOL is the Language of the Future! << -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
01 segment.record. 05 name1-length pic s9(04) comp. 05 name2-length pic s9(04) comp. 05 name3-length pic s9(04) comp. 05 name4-length pic s9(04) comp. 05 name5-length pic s9(04) comp. 05 name1-text. 10 name1-char pic x(01) occurs 0 to 99 times depending on name1-length. 05 name2-text. 10 name2-char pic x(01) occurs 0 to 99 times depending on name2-length. 05 name3-text. 10 name3-char pic x(01) occurs 0 to 99 times depending on name3-length. 05 name4-text. 10 name4-char pic x(01) occurs 0 to 99 times depending on name4-length. 05 name5-text. 10 name5-char pic x(01) occurs 0 to 99 times depending on name5-length. On Tue, May 21, 2024 at 11:39 AM Mike Schwab wrote: > > All thength fields MUST occur before length depending on. > > On Tue, May 21, 2024 at 10:23 AM Binyamin Dissen > wrote: > > > > On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" > > wrote: > > > > :>3. Depends on the language. IMS can have variable length segments but not > > variable length fields, so any variable length fields are defined by the > > application. If the application is in COBOL, there's no native variable > > length field format. Applications can simulate variable length fields but > > how they do it is in application logic. > > > > Well, I don't know about that. > > > > > > 01 var-field-length pic s9(4) comp. > > > > 01 var-field. > >05 var-field-character pic x occurs 1 to 1000 times depending on > > var-field-length. > > > > > > When a MOVE is performed to or from VAR-FIELD the length from > > VAR-FIELD-LENGTH > > is used. > > > > -- > > Binyamin Dissen > > http://www.dissensoftware.com > > > > Director, Dissen Software, Bar & Grill - Israel > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
All thength fields MUST occur before length depending on. On Tue, May 21, 2024 at 10:23 AM Binyamin Dissen wrote: > > On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" > wrote: > > :>3. Depends on the language. IMS can have variable length segments but not > variable length fields, so any variable length fields are defined by the > application. If the application is in COBOL, there's no native variable > length field format. Applications can simulate variable length fields but how > they do it is in application logic. > > Well, I don't know about that. > > > 01 var-field-length pic s9(4) comp. > > 01 var-field. >05 var-field-character pic x occurs 1 to 1000 times depending on > var-field-length. > > > When a MOVE is performed to or from VAR-FIELD the length from VAR-FIELD-LENGTH > is used. > > -- > Binyamin Dissen > http://www.dissensoftware.com > > Director, Dissen Software, Bar & Grill - Israel > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
One thing that I haven't seen mentioned is that converting from EBCDIC to UTF-8 is that some characters in UTF-8 are 2 or 4 bytes, thus changing the lengths of fields. -- Tom Marchant On Tue, 21 May 2024 04:08:16 -0500, Jason Cai wrote: ... > We have all the necessary copybooks and are currently looking to convert > these IMS unload files from EBCDIC to UTF-8 encoding. ... -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
But the location of the ODO length is application dependent. It might even be a calculated field in working storage. It isn't like there's a defined field type: "Var-length field is binary fullword length followed by a stream of bytes where the length of the stream is the fullword length, not inclusive" So if an IMS segment contains variable length fields, you have to know how the application is defining them. Where's the length? How long is the length field? Does the length include the length of the length field? Or is the field terminated by some value (e.g. a null)? Is the starting position of the field variable? Does the length use a multiplier? Is the length 0 or 1 based? -Original Message- From: IBM Mainframe Discussion List On Behalf Of Binyamin Dissen Sent: Tuesday, May 21, 2024 10:23 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding. On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" wrote: :>3. Depends on the language. IMS can have variable length segments but not variable length fields, so any variable length fields are defined by the application. If the application is in COBOL, there's no native variable length field format. Applications can simulate variable length fields but how they do it is in application logic. Well, I don't know about that. 01 var-field-length pic s9(4) comp. 01 var-field. 05 var-field-character pic x occurs 1 to 1000 times depending on var-field-length. When a MOVE is performed to or from VAR-FIELD the length from VAR-FIELD-LENGTH is used. -- Binyamin Dissen http://www.dissensoftware.com/ Director, Dissen Software, Bar & Grill - Israel -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
On Tue, 21 May 2024 15:01:12 + "Schmitt, Michael" wrote: :>3. Depends on the language. IMS can have variable length segments but not variable length fields, so any variable length fields are defined by the application. If the application is in COBOL, there's no native variable length field format. Applications can simulate variable length fields but how they do it is in application logic. Well, I don't know about that. 01 var-field-length pic s9(4) comp. 01 var-field. 05 var-field-character pic x occurs 1 to 1000 times depending on var-field-length. When a MOVE is performed to or from VAR-FIELD the length from VAR-FIELD-LENGTH is used. -- Binyamin Dissen http://www.dissensoftware.com Director, Dissen Software, Bar & Grill - Israel -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
Arrgh, meant to type that unload from image copy restores to a DATASPACE, i.e. in memory. But my point is it requires an IMS system. -Original Message- From: IBM Mainframe Discussion List On Behalf Of Schmitt, Michael Sent: Tuesday, May 21, 2024 10:01 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding. Some others have answered your questions, but here's my $0.02 USD: 1. You should convert all the numeric fields to edited text. Otherwise, unless you're using a COBOL system on Linux, you'll have trouble with the field formats. Think packed decimal, implicit signs, little/big endian, etc. 2. When a segment has multiple layouts, usually the application has some field in the segment that it can use to know which layout to use. But it is *possible* that it uses different layouts based on *other* segment's data. It could be in the same database record (a "record" is a root and all its children). Or a different record. Or even in a different database. 3. Depends on the language. IMS can have variable length segments but not variable length fields, so any variable length fields are defined by the application. If the application is in COBOL, there's no native variable length field format. Applications can simulate variable length fields but how they do it is in application logic. 4. You have several problems in reading unconverted files on Linux: A. Unloads in are in mainframe variable length record format. Unix doesn't have a similar format; files in Unix are just a stream of bytes. How are you going to read the file? (you probably should convert to a text file with LF to separate the records -- but you can't do this BEFORE converting it, because the file's binary data can contain LF characters) B. Need to be able to interpret the unload file format. It is possible (the format is documented) but it is non-trivial. C. Text fields and zoned decimal are in EBCDIC. Your program to read the file would need to convert to ASCII. D. Records in an unload do not contain concatenated keys. E. The format for HALDB databases is different than non-HALDB. F. I don't know how to interpret the data in an image copy at all. I'm not aware of pre-built tools for reading unload files on other than z/OS, so you may need to write your own utilities. 5. An unload file contains the segment data (the logical data the application obtains when it reads the segment), the segment name, and some other data. There's little meta data. For example, you can't use an unload file to load a database without a DBD. The unload is in hierarchical order, but if the database is HDAM or PHDAM then the roots are in random order. The native IMS tools can't create an unload from an image copy. The extra-cost High Performance IMS tools from IBM may be able to do it. But if so, then I bet it would work like the Broadcom tool: it creates the unload from the image copy by restoring the database (from the image copy) to a database, then unloading it. That means you must have an IMS system to run it on. -Original Message- From: Jason Cai Sent: Tuesday, May 21, 2024 4:08 AM To: IBM-MAIN@LISTSERV.UA.EDU; Schmitt, Michael Cc: Jason Cai Subject: How to convert these IMS unload files from EBCDIC to UTF-8 encoding. Dear Michael and all Thank you very much for your response and the valuable information provided. Over the past few days, I discussed the DBD definitions with our IMS DBA, and we have a similar setup where “The only fields that are required to be defined in a DBD are the keys, indexed fields, and any other fields that the application wants to be able to reference in Segment Search Arguments (SSAs). In the IMS system I work on, 99% of the segment fields are not defined in the DBD.” We are considering moving away from the mainframe, and our historical data is preserved on tape in IMS unload format and IMS IC. Initially, we wanted to convert the IMS database to a DB2 and then into a MySQL database to facilitate the migration of backup data from the mainframe to linux. Now, we are only planning to transcode the IMS unload files to make them readable on the Linux platform. We have all the necessary copybooks and are currently looking to convert these IMS unload files from EBCDIC to UTF-8 encoding. Here are a few questions regarding the conversion process: 1.For fields in the copybook defined as PIC A, PIC X, and PIC 9, these require conversion, but for numeric fields like PIC 9(4) COMP-1, no conversion is needed, correct? 2 How do we handle the conversion if the database has multiple copybooks? 3 What is the process for converting variable-length fields? 4 If we choose not to convert anything, are there any tools available on Linux that can read these IMS unload files in EBCDIC encoding? 5.Does the IMS unload file include informatio
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
Some others have answered your questions, but here's my $0.02 USD: 1. You should convert all the numeric fields to edited text. Otherwise, unless you're using a COBOL system on Linux, you'll have trouble with the field formats. Think packed decimal, implicit signs, little/big endian, etc. 2. When a segment has multiple layouts, usually the application has some field in the segment that it can use to know which layout to use. But it is *possible* that it uses different layouts based on *other* segment's data. It could be in the same database record (a "record" is a root and all its children). Or a different record. Or even in a different database. 3. Depends on the language. IMS can have variable length segments but not variable length fields, so any variable length fields are defined by the application. If the application is in COBOL, there's no native variable length field format. Applications can simulate variable length fields but how they do it is in application logic. 4. You have several problems in reading unconverted files on Linux: A. Unloads in are in mainframe variable length record format. Unix doesn't have a similar format; files in Unix are just a stream of bytes. How are you going to read the file? (you probably should convert to a text file with LF to separate the records -- but you can't do this BEFORE converting it, because the file's binary data can contain LF characters) B. Need to be able to interpret the unload file format. It is possible (the format is documented) but it is non-trivial. C. Text fields and zoned decimal are in EBCDIC. Your program to read the file would need to convert to ASCII. D. Records in an unload do not contain concatenated keys. E. The format for HALDB databases is different than non-HALDB. F. I don't know how to interpret the data in an image copy at all. I'm not aware of pre-built tools for reading unload files on other than z/OS, so you may need to write your own utilities. 5. An unload file contains the segment data (the logical data the application obtains when it reads the segment), the segment name, and some other data. There's little meta data. For example, you can't use an unload file to load a database without a DBD. The unload is in hierarchical order, but if the database is HDAM or PHDAM then the roots are in random order. The native IMS tools can't create an unload from an image copy. The extra-cost High Performance IMS tools from IBM may be able to do it. But if so, then I bet it would work like the Broadcom tool: it creates the unload from the image copy by restoring the database (from the image copy) to a database, then unloading it. That means you must have an IMS system to run it on. -Original Message- From: Jason Cai Sent: Tuesday, May 21, 2024 4:08 AM To: IBM-MAIN@LISTSERV.UA.EDU; Schmitt, Michael Cc: Jason Cai Subject: How to convert these IMS unload files from EBCDIC to UTF-8 encoding. Dear Michael and all Thank you very much for your response and the valuable information provided. Over the past few days, I discussed the DBD definitions with our IMS DBA, and we have a similar setup where “The only fields that are required to be defined in a DBD are the keys, indexed fields, and any other fields that the application wants to be able to reference in Segment Search Arguments (SSAs). In the IMS system I work on, 99% of the segment fields are not defined in the DBD.” We are considering moving away from the mainframe, and our historical data is preserved on tape in IMS unload format and IMS IC. Initially, we wanted to convert the IMS database to a DB2 and then into a MySQL database to facilitate the migration of backup data from the mainframe to linux. Now, we are only planning to transcode the IMS unload files to make them readable on the Linux platform. We have all the necessary copybooks and are currently looking to convert these IMS unload files from EBCDIC to UTF-8 encoding. Here are a few questions regarding the conversion process: 1.For fields in the copybook defined as PIC A, PIC X, and PIC 9, these require conversion, but for numeric fields like PIC 9(4) COMP-1, no conversion is needed, correct? 2 How do we handle the conversion if the database has multiple copybooks? 3 What is the process for converting variable-length fields? 4 If we choose not to convert anything, are there any tools available on Linux that can read these IMS unload files in EBCDIC encoding? 5.Does the IMS unload file include information about segments, and how are they encoded? Furthermore, we do not have access to Broadcom IMS tools or File Manager. Could you please inform me if IBM has any utilities that can create the unload format file from an image copy input? Any advice or suggestions you can provide would be greatly appreciated. Best regards， -- For IBM-MAIN subscribe / signoff / archive access ins
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
Each segment must be its own table with the combined key that includes the key of the parent in parent order. On Tue, May 21, 2024 at 6:56 AM Binyamin Dissen wrote: > > On Tue, 21 May 2024 04:08:16 -0500 Jason Cai wrote: > > :>In the IMS system I work on, 99% of the segment fields are not defined in > the DBD.” > > From my IMS recollection, that is typical. The application code has maps of > the segments. Only fields used in searches needed to be defined. > > So if you think you can ignore the segment data in the download/conversion > > > -- > Binyamin Dissen > http://www.dissensoftware.com > > Director, Dissen Software, Bar & Grill - Israel > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
On Tue, 21 May 2024 04:08:16 -0500 Jason Cai wrote: :>In the IMS system I work on, 99% of the segment fields are not defined in the DBD. >From my IMS recollection, that is typical. The application code has maps of the segments. Only fields used in searches needed to be defined. So if you think you can ignore the segment data in the download/conversion -- Binyamin Dissen http://www.dissensoftware.com Director, Dissen Software, Bar & Grill - Israel -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
Jason and all, Attila words are wise, but there is more. From your email, I'm not certain if you are talking about one data base or a collection of data bases. It's good that you have an image copy (copies) and apparently the DBDs, but there is more to it. Do you know if your data bases have compressed segments? Do you know the format of the image copy? There are many. Do you know if there are logical relationships? Even with a single data base, they can be recursive. Do you know if any of the data bases are Fast Path? Do you know if any of the data bases are PDF (BMC Software product)? These and other issues mentioned can make what you are trying to accomplish very complex especially without the right tools and skills. Tom Harper Phoenix Software International On 5/21/2024 3:57 AM, Attila Fogarasi wrote: Jason, your applications will be in for a big surprise if you follow this conversion path. For example, PIC 9(4) COMP is a binary numeric field which is BIG ENDIAN on mainframe, most (but not all) LUW and cloud systems are LITTLE ENDIAN. So you need to convert these fields, otherwise the numbers change dramatically. There are many other application specific data types that were possible -- you will have to examine all of the application programs for their field definitions and usage in order to determine the conversion rules needed to preserve application data. For example, on your question about multiple copy books this means that different parts of the application have a different schema view of the database. You will have to understand the significance of this, for example a very common past technique is to have flag bytes which indicate which of the many copy books to use for mapping the specific IMS segment. So the same data, from the IMS database viewpoint, can look very different for the application. You will need to capture this logic in the conversion process, otherwise the data will be partly corrupted. Your questions indicate you are focusing on the easy part of the problem, and have not looked at the hard part at all. If you can't convert the data, then what happens to the applications? Is there a regulatory requirement? It may be cheaper to keep the old IMS system running as a data reference site and run ad hoc application exports using custom logic. Of course that requires understanding the applications in some depth. On Tue, May 21, 2024 at 7:08 PM Jason Cai wrote: Dear Micael and all Thank you very much for your response and the valuable information provided. Over the past few days, I discussed the DBD definitions with our IMS DBA, and we have a similar setup where “The only fields that are required to be defined in a DBD are the keys, indexed fields, and any other fields that the application wants to be able to reference in Segment Search Arguments (SSAs). In the IMS system I work on, 99% of the segment fields are not defined in the DBD.” We are considering moving away from the mainframe, and our historical data is preserved on tape in IMS unload format and IMS IC. Initially, we wanted to convert the IMS database to a DB2 and then into a MySQL database to facilitate the migration of backup data from the mainframe to linux. Now, we are only planning to transcode the IMS unload files to make them readable on the Linux platform. We have all the necessary copybooks and are currently looking to convert these IMS unload files from EBCDIC to UTF-8 encoding. Here are a few questions regarding the conversion process: 1.For fields in the copybook defined as PIC A, PIC X, and PIC 9, these require conversion, but for numeric fields like PIC 9(4) COMP-1, no conversion is needed, correct? 2 How do we handle the conversion if the database has multiple copybooks? 3 What is the process for converting variable-length fields? 4 If we choose not to convert anything, are there any tools available on Linux that can read these IMS unload files in EBCDIC encoding? 5.Does the IMS unload file include information about segments, and how are they encoded? Furthermore, we do not have access to Broadcom IMS tools or File Manager. Could you please inform me if IBM has any utilities that can create the unload format file from an image copy input? Any advice or suggestions you can provide would be greatly appreciated. Best regards， -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN This e-mail message, including any attachments, appended messages and the information contained therein, is for the sole use of the inte
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
Jason, your applications will be in for a big surprise if you follow this conversion path. For example, PIC 9(4) COMP is a binary numeric field which is BIG ENDIAN on mainframe, most (but not all) LUW and cloud systems are LITTLE ENDIAN. So you need to convert these fields, otherwise the numbers change dramatically. There are many other application specific data types that were possible -- you will have to examine all of the application programs for their field definitions and usage in order to determine the conversion rules needed to preserve application data. For example, on your question about multiple copy books this means that different parts of the application have a different schema view of the database. You will have to understand the significance of this, for example a very common past technique is to have flag bytes which indicate which of the many copy books to use for mapping the specific IMS segment. So the same data, from the IMS database viewpoint, can look very different for the application. You will need to capture this logic in the conversion process, otherwise the data will be partly corrupted. Your questions indicate you are focusing on the easy part of the problem, and have not looked at the hard part at all. If you can't convert the data, then what happens to the applications? Is there a regulatory requirement? It may be cheaper to keep the old IMS system running as a data reference site and run ad hoc application exports using custom logic. Of course that requires understanding the applications in some depth. On Tue, May 21, 2024 at 7:08 PM Jason Cai wrote: > Dear Micael and all > > Thank you very much for your response and the valuable information > provided. Over the past few days, I discussed the DBD definitions with our > IMS DBA, and we have a similar setup > > where “The only fields that are required to be defined in a DBD are the > keys, indexed fields, and any other fields that the application wants to be > able to reference in Segment Search Arguments (SSAs). > > In the IMS system I work on, 99% of the segment fields are not defined in > the DBD.” > >We are considering moving away from the mainframe, and our historical > data is preserved on tape in IMS unload format and IMS IC. > > Initially, we wanted to convert the IMS database to a DB2 and then into a > MySQL database to facilitate the migration of backup data from the > mainframe to linux. > > Now, we are only planning to transcode the IMS unload files to make them > readable on the Linux platform. > > > We have all the necessary copybooks and are currently looking to convert > these IMS unload files from EBCDIC to UTF-8 encoding. > > Here are a few questions regarding the conversion process: > > 1.For fields in the copybook defined as PIC A, PIC X, and PIC 9, these > require conversion, but for numeric fields like PIC 9(4) COMP-1, no > conversion is needed, correct? > > 2 How do we handle the conversion if the database has multiple copybooks? > > 3 What is the process for converting variable-length fields? > > 4 If we choose not to convert anything, are there any tools available on > Linux that can read these IMS unload files in EBCDIC encoding? > > 5.Does the IMS unload file include information about segments, and how are > they encoded? > > Furthermore, we do not have access to Broadcom IMS tools or File > Manager. > > Could you please inform me if IBM has any utilities that can create the > unload format file from an image copy input? > > Any advice or suggestions you can provide would be greatly appreciated. > > Best regards， > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to convert these IMS unload files from EBCDIC to UTF-8 encoding.
json, You can process the unload file if you know the record format. This, the record format, depends on the utility used to perform the unload. Once you have it, map the segment name to segment layout from the copybook. If you have easyTrieve installed, you can convert the copybooks to offset based with the correct length. This will help you process the file using languages such as rexx and easytrieve. As per the multiple layouts of the ims segment, use the logic in the program to determine the layout field dependency. Each copybook must be converted to a different table. I would convert the file to a sql database insert calls using rexx or easytrieve anit to load the sql server tables directly without the need to go via Db2 first. HTH ITschak Mugzach *|** IronSphere Platform* *|* *Information Security Continuous Monitoring for z/OS, x/Linux & IBM I **| z/VM coming soon * On Tue, May 21, 2024 at 12:08 PM Jason Cai wrote: > Dear Micael and all > > Thank you very much for your response and the valuable information > provided. Over the past few days, I discussed the DBD definitions with our > IMS DBA, and we have a similar setup > > where “The only fields that are required to be defined in a DBD are the > keys, indexed fields, and any other fields that the application wants to be > able to reference in Segment Search Arguments (SSAs). > > In the IMS system I work on, 99% of the segment fields are not defined in > the DBD.” > >We are considering moving away from the mainframe, and our historical > data is preserved on tape in IMS unload format and IMS IC. > > Initially, we wanted to convert the IMS database to a DB2 and then into a > MySQL database to facilitate the migration of backup data from the > mainframe to linux. > > Now, we are only planning to transcode the IMS unload files to make them > readable on the Linux platform. > > > We have all the necessary copybooks and are currently looking to convert > these IMS unload files from EBCDIC to UTF-8 encoding. > > Here are a few questions regarding the conversion process: > > 1.For fields in the copybook defined as PIC A, PIC X, and PIC 9, these > require conversion, but for numeric fields like PIC 9(4) COMP-1, no > conversion is needed, correct? > > 2 How do we handle the conversion if the database has multiple copybooks? > > 3 What is the process for converting variable-length fields? > > 4 If we choose not to convert anything, are there any tools available on > Linux that can read these IMS unload files in EBCDIC encoding? > > 5.Does the IMS unload file include information about segments, and how are > they encoded? > > Furthermore, we do not have access to Broadcom IMS tools or File > Manager. > > Could you please inform me if IBM has any utilities that can create the > unload format file from an image copy input? > > Any advice or suggestions you can provide would be greatly appreciated. > > Best regards， > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
