Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Actually, no. Binary trees are something quite different (and generally more familiar). === Gregory Woodhouse [EMAIL PROTECTED] "Nothing is as powerful than an idea whose time has come." -- Victor Hugo On Aug 23, 2005, at 6:05 PM, Nancy Anthracite wrote: B, I believe, is for Binary. On Tuesday 23 August 2005 06:11 pm, Gregory Woodhouse wrote: I think Kevin was asking whether or not strings are null terminated. I know nothing about the GT.M source, but as a general sort answer: Databases don't typically store data in a "packed" format (like the run-time heap), but instead storage is allocated in fixed size chunks, which are then typically organized into a structure called a B-tree. This makes it possible to add and delete records (nodes) or to modify the size of an existing node without having to drastically modify the entire structure. (So far as I know, the origin of the term B-tree is unknown, but I like to think of them as "bushy" trees.) --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Nancy; Originally it was "binary", two branches at each branch point. The performance sucked. Then there was the discovery of the multi-way B-tree. The speed is much faster. Roughly, the branching is 50 to 1 as opposed to 2 to 1 for binary trees. Also the data layer with the multi-way b-tree is easier to walk sequentially than having to do a tree climb. Chris . - Original Message - From: "Nancy Anthracite" <[EMAIL PROTECTED]> To: Sent: Tuesday, August 23, 2005 6:05 PM Subject: Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? > B, I believe, is for Binary. > > On Tuesday 23 August 2005 06:11 pm, Gregory Woodhouse wrote: > > I think Kevin was asking whether or not strings are null terminated. > > I know nothing about the GT.M source, but as a general sort answer: > > Databases don't typically store data in a "packed" format (like the > > run-time heap), but instead storage is allocated in fixed size > > chunks, which are then typically organized into a structure called a > > B-tree. This makes it possible to add and delete records (nodes) or > > to modify the size of an existing node without having to drastically > > modify the entire structure. (So far as I know, the origin of the > > term B-tree is unknown, but I like to think of them as "bushy" trees.) > > > > === > > Gregory Woodhouse > > [EMAIL PROTECTED] > > > > "Design quality doesn't ensure success, but design failure can ensure > > failure." > > > > --Kent Beck > > > > On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: > > > Probably not of much value to ask unless you are a GT.M internals > > > developer - details are in the source code. As a gross simplification > > > (along the lines of saying that living things are made up of cells), > > > GT.M stores the length and actual value of each string. But there are > > > all sorts of optimizations, including key compression when stored > > > in the > > > database. > > > > > > -- Bhaskar > > > > > > On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > > >> So Bhaskar, is of any value to ask how the data is stored behind the > > >> scenes? I was worried that the strings were null-terminated etc and > > >> that there might be some binary data that would crash GT.M. when > > >> storing is in a global value. > > >> > > >> I'm glad to hear that is not the case. > > >> > > >> Kevin > > > > --- > > SF.Net email is Sponsored by the Better Software Conference & EXPO > > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > > ___ > > Hardhats-members mailing list > > Hardhats-members@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > -- > Nancy Anthracite > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
B, I believe, is for Binary. On Tuesday 23 August 2005 06:11 pm, Gregory Woodhouse wrote: > I think Kevin was asking whether or not strings are null terminated. > I know nothing about the GT.M source, but as a general sort answer: > Databases don't typically store data in a "packed" format (like the > run-time heap), but instead storage is allocated in fixed size > chunks, which are then typically organized into a structure called a > B-tree. This makes it possible to add and delete records (nodes) or > to modify the size of an existing node without having to drastically > modify the entire structure. (So far as I know, the origin of the > term B-tree is unknown, but I like to think of them as "bushy" trees.) > > === > Gregory Woodhouse > [EMAIL PROTECTED] > > "Design quality doesn't ensure success, but design failure can ensure > failure." > > --Kent Beck > > On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: > > Probably not of much value to ask unless you are a GT.M internals > > developer - details are in the source code. As a gross simplification > > (along the lines of saying that living things are made up of cells), > > GT.M stores the length and actual value of each string. But there are > > all sorts of optimizations, including key compression when stored > > in the > > database. > > > > -- Bhaskar > > > > On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > >> So Bhaskar, is of any value to ask how the data is stored behind the > >> scenes? I was worried that the strings were null-terminated etc and > >> that there might be some binary data that would crash GT.M. when > >> storing is in a global value. > >> > >> I'm glad to hear that is not the case. > >> > >> Kevin > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members -- Nancy Anthracite --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
On Sun, 2005-08-21 at 14:19 -0500, Kevin Toppenberg wrote: [KSB] <...snip...> > I'll check with Bhaskar whether or not there is any risk in storing > non-printable characters in a global. I did some testing and filled > a > global with $char(0) and it didn't seem to loose them, or cause the > underlying string to terminate (i.e. null-terminated strings). [KSB] With GT.M, the subscripts as well as the values stored in an M node can be any arbitrary binary sequence of bytes. -- Bhaskar --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
RE: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Surprise for me. I thought Donald Knuth the master of the b-tree. Also thought "b" meant branching. Now I don't know. ../t > -Original Message- > From: [EMAIL PROTECTED] [mailto:hardhats- > [EMAIL PROTECTED] On Behalf Of Kevin Toppenberg > Sent: Tuesday, August 23, 2005 4:25 PM > To: hardhats-members@lists.sourceforge.net > Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? > > OK. Thanks. I had always thought that B-Tree ment "Binary tree". > But quick search turned on wikipedia turned up this: > > http://en.wikipedia.org/wiki/B-tree > > The B-tree's creator, Rudolf Bayer, has not explained what the B > stands for. The most common belief is that B stands for balanced, as > all the leaf nodes are at the same level in the tree. B may also stand > for Bayer, or for Boeing, because he was working for Boeing Scientific > Research Labs. > > Kevin > > > On 8/23/05, Gregory Woodhouse <[EMAIL PROTECTED]> wrote: > > I think Kevin was asking whether or not strings are null terminated. > > I know nothing about the GT.M source, but as a general sort answer: > > Databases don't typically store data in a "packed" format (like the > > run-time heap), but instead storage is allocated in fixed size > > chunks, which are then typically organized into a structure called a > > B-tree. This makes it possible to add and delete records (nodes) or > > to modify the size of an existing node without having to drastically > > modify the entire structure. (So far as I know, the origin of the > > term B-tree is unknown, but I like to think of them as "bushy" trees.) > > > > === > > Gregory Woodhouse > > [EMAIL PROTECTED] > > > > "Design quality doesn't ensure success, but design failure can ensure > > failure." > > > > --Kent Beck > > > > On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: > > > > > Probably not of much value to ask unless you are a GT.M internals > > > developer - details are in the source code. As a gross simplification > > > (along the lines of saying that living things are made up of cells), > > > GT.M stores the length and actual value of each string. But there are > > > all sorts of optimizations, including key compression when stored > > > in the > > > database. > > > > > > -- Bhaskar > > > > > > On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > > > > > >> So Bhaskar, is of any value to ask how the data is stored behind the > > >> scenes? I was worried that the strings were null-terminated etc and > > >> that there might be some binary data that would crash GT.M. when > > >> storing is in a global value. > > >> > > >> I'm glad to hear that is not the case. > > >> > > >> Kevin > > > > > > > > > > > --- > > SF.Net email is Sponsored by the Better Software Conference & EXPO > > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & > QA > > Security * Process Improvement & Measurement * > http://www.sqe.com/bsce5sf > > ___ > > Hardhats-members mailing list > > Hardhats-members@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Sorry for bad info. That is what I thought my husband told me it meant, but I just looked it up in his "Algorithms" book by Sedgewick, and it says something very similar to what the wikipedia had to say. On Tuesday 23 August 2005 06:25 pm, Kevin Toppenberg wrote: > OK. Thanks. I had always thought that B-Tree ment "Binary tree". > But quick search turned on wikipedia turned up this: > > http://en.wikipedia.org/wiki/B-tree > > The B-tree's creator, Rudolf Bayer, has not explained what the B > stands for. The most common belief is that B stands for balanced, as > all the leaf nodes are at the same level in the tree. B may also stand > for Bayer, or for Boeing, because he was working for Boeing Scientific > Research Labs. > > Kevin > > On 8/23/05, Gregory Woodhouse <[EMAIL PROTECTED]> wrote: > > I think Kevin was asking whether or not strings are null terminated. > > I know nothing about the GT.M source, but as a general sort answer: > > Databases don't typically store data in a "packed" format (like the > > run-time heap), but instead storage is allocated in fixed size > > chunks, which are then typically organized into a structure called a > > B-tree. This makes it possible to add and delete records (nodes) or > > to modify the size of an existing node without having to drastically > > modify the entire structure. (So far as I know, the origin of the > > term B-tree is unknown, but I like to think of them as "bushy" trees.) > > > > === > > Gregory Woodhouse > > [EMAIL PROTECTED] > > > > "Design quality doesn't ensure success, but design failure can ensure > > failure." > > > > --Kent Beck > > > > On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: > > > Probably not of much value to ask unless you are a GT.M internals > > > developer - details are in the source code. As a gross simplification > > > (along the lines of saying that living things are made up of cells), > > > GT.M stores the length and actual value of each string. But there are > > > all sorts of optimizations, including key compression when stored > > > in the > > > database. > > > > > > -- Bhaskar > > > > > > On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > > >> So Bhaskar, is of any value to ask how the data is stored behind the > > >> scenes? I was worried that the strings were null-terminated etc and > > >> that there might be some binary data that would crash GT.M. when > > >> storing is in a global value. > > >> > > >> I'm glad to hear that is not the case. > > >> > > >> Kevin > > > > --- > > SF.Net email is Sponsored by the Better Software Conference & EXPO > > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > > Practices Agile & Plan-Driven Development * Managing Projects & Teams * > > Testing & QA Security * Process Improvement & Measurement * > > http://www.sqe.com/bsce5sf > > ___ > > Hardhats-members mailing list > > Hardhats-members@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members -- Nancy Anthracite --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
OK. Thanks. I had always thought that B-Tree ment "Binary tree". But quick search turned on wikipedia turned up this: http://en.wikipedia.org/wiki/B-tree The B-tree's creator, Rudolf Bayer, has not explained what the B stands for. The most common belief is that B stands for balanced, as all the leaf nodes are at the same level in the tree. B may also stand for Bayer, or for Boeing, because he was working for Boeing Scientific Research Labs. Kevin On 8/23/05, Gregory Woodhouse <[EMAIL PROTECTED]> wrote: > I think Kevin was asking whether or not strings are null terminated. > I know nothing about the GT.M source, but as a general sort answer: > Databases don't typically store data in a "packed" format (like the > run-time heap), but instead storage is allocated in fixed size > chunks, which are then typically organized into a structure called a > B-tree. This makes it possible to add and delete records (nodes) or > to modify the size of an existing node without having to drastically > modify the entire structure. (So far as I know, the origin of the > term B-tree is unknown, but I like to think of them as "bushy" trees.) > > === > Gregory Woodhouse > [EMAIL PROTECTED] > > "Design quality doesn't ensure success, but design failure can ensure > failure." > > --Kent Beck > > On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: > > > Probably not of much value to ask unless you are a GT.M internals > > developer - details are in the source code. As a gross simplification > > (along the lines of saying that living things are made up of cells), > > GT.M stores the length and actual value of each string. But there are > > all sorts of optimizations, including key compression when stored > > in the > > database. > > > > -- Bhaskar > > > > On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > > > >> So Bhaskar, is of any value to ask how the data is stored behind the > >> scenes? I was worried that the strings were null-terminated etc and > >> that there might be some binary data that would crash GT.M. when > >> storing is in a global value. > >> > >> I'm glad to hear that is not the case. > >> > >> Kevin > > > > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members > --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
I think Kevin was asking whether or not strings are null terminated. I know nothing about the GT.M source, but as a general sort answer: Databases don't typically store data in a "packed" format (like the run-time heap), but instead storage is allocated in fixed size chunks, which are then typically organized into a structure called a B-tree. This makes it possible to add and delete records (nodes) or to modify the size of an existing node without having to drastically modify the entire structure. (So far as I know, the origin of the term B-tree is unknown, but I like to think of them as "bushy" trees.) === Gregory Woodhouse [EMAIL PROTECTED] "Design quality doesn't ensure success, but design failure can ensure failure." --Kent Beck On Aug 23, 2005, at 2:54 PM, K.S. Bhaskar wrote: Probably not of much value to ask unless you are a GT.M internals developer - details are in the source code. As a gross simplification (along the lines of saying that living things are made up of cells), GT.M stores the length and actual value of each string. But there are all sorts of optimizations, including key compression when stored in the database. -- Bhaskar On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: So Bhaskar, is of any value to ask how the data is stored behind the scenes? I was worried that the strings were null-terminated etc and that there might be some binary data that would crash GT.M. when storing is in a global value. I'm glad to hear that is not the case. Kevin --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
RE: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Sowinski, Richard J. wrote: >What I see happen when you have control characters in Globals is that when >you try to display the global, it reeks havoc with your screen. This is an issue with your display device and user interface (or lack of one) when working with roll-and-scroll on dumb terminals. We addressed this issue in VMACS over 20 years ago by taking over display management and using tools for development that render the control codes displayable. Unfortunately, it is not easy to adopt such a solution in small steps... There is a nearly identical problem in working with web applications where data may contain angle brackets and other character that are like control characters in HTML. In order to maintain the integrity of dynamically generated web pages, you have to use tools that filter out such characters when they would break the display or take it out of the intended control of your applications. M2Web includes some powerful tools for handling this sort of thing. >I have seen M applications written specifically with control characters in >the globals to be executed later. A highly efficient technique for generating and working with forms on dumb terminals is to compile all the fixed parts into strings with the controls embedded so that at run time you simply write the strings to paint the forms. All graphic renditions such as bold, reverse, colors, etc and cursor positioning etc are taken care of ahead of time so it's very simple and fast when interacting with the users. --- Jim Self Systems Architect, Lead Developer VMTH Computer Services, UC Davis (http://www.vmth.ucdavis.edu/us/jaself) --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Probably not of much value to ask unless you are a GT.M internals developer - details are in the source code. As a gross simplification (along the lines of saying that living things are made up of cells), GT.M stores the length and actual value of each string. But there are all sorts of optimizations, including key compression when stored in the database. -- Bhaskar On Tue, 2005-08-23 at 16:22 -0500, Kevin Toppenberg wrote: > So Bhaskar, is of any value to ask how the data is stored behind the > scenes? I was worried that the strings were null-terminated etc and > that there might be some binary data that would crash GT.M. when > storing is in a global value. > > I'm glad to hear that is not the case. > > Kevin --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
So Bhaskar, is of any value to ask how the data is stored behind the scenes? I was worried that the strings were null-terminated etc and that there might be some binary data that would crash GT.M. when storing is in a global value. I'm glad to hear that is not the case. Kevin On 8/23/05, K.S. Bhaskar <[EMAIL PROTECTED]> wrote: > Rich -- > > There are two issues here: (a) what GT.M does and does not allow, and > (b) the application logic, which includes the execution settings. > > GT.M allows you to use any arbitrary sequence of bytes both as an index > and as a value. > > Whether a specific piece of application code should store and display an > arbitrary sequence of bytes is an orthogonal issue - e.g., many Eastern > European character sets use all 8 bits of a byte. So, for example, you > may well have a global with strings to be used to display when Polish is > selected as a language, and they may well cause display problems if you > try to display them on a terminal emulator set for US ASCII > > -- Bhaskar > > On Tue, 2005-08-23 at 14:55 -0500, Sowinski, Richard J. wrote: > > What I see happen when you have control characters in Globals is that > > when > > you try to display the global, it reeks havoc with your screen. > > > > I have seen M applications written specifically with control > > characters in > > the globals to be executed later. > > > > -Original Message- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of > > K.S. > > Bhaskar > > Sent: Tuesday, August 23, 2005 2:39 PM > > To: hardhats-members@lists.sourceforge.net > > Subject: Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? > > > > On Sun, 2005-08-21 at 14:19 -0500, Kevin Toppenberg wrote: > > > > [KSB] <...snip...> > > > > > I'll check with Bhaskar whether or not there is any risk in > > storing > > > non-printable characters in a global. I did some testing and > > filled > > > a > > > global with $char(0) and it didn't seem to loose them, or cause > > the > > > underlying string to terminate (i.e. null-terminated strings). > > > > [KSB] With GT.M, the subscripts as well as the values stored in an M > > node can be any arbitrary binary sequence of bytes. > > > > -- Bhaskar > > > > > > > > --- > > SF.Net email is Sponsored by the Better Software Conference & EXPO > > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > > Practices > > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > > & QA > > Security * Process Improvement & Measurement * > > http://www.sqe.com/bsce5sf > > ___ > > Hardhats-members mailing list > > Hardhats-members@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > > > > > --- > > SF.Net email is Sponsored by the Better Software Conference & EXPO > > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > > Practices > > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > > & QA > > Security * Process Improvement & Measurement * > > http://www.sqe.com/bsce5sf > > ___ > > Hardhats-members mailing list > > Hardhats-members@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > > > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members > --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
RE: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Rich -- There are two issues here: (a) what GT.M does and does not allow, and (b) the application logic, which includes the execution settings. GT.M allows you to use any arbitrary sequence of bytes both as an index and as a value. Whether a specific piece of application code should store and display an arbitrary sequence of bytes is an orthogonal issue - e.g., many Eastern European character sets use all 8 bits of a byte. So, for example, you may well have a global with strings to be used to display when Polish is selected as a language, and they may well cause display problems if you try to display them on a terminal emulator set for US ASCII -- Bhaskar On Tue, 2005-08-23 at 14:55 -0500, Sowinski, Richard J. wrote: > What I see happen when you have control characters in Globals is that > when > you try to display the global, it reeks havoc with your screen. > > I have seen M applications written specifically with control > characters in > the globals to be executed later. > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > K.S. > Bhaskar > Sent: Tuesday, August 23, 2005 2:39 PM > To: hardhats-members@lists.sourceforge.net > Subject: Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? > > On Sun, 2005-08-21 at 14:19 -0500, Kevin Toppenberg wrote: > > [KSB] <...snip...> > > > I'll check with Bhaskar whether or not there is any risk in > storing > > non-printable characters in a global. I did some testing and > filled > > a > > global with $char(0) and it didn't seem to loose them, or cause > the > > underlying string to terminate (i.e. null-terminated strings). > > [KSB] With GT.M, the subscripts as well as the values stored in an M > node can be any arbitrary binary sequence of bytes. > > -- Bhaskar > > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > & QA > Security * Process Improvement & Measurement * > http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members > > > --- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > & QA > Security * Process Improvement & Measurement * > http://www.sqe.com/bsce5sf > ___ > Hardhats-members mailing list > Hardhats-members@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/hardhats-members > --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
RE: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
What I see happen when you have control characters in Globals is that when you try to display the global, it reeks havoc with your screen. I have seen M applications written specifically with control characters in the globals to be executed later. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of K.S. Bhaskar Sent: Tuesday, August 23, 2005 2:39 PM To: hardhats-members@lists.sourceforge.net Subject: Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? On Sun, 2005-08-21 at 14:19 -0500, Kevin Toppenberg wrote: [KSB] <...snip...> > I'll check with Bhaskar whether or not there is any risk in storing > non-printable characters in a global. I did some testing and filled > a > global with $char(0) and it didn't seem to loose them, or cause the > underlying string to terminate (i.e. null-terminated strings). [KSB] With GT.M, the subscripts as well as the values stored in an M node can be any arbitrary binary sequence of bytes. -- Bhaskar --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
On Sun, 2005-08-21 at 14:19 -0500, Kevin Toppenberg wrote: [KSB] <...snip...> > I'll check with Bhaskar whether or not there is any risk in storing > non-printable characters in a global. I did some testing and filled > a > global with $char(0) and it didn't seem to loose them, or cause the > underlying string to terminate (i.e. null-terminated strings). [KSB] With GT.M, the subscripts as well as the values stored in an M node can be any arbitrary binary sequence of bytes. -- Bhaskar --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Hardhats-members mailing list Hardhats-members@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hardhats-members
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Kevin;
I'm not sure which MUMPS engine you arte running, but I believe you have
to turn off IO termination on control characters. Looks like the fixed
reads are being terminated early.
Chris
- Original Message -
From: "Kevin Toppenberg" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Sunday, August 21, 2005 11:22 AM
Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Chris,
I am having trouble getting a buffered read to work.
GTM>set path="/tmp/"
GTM>set fname="killthis"
GTM>do OPEN^%ZISH("Handle",path,fname,"R")
GTM>w POP
0
GTM>F Q:($ZEOF) U IO:(NOTERMINATOR) R X#255:5 U $P W $L(X),"; "
51; 17; 21; 24; 51; 52; 56; 27; 51; 38; 0;
%GTM-E-IOEOF, Attempt to read past an end-of-file
Don't worry about that EOF error. The issue is that I am not getting
in 255 bytes at a time as I requested. In this case the file is a
text file, but it should be amendible for binary processing. I think
that (NOTERMINATOR) is not working.
Can you show how you would do a buffered read?
Thanks
Kevin
On 8/21/05, Chris Richardson <[EMAIL PROTECTED]> wrote:
> Kevin;
>
>You are working way too hard. Do a buffered read and then use $ASCII
to
> take it apart one octet at a time. The encoding is much easier than you
> have made it. Each octet will be something like this;
>
> N BUF,C,B1,B2,OBUF
> S OBUF=""
> R BUF#255
> F I=1:1:$L(BUF) D
> . S C=$ASCII(BUF,I) ; Converts to the value of the character (0 to
255)
> . S OBUF=OBUF_$$BYT2BIN(C)
> .QUIT
>
> BYT2BIN(V) ; Take one BYTE and return HEX Values
> N HV,B1,B2
> S NV="0123456789ABCDEF"
> S B1=(V#16)+1 ; 0 to 15 becomes 1 to 16
> S B2=(V\16)+1
> QUIT $E(NV,B1)_$E(NV,B2) ; You figure out the byte order 1-2 or 2-1
>
> The star reads are eating your lunch. This will be much faster.
>
>
> ----- Original Message -
> From: "Kevin Toppenberg" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Sunday, August 21, 2005 5:53 AM
> Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
>
>
> Here is the code. I will also attach it incase wrapping ruins it here...
>
>
> ;"TMG BIN <-->GBL FUNCTION
> ;"Kevin Toppenberg MD
> ;"GNU General Public License (GPL) applies
> ;"8-20-2005
>
> ;"===
> ;" API -- Public Functions.
> ;"===
> ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr)
> ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename)
>
> ;"===
> ;"PRIVATE API FUNCTIONS
> ;"===
>
>
>
> ;"===
> BIN2GBL(path,filename,globalRef,incSubscr)
> ;"Purpose: To load a binary file from the host filesystem into
> a WP field, storing
> ;" the composit bytes as ascii hex codes.
> ;"Input: path --full path, up to but not including the
> filename (required)
> ;" filename -- name of the file to open (required)
> ;" globalRef-- Global reference to WRITE the host
> binary file to, in fully resolved
> ;" (closed root) format. This
> function does not kill the global before
> ;" writing to it. (required)
> ;" Note:
> ;" At least one subscript must be
> numeric. This will be the incrementing
> ;" subscript (i.e. the subscript
> that $$BIN2WP^TMGBINWP will increment
> ;" to store each new global node).
> This subscript need not be the final
> ;" subscript. For example, to load
> into a WORD PROCESSING field, the
> ;" incrementing node is the
> second-to-last subscript; the final subscript
> ;" is always zero.
> ;"incSubscr-- (required) Identifies the incrementing
> subscript level. For example, if you
> ;" pass ^TMP(115,1,1,0) as the
> global_ref parameter and pass 3 as the
> ;" inc_subscr parameter, $$BIN2GBL
> will increment the third subscript,
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Yes, it is going to be interesting to see what happens if many offices adopt VistAOffice, and don't have the support that is standard in a hospital setting. I understand your feeling of staying well away from issues that caused problems in the past. I'll check with Bhaskar whether or not there is any risk in storing non-printable characters in a global. I did some testing and filled a global with $char(0) and it didn't seem to loose them, or cause the underlying string to terminate (i.e. null-terminated strings). Actually I would consider it a bug of the M implementation if a user could put a character into a global that would cause problems on retrieval etc. I don't know about this issue in subscripts. I suspect that it would be difficult to figure out what character had been put in there, and thus be difficult to get back out. But you could $order through it to get the hidden subscript name and then delete the node. But then things don't always work like they are supposed to, do they? Thanks for your feedback. Kevin On 8/21/05, smcphelan <[EMAIL PROTECTED]> wrote: > Non-printable ASCII characters in either the data or the global subscripts > can and has caused problem depending upon the M implementation and the > vehicle you are using to access the M account. If your data is on a global > node that only your specific application will touch, then you may get by > with it. But if you have to use any of the M vendor's tools, you may > encounter difficulty. I have had direct experience in both cases where > non-printable control codes were in the subscripts and data. I had a DSM > system once that was down for almost a day because of one control character > in one subscript in one global. Even DSM's FIX utility could not fix it. > We had to have DEC dial in and fix it. Of course that was years ago and I > am sure things are better now. But I, for one, am not interested in > anything which intentionally stores non-printable ASCII characters. There > are a few exceptions. I have never had show stopper problems with ASCII > 7,9,10,11, and 13 as well as some others. > > Without having a DEC support contract, I do not know what we would have > done. I guess we could have paid hourly rates to DEC. You go through a > few > experiences like this actually experiencing major impact upon your facility > providing patient care and possibly jeopardizing a patient's well being, > you > will never want to be in the situation where you do not have some sort of > maintenance support immediately available. Small doctor offices maybe can. > But I cannot imagine any hospital going without all the support contracts > in > place. > > - Original Message ----- > From: "Kevin Toppenberg" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Sunday, August 21, 2005 9:05 AM > Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? > > > A more fundamental question is whether M globals are binary friendly? > I know that the underlying data is stored as strings. So would the > storage of a $char(0) crash the data? > > I just tested this, and it doesn't. Thus I could store the actual > binary data in the global instead of converting each byte to a hex > equivalent and storing that. > > I could therefore read in 256 bytes at a time and store the data > directly into the global. It would be much faster. > > But since my goal is to use the RPCBroker functionality that can > transfer globals, I will next need to investigate what would happen if > the global contained a control character (i.e. bytes 0-31). > > I think that when a WP field is transferred to Delphi via RPCBroker, > it is put into a TMemo field. Each of the lines in a TMemo is a > zero-terminated string (I think), so binary data might run into > problems there. > > I'll have to think more about this, but any input would be appreciated. > > Kevin > > > On 8/21/05, Kevin Toppenberg <[EMAIL PROTECTED]> wrote: > > Here is the code. I will also attach it incase wrapping ruins it here... > > > > > > ;"TMG BIN <-->GBL FUNCTION > > ;"Kevin Toppenberg MD > > ;"GNU General Public License (GPL) applies > > ;"8-20-2005 > > > > ;"=== > > ;" API -- Public Functions. > > ;"=== > > ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr) > > ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename) > > > > ;"=== > > ;"PRIVATE
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Non-printable ASCII characters in either the data or the global subscripts can and has caused problem depending upon the M implementation and the vehicle you are using to access the M account. If your data is on a global node that only your specific application will touch, then you may get by with it. But if you have to use any of the M vendor's tools, you may encounter difficulty. I have had direct experience in both cases where non-printable control codes were in the subscripts and data. I had a DSM system once that was down for almost a day because of one control character in one subscript in one global. Even DSM's FIX utility could not fix it. We had to have DEC dial in and fix it. Of course that was years ago and I am sure things are better now. But I, for one, am not interested in anything which intentionally stores non-printable ASCII characters. There are a few exceptions. I have never had show stopper problems with ASCII 7,9,10,11, and 13 as well as some others. Without having a DEC support contract, I do not know what we would have done. I guess we could have paid hourly rates to DEC. You go through a few experiences like this actually experiencing major impact upon your facility providing patient care and possibly jeopardizing a patient's well being, you will never want to be in the situation where you do not have some sort of maintenance support immediately available. Small doctor offices maybe can. But I cannot imagine any hospital going without all the support contracts in place. - Original Message - From: "Kevin Toppenberg" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Sunday, August 21, 2005 9:05 AM Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? A more fundamental question is whether M globals are binary friendly? I know that the underlying data is stored as strings. So would the storage of a $char(0) crash the data? I just tested this, and it doesn't. Thus I could store the actual binary data in the global instead of converting each byte to a hex equivalent and storing that. I could therefore read in 256 bytes at a time and store the data directly into the global. It would be much faster. But since my goal is to use the RPCBroker functionality that can transfer globals, I will next need to investigate what would happen if the global contained a control character (i.e. bytes 0-31). I think that when a WP field is transferred to Delphi via RPCBroker, it is put into a TMemo field. Each of the lines in a TMemo is a zero-terminated string (I think), so binary data might run into problems there. I'll have to think more about this, but any input would be appreciated. Kevin On 8/21/05, Kevin Toppenberg <[EMAIL PROTECTED]> wrote: > Here is the code. I will also attach it incase wrapping ruins it here... > > > ;"TMG BIN <-->GBL FUNCTION > ;"Kevin Toppenberg MD > ;"GNU General Public License (GPL) applies > ;"8-20-2005 > > ;"=== > ;" API -- Public Functions. > ;"=== > ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr) > ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename) > > ;"=== > ;"PRIVATE API FUNCTIONS > ;"=== > > > > ;"=== > BIN2GBL(path,filename,globalRef,incSubscr) > ;"Purpose: To load a binary file from the host filesystem into > a WP field, storing > ;" the composit bytes as ascii hex codes. > ;"Input: path --full path, up to but not including the > filename (required) > ;" filename -- name of the file to open (required) > ;" globalRef-- Global reference to WRITE the host > binary file to, in fully resolved > ;" (closed root) format. This > function does not kill the global before > ;" writing to it. (required) > ;" Note: > ;" At least one subscript must be > numeric. This will be the incrementing > ;" subscript (i.e. the subscript > that $$BIN2WP^TMGBINWP will increment > ;" to store each new global node). > This subscript need not be the final > ;" subscript. For example, to load > into a WORD PROCESSING field, the > ;"
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Chris,
I am having trouble getting a buffered read to work.
GTM>set path="/tmp/"
GTM>set fname="killthis"
GTM>do OPEN^%ZISH("Handle",path,fname,"R")
GTM>w POP
0
GTM>F Q:($ZEOF) U IO:(NOTERMINATOR) R X#255:5 U $P W $L(X),"; "
51; 17; 21; 24; 51; 52; 56; 27; 51; 38; 0;
%GTM-E-IOEOF, Attempt to read past an end-of-file
Don't worry about that EOF error. The issue is that I am not getting
in 255 bytes at a time as I requested. In this case the file is a
text file, but it should be amendible for binary processing. I think
that (NOTERMINATOR) is not working.
Can you show how you would do a buffered read?
Thanks
Kevin
On 8/21/05, Chris Richardson <[EMAIL PROTECTED]> wrote:
> Kevin;
>
>You are working way too hard. Do a buffered read and then use $ASCII to
> take it apart one octet at a time. The encoding is much easier than you
> have made it. Each octet will be something like this;
>
> N BUF,C,B1,B2,OBUF
> S OBUF=""
> R BUF#255
> F I=1:1:$L(BUF) D
> . S C=$ASCII(BUF,I) ; Converts to the value of the character (0 to 255)
> . S OBUF=OBUF_$$BYT2BIN(C)
> .QUIT
>
> BYT2BIN(V) ; Take one BYTE and return HEX Values
> N HV,B1,B2
> S NV="0123456789ABCDEF"
> S B1=(V#16)+1 ; 0 to 15 becomes 1 to 16
> S B2=(V\16)+1
> QUIT $E(NV,B1)_$E(NV,B2) ; You figure out the byte order 1-2 or 2-1
>
> The star reads are eating your lunch. This will be much faster.
>
>
> ----- Original Message -
> From: "Kevin Toppenberg" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Sunday, August 21, 2005 5:53 AM
> Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
>
>
> Here is the code. I will also attach it incase wrapping ruins it here...
>
>
> ;"TMG BIN <-->GBL FUNCTION
> ;"Kevin Toppenberg MD
> ;"GNU General Public License (GPL) applies
> ;"8-20-2005
>
> ;"===
> ;" API -- Public Functions.
> ;"===
> ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr)
> ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename)
>
> ;"===
> ;"PRIVATE API FUNCTIONS
> ;"===
>
>
>
> ;"===
> BIN2GBL(path,filename,globalRef,incSubscr)
> ;"Purpose: To load a binary file from the host filesystem into
> a WP field, storing
> ;" the composit bytes as ascii hex codes.
> ;"Input: path --full path, up to but not including the
> filename (required)
> ;" filename -- name of the file to open (required)
> ;" globalRef-- Global reference to WRITE the host
> binary file to, in fully resolved
> ;" (closed root) format. This
> function does not kill the global before
> ;" writing to it. (required)
> ;" Note:
> ;" At least one subscript must be
> numeric. This will be the incrementing
> ;" subscript (i.e. the subscript
> that $$BIN2WP^TMGBINWP will increment
> ;" to store each new global node).
> This subscript need not be the final
> ;" subscript. For example, to load
> into a WORD PROCESSING field, the
> ;" incrementing node is the
> second-to-last subscript; the final subscript
> ;" is always zero.
> ;"incSubscr-- (required) Identifies the incrementing
> subscript level. For example, if you
> ;" pass ^TMP(115,1,1,0) as the
> global_ref parameter and pass 3 as the
> ;" inc_subscr parameter, $$BIN2GBL
> will increment the third subscript, such
> ;" as ^TMP(115,1,x), but will WRITE
> notes at the full global reference, such
> ;" as ^TMP(115,1,x,0).
> ;"Result: 1=success, 0=failure
> ;"
> ;"Note: Each line of the global will contain up to 128 bytes
> (256 characters)
> ;" (2 ascii hex characters = 1 source byte)
>
Re: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Kevin; You are working way too hard. Do a buffered read and then use $ASCII to take it apart one octet at a time. The encoding is much easier than you have made it. Each octet will be something like this; N BUF,C,B1,B2,OBUF S OBUF="" R BUF#255 F I=1:1:$L(BUF) D . S C=$ASCII(BUF,I) ; Converts to the value of the character (0 to 255) . S OBUF=OBUF_$$BYT2BIN(C) .QUIT BYT2BIN(V) ; Take one BYTE and return HEX Values N HV,B1,B2 S NV="0123456789ABCDEF" S B1=(V#16)+1 ; 0 to 15 becomes 1 to 16 S B2=(V\16)+1 QUIT $E(NV,B1)_$E(NV,B2) ; You figure out the byte order 1-2 or 2-1 The star reads are eating your lunch. This will be much faster. - Original Message - From: "Kevin Toppenberg" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Sunday, August 21, 2005 5:53 AM Subject: [Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly? Here is the code. I will also attach it incase wrapping ruins it here... ;"TMG BIN <-->GBL FUNCTION ;"Kevin Toppenberg MD ;"GNU General Public License (GPL) applies ;"8-20-2005 ;"=== ;" API -- Public Functions. ;"=== ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr) ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename) ;"=== ;"PRIVATE API FUNCTIONS ;"=== ;"=== BIN2GBL(path,filename,globalRef,incSubscr) ;"Purpose: To load a binary file from the host filesystem into a WP field, storing ;" the composit bytes as ascii hex codes. ;"Input: path --full path, up to but not including the filename (required) ;" filename -- name of the file to open (required) ;" globalRef-- Global reference to WRITE the host binary file to, in fully resolved ;" (closed root) format. This function does not kill the global before ;" writing to it. (required) ;" Note: ;" At least one subscript must be numeric. This will be the incrementing ;" subscript (i.e. the subscript that $$BIN2WP^TMGBINWP will increment ;" to store each new global node). This subscript need not be the final ;" subscript. For example, to load into a WORD PROCESSING field, the ;" incrementing node is the second-to-last subscript; the final subscript ;" is always zero. ;"incSubscr-- (required) Identifies the incrementing subscript level. For example, if you ;" pass ^TMP(115,1,1,0) as the global_ref parameter and pass 3 as the ;" inc_subscr parameter, $$BIN2GBL will increment the third subscript, such ;" as ^TMP(115,1,x), but will WRITE notes at the full global reference, such ;" as ^TMP(115,1,x,0). ;"Result: 1=success, 0=failure ;" ;"Note: Each line of the global will contain up to 128 bytes (256 characters) ;" (2 ascii hex characters = 1 source byte) ;"Example: ;" ^TMP(115,1,1,0)="A12C4F12E2791D9723C3297D3C30B73C1532A1...(continues to 256 characters)" ;" ^TMP(115,1,2,0)="91D9723C3297D314ADF31B85F41A12C4F12E27...(continues to 256 characters)" ;" ^TMP(115,1,3,0)="3A12C4F12E271B85F4C2ED9723C3297D314ADF...(continues to 256 characters)" ;" ^TMP(115,1,4,0)="85F73C1532AA12C4F12E2791D9723C3297D314...(continues to 256 characters)" ;" ^TMP(115,1,5,0)="61A85C30B73C1532AA12C4F12E2791D972" <-- not padded with terminal zeros new result set result=0 ;"default to failure new handle set handle="TMGHANDLE" new abort set abort=0 new byteIn new $ETRAP new oneLine set oneLine="" new curRef set curRef=globalRef set path=$$DEFDIR^%ZISH($get(path)) do OPEN^%ZISH(handle,path,filename,"R") if POP goto B2GDone set $ETRAP="set abort=1,$ECODE="""" quit" use IO for do quit:($ZEOF)!(abort=1)!(byteIn=-1) . read *byteIn:2 . if (byteIn=-1) quit . set
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
A more fundamental question is whether M globals are binary friendly? I know that the underlying data is stored as strings. So would the storage of a $char(0) crash the data? I just tested this, and it doesn't. Thus I could store the actual binary data in the global instead of converting each byte to a hex equivalent and storing that. I could therefore read in 256 bytes at a time and store the data directly into the global. It would be much faster. But since my goal is to use the RPCBroker functionality that can transfer globals, I will next need to investigate what would happen if the global contained a control character (i.e. bytes 0-31). I think that when a WP field is transferred to Delphi via RPCBroker, it is put into a TMemo field. Each of the lines in a TMemo is a zero-terminated string (I think), so binary data might run into problems there. I'll have to think more about this, but any input would be appreciated. Kevin On 8/21/05, Kevin Toppenberg <[EMAIL PROTECTED]> wrote: > Here is the code. I will also attach it incase wrapping ruins it here... > > > ;"TMG BIN <-->GBL FUNCTION > ;"Kevin Toppenberg MD > ;"GNU General Public License (GPL) applies > ;"8-20-2005 > > ;"=== > ;" API -- Public Functions. > ;"=== > ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr) > ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename) > > ;"=== > ;"PRIVATE API FUNCTIONS > ;"=== > > > > ;"=== > BIN2GBL(path,filename,globalRef,incSubscr) > ;"Purpose: To load a binary file from the host filesystem into > a WP field, storing > ;" the composit bytes as ascii hex codes. > ;"Input: path --full path, up to but not including the > filename (required) > ;" filename -- name of the file to open (required) > ;" globalRef-- Global reference to WRITE the host > binary file to, in fully resolved > ;" (closed root) format. This > function does not kill the global before > ;" writing to it. (required) > ;" Note: > ;" At least one subscript must be > numeric. This will be the incrementing > ;" subscript (i.e. the subscript > that $$BIN2WP^TMGBINWP will increment > ;" to store each new global node). > This subscript need not be the final > ;" subscript. For example, to load > into a WORD PROCESSING field, the > ;" incrementing node is the > second-to-last subscript; the final subscript > ;" is always zero. > ;"incSubscr-- (required) Identifies the incrementing > subscript level. For example, if you > ;" pass ^TMP(115,1,1,0) as the > global_ref parameter and pass 3 as the > ;" inc_subscr parameter, $$BIN2GBL > will increment the third subscript, such > ;" as ^TMP(115,1,x), but will WRITE > notes at the full global reference, such > ;" as ^TMP(115,1,x,0). > ;"Result: 1=success, 0=failure > ;" > ;"Note: Each line of the global will contain up to 128 bytes > (256 characters) > ;" (2 ascii hex characters = 1 source byte) > ;"Example: > ;" > ^TMP(115,1,1,0)="A12C4F12E2791D9723C3297D3C30B73C1532A1...(continues > to 256 characters)" > ;" > ^TMP(115,1,2,0)="91D9723C3297D314ADF31B85F41A12C4F12E27...(continues > to 256 characters)" > ;" > ^TMP(115,1,3,0)="3A12C4F12E271B85F4C2ED9723C3297D314ADF...(continues > to 256 characters)" > ;" > ^TMP(115,1,4,0)="85F73C1532AA12C4F12E2791D9723C3297D314...(continues > to 256 characters)" > ;" ^TMP(115,1,5,0)="61A85C30B73C1532AA12C4F12E2791D972" <-- > not padded with terminal zeros > > new result set result=0 ;"default to failure > new handle set handle="TMGHANDLE" > new abort set abort=0 > new byteIn > new $ETRAP > new oneLine set oneLine="" > new curRef set curRef=globalRef > > set path=$$DEFDIR^%ZISH($get(path)) > do OPEN^%ZISH(handle,path,filename,"R") > if POP goto B2GDone > set $ETRAP="set abort=1,$ECODE= quit" > use IO > for do quit:($ZEOF)!(abort=1)!(byteIn=-1) > . read *byteIn:2 > . if (byteIn=-1) quit > . set oneLine=oneLine_$$HEXCHR(byteIn,2) > . if
[Hardhats-members] Re: Is $$GTF~%ZISH() binary friendly?
Here is the code. I will also attach it incase wrapping ruins it here... ;"TMG BIN <-->GBL FUNCTION ;"Kevin Toppenberg MD ;"GNU General Public License (GPL) applies ;"8-20-2005 ;"=== ;" API -- Public Functions. ;"=== ;"$$BIN2GBL^TMGBINF(path,filename,globalRef,incSubscr) ;"$$GBL2BIN^TMGBINF(globalRef,incSubscr,path,filename) ;"=== ;"PRIVATE API FUNCTIONS ;"=== ;"=== BIN2GBL(path,filename,globalRef,incSubscr) ;"Purpose: To load a binary file from the host filesystem into a WP field, storing ;" the composit bytes as ascii hex codes. ;"Input: path --full path, up to but not including the filename (required) ;" filename -- name of the file to open (required) ;" globalRef-- Global reference to WRITE the host binary file to, in fully resolved ;" (closed root) format. This function does not kill the global before ;" writing to it. (required) ;" Note: ;" At least one subscript must be numeric. This will be the incrementing ;" subscript (i.e. the subscript that $$BIN2WP^TMGBINWP will increment ;" to store each new global node). This subscript need not be the final ;" subscript. For example, to load into a WORD PROCESSING field, the ;" incrementing node is the second-to-last subscript; the final subscript ;" is always zero. ;"incSubscr-- (required) Identifies the incrementing subscript level. For example, if you ;" pass ^TMP(115,1,1,0) as the global_ref parameter and pass 3 as the ;" inc_subscr parameter, $$BIN2GBL will increment the third subscript, such ;" as ^TMP(115,1,x), but will WRITE notes at the full global reference, such ;" as ^TMP(115,1,x,0). ;"Result: 1=success, 0=failure ;" ;"Note: Each line of the global will contain up to 128 bytes (256 characters) ;" (2 ascii hex characters = 1 source byte) ;"Example: ;" ^TMP(115,1,1,0)="A12C4F12E2791D9723C3297D3C30B73C1532A1...(continues to 256 characters)" ;" ^TMP(115,1,2,0)="91D9723C3297D314ADF31B85F41A12C4F12E27...(continues to 256 characters)" ;" ^TMP(115,1,3,0)="3A12C4F12E271B85F4C2ED9723C3297D314ADF...(continues to 256 characters)" ;" ^TMP(115,1,4,0)="85F73C1532AA12C4F12E2791D9723C3297D314...(continues to 256 characters)" ;" ^TMP(115,1,5,0)="61A85C30B73C1532AA12C4F12E2791D972" <-- not padded with terminal zeros new result set result=0 ;"default to failure new handle set handle="TMGHANDLE" new abort set abort=0 new byteIn new $ETRAP new oneLine set oneLine="" new curRef set curRef=globalRef set path=$$DEFDIR^%ZISH($get(path)) do OPEN^%ZISH(handle,path,filename,"R") if POP goto B2GDone set $ETRAP="set abort=1,$ECODE= quit" use IO for do quit:($ZEOF)!(abort=1)!(byteIn=-1) . read *byteIn:2 . if (byteIn=-1) quit . set oneLine=oneLine_$$HEXCHR(byteIn,2) . if $length(oneLine)>255 do . . set @curRef=oneLine . . set curRef=$$NEXTNODE(curRef,incSubscr) . . set oneLine="" if (oneLine'="")&(abort=0) do . set @curRef=oneLine . set oneLine="" if (abort'=1) set result=1 ;"SUCCESS do CLOSE^%ZISH(handle) B2GDone quit result NEXTNODE(curRef,incSubscr) ;"Purpose: to take a global reference, and increment the node specified by incSubscr ;"Input: curRef --The reference to alter, e.g. '^TMP(115,1,4,0)' ;" incSubscr--The node to alter, e.g. ;" 1-->^TMG(x,1,4,0)x would be incremented ;" 2-->^TMG(115,x,4,0) x would be incremented ;" 3-->^TMG(115,1,x,0) x would be incremented ;" 4-->^TMG(115,1,4,x) x would be incremented ;"Note: the node that incSubscr references should be numeric (i.e. not a name) ;" otherwise the alpha node will be treated as a 0 ;"result: returns the new reference
