Re: [Plplot-devel] Exit calls and memory management
@Hazen and Alan I think an email went AWOL from Hazen somewhere, but from Alan's reply I guess it probably said that we should not be rebasing public branches. I must confess that I think almost everything I have read about rebasing says do not rebase a public branch. I'm certainly no expert, but it does make me nervous that we are suggesting to fly in the face of the perceived wisdom. I think I would be much more comfortable with accepting that where we need to collaborate on things we and use public branches, we merge these instead of rebasing them. I know this makes for a slightly less tidy history, but such large changes don't happen often. Don't forget it isn't only us that can download a public branch - anyone can download from my repo, find and fix a bug, then make a push request - if we have to say sorry that history doesn't exist we can't accept your push, then it doesn't make us look great. That is of course a very idealistic way of looking at things and I'm all for practicality over idealism. So if you still want to go for the rebase option then I have no problem with that. Just so long as we have considered the potential problems. @Andrew. You are of course correct that these changes in themselves will not remove the exit calls, but my intention is that they will make failed memory allocations easier to deal with - even if it is just that by using an array's internal element count it will avoid buffer overruns even if memory allocation fails - i.e. it removes the need for each reallocation to be checked and the element count be either zero'd or reset to its old value. Regarding the replacement to plexit. Two options were discussed at some point. Either 1) Use the plabort callback set by the user to report error messages or 2)Have an error code and message which we save and add a plgeterror() function to the API, to retrieve these values. I am tending towards 1) for the following reasons: a) Almost nobody ever bothers to actually check the error code, but if they have created a callback then they are more likely to make use of the error. (I am as guilty as that as anyone so I now pepper my code with exception throws to force at least some occurance if something fails) b) For 2 we need to reset the error code for every api call. This isn't too bad, but we also have to make sure we never accidentally reset it by calling an API function internally - that will be a real pain to disentangle and enforce. Some arguments for 2 though are i) If we ever make plplot threadsafe then we need some way to make sure the plabort callback is also threadsafe - but I guess we would need that anyway ii) I'm not sure what would happen if our callback threw an exception - it would probably leave the library in some unresolved state. But that is the case for the callback now too. As you said the error code can be set and checked internally and a function can be used to allow it to be easily set by other bindings. Actually in a lot of cases Plplot can probably carry on regardless - providing the array size has been remembered correctly, hence the advantage of arrays which know their own size. However I am certainly still open to suggestions - I think based on the other comments on this thread I will set up a github repository and people can see what I've done so far and comment and suggest as needed. Phil From: Alan W. Irwin ir...@beluga.phys.uvic.ca To: Hazen Babcock hbabc...@mac.com; phil rosenberg philip_rosenb...@yahoo.com Cc: PLplot development list plplot-devel@lists.sourceforge.net; Hezekiah M. Carty h...@0ok.org Sent: Monday, 22 September 2014, 23:00 Subject: Re: [Plplot-devel] Exit calls and memory management On 2014-09-22 16:41-0400 Hazen Babcock wrote: On 9/22/2014 3:37 PM, Alan W. Irwin wrote: On 2014-09-22 12:08-0400 Hazen Babcock wrote: I think creating a branch on github (or some other public repository) is the only way that you can proceed if you want others to see what you are doing. Though not perhaps ideal, you should be able to rebase master off a public branch. Since our advice to Phil is completely contradictory, we should get this important question of workflow decided for the case of experimental topic development work (as opposed to the workflow for our more usual less controversial topic development which is normally pushed to origin master as soon as the changes by a developer pass his own tests). And once the workflow for this special case of experimental development topics is decided, we should document it in README.developers. You are much more experienced with git than me. However, I thought that rebasing a public branch was always a bad idea for the reasons I mentioned concerning disappearing commits. I am positive a number of resources I read when we were considering using a rebase workflow mentioned this drawback to that approach. Were those resources overstating
Re: [Plplot-devel] Exit calls and memory management
On 2014-09-23 03:30-0700 phil rosenberg wrote: @Hazen and Alan I think an email went AWOL from Hazen somewhere, but from Alan's reply I guess it probably said that we should not be rebasing public branches. I must confess that I think almost everything I have read about rebasing says do not rebase a public branch. I'm certainly no expert, but it does make me nervous that we are suggesting to fly in the face of the perceived wisdom. I think I would be much more comfortable with accepting that where we need to collaborate on things we and use public branches, we merge these instead of rebasing them. I know this makes for a slightly less tidy history, but such large changes don't happen often. Don't forget it isn't only us that can download a public branch - anyone can download from my repo, find and fix a bug, then make a push request - if we have to say sorry that history doesn't exist we can't accept your push, then it doesn't make us look great. That is of course a very idealistic way of looking at things and I'm all for practicality over idealism. So if you still want to go for the rebase option then I have no problem with that. Just so long as we have considered the potential problems. It appears e-mails are still going astray. For example, yesterday, I believe Hazen and I came to consensus on this list concerning this issue, and I wrote that consensus up as an additional paragraph in README.developers and asked for your and Hazen's comments on that paragraph. I believe that paragraph answers the concerns you expressed above, but please let me know if it doesn't. [...]However I am certainly still open to suggestions - I think based on the other comments on this thread I will set up a github repository and people can see what I've done so far and comment and suggest as needed. See the remarks in README.developers concerning that possibility. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
Hi, Alan, On Sep 22, 2014, at 12:37 PM, Alan W. Irwin wrote: You are much more experienced with git than me. However, I thought that rebasing a public branch was always a bad idea for the reasons I mentioned concerning disappearing commits. I am positive a number of resources I read when we were considering using a rebase workflow mentioned this drawback to that approach. I agree that rebasing any part of a branch that has been pushed is a bad idea. It makes things complicated/inconvenient for those who are following the branch. If they are right, then for rebase workflows like ours experimental branches should not be made accessible on public git repos since the bad choices afterwards are to rebase them periodically (which loses prior public commits) to keep up with master development or else become out of date with master. I can understand the desire for a nice clean linear history, especially since the code was just migrated from Subversion. Strict adherence to a rebase only workflow will definitely result in a nice clean linear history. Starting with a strict rebase only workflow requirement, the statement quoted above sounds reasonable, but the experimental branches should not be made accessible on public git repos part of it really sounds wrong to me. It makes me wonder whether a strict rebase only workflow is really best. Which means some other alternative for sharing experimental development topics must be used such as sending patches to this list. My feeling is that working with such patches is unlikely to be much of a burden since git has such good facilities (format-patch and am) for generating and using such patches. Emailing patches is tedious and error prone. Yes, git has good facilities for this, but it has even better facilities for sharing changes: pushing to public repos. Asking developers to refrain from pushing their experimental branches to public repos is somewhat hobbling, IMHO. Of course, if after we have gained some substantial experience with the patch method for the special case of sharing experimental development, we find that method is inhibiting our experimental development, we should move to a merge workflow instead like suggested by Brad King. But for now, let's give a good trial (at least 6 months) to our current rebase workflow including its constraint (if the resources I read are not overstating the case) that developers should refrain from sharing their experimental branches via public repos because it leads to bad further choices that typically come back to haunt you with a rebase workflow. I am not familiar with Brad King's merge workflow, but I think allowing some merge commits can be a good thing. A compromise between a rebase-only workflow and an unconstrained merge workflow would be to allow merge commits on master but only if the first parent of such merge commits is also on the master branch. Developers would still be allowed (encouraged?) to follow a rebase-as-much-as-possible workflow, but they could also push their experimental branches to public repos without having to worry about rebasing them later. This would allow for a concise summary of changes on master by running git log --oneline --first-parent master. Each non-merge commit on master would be shown along with the merge commit from each (non-rebased) topic branch, but the topic branch commits themselves would not be shown (because we passed --first-parent). Personally, I find merge commits to be a helpful way of partitioning the history of a project, especially when using gitk or git log --graph. They are kind of analogous to subdirectories in that not having merge commits is kind of like keeping all your files in one directory. When running git log --graph, merge commits make it easy to see which commits go together because they are shown on the same topic/experimental branch. Without merge commits, everything is just one lone thread of development. Of course it is possible to overuse merge commits, but generally if a topic branch is significant enough to share before rebasing it to master, then it's probably significant enough to warrant a merge commit on master when the time comes to bring the topic branch back into the fold. Hope this helps, Dave -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
[Plplot-devel] Exit calls and memory management
This one is mostly directed at Alan but probably others will be interested and may have comments. Git might also be a big help here, but I'm not sure how it will work so advice from all appreciated. I have been trying to improve memory management in plplot as part of my attempt to try to remove exit calls. To do this I am removing raw pointer arrays and replacing them with a struct which will give us more object oriented access to memory allocation. I think this will be a generally good thing as the struct can have function pointers which will allow trivial resizing and adding elements. It will also be extendible, to allow further functionality similar to std::vector if we want. Arrays will also know their own size. However the cost is that arrays in the plstream will all become structs so will need accessing by somearray.mem[index] or maybe somearray.getMem()[index] rather than somearray[index]. This has the potential to break some or all the drivers. Although the fixes might be trivial, I don't have easy access to all the drivers on my windows machine. I'm therefore looking for a suggestion of how to proceed. I can transfer stuff to my Linux machine, but I'm not sure how best to do that with git. I could create a fork on github so other developers can contribute, but if I create a public fork then we should merge that into the master branch rather than rebase it which I think is not compliant with our current workflow. Any suggestions greatly appreciated. Phil-- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
Unless the memory calls have changed, the raw pointers was something I implemented 5+ years ago when I submitted a patch to transition away from temporary files. The design goal I had in mind was speed, to keep the implementation portable, and to make the memory buffers agnostic to the data. Speed was important to me because I was using plplot to display data real time. I implemented a simple double the memory buffer when the buffer got full. I thought about a making the code smarter about memory management, but I decided that the compiler and OS would suffice and the benefit was (what I thought) simpler code. I did not want to play whack-a-mole fixing memory allocation bugs. That said, tools have gotten better and a fresh look is a good idea. On Sep 22, 2014, at 10:17 AM, Phil Rosenberg philip_rosenb...@yahoo.com wrote: This one is mostly directed at Alan but probably others will be interested and may have comments. Git might also be a big help here, but I'm not sure how it will work so advice from all appreciated. I have been trying to improve memory management in plplot as part of my attempt to try to remove exit calls. To do this I am removing raw pointer arrays and replacing them with a struct which will give us more object oriented access to memory allocation. I think this will be a generally good thing as the struct can have function pointers which will allow trivial resizing and adding elements. It will also be extendible, to allow further functionality similar to std::vector if we want. Arrays will also know their own size. However the cost is that arrays in the plstream will all become structs so will need accessing by somearray.mem[index] or maybe somearray.getMem()[index] rather than somearray[index]. This has the potential to break some or all the drivers. Although the fixes might be trivial, I don't have easy access to all the drivers on my windows machine. I'm therefore looking for a suggestion of how to proceed. I can transfer stuff to my Linux machine, but I'm not sure how best to do that with git. I could create a fork on github so other developers can contribute, but if I create a public fork then we should merge that into the master branch rather than rebase it which I think is not compliant with our current workflow. Any suggestions greatly appreciated. Phil -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
Message: 5 Date: Mon, 22 Sep 2014 15:17:36 +0100 From: Phil Rosenberg philip_rosenb...@yahoo.com Subject: [Plplot-devel] Exit calls and memory management To: PLplot development list Plplot-devel@lists.sourceforge.net Message-ID: 218078.55503...@smtp115.mail.ir2.yahoo.com Content-Type: text/plain; charset=windows-1252 This one is mostly directed at Alan but probably others will be interested and may have comments. Git might also be a big help here, but I'm not sure how it will work so advice from all appreciated. I have been trying to improve memory management in plplot as part of my attempt to try to remove exit calls. To do this I am removing raw pointer arrays and replacing them with a struct which will give us more object oriented access to memory allocation. I think this will be a generally good thing as the struct can have function pointers which will allow trivial resizing and adding elements. It will also be extendible, to allow further functionality similar to std::vector if we want. Arrays will also know their own size. However the cost is that arrays in the plstream will all become structs so will need accessing by somearray.mem[index] or maybe somearray.getMem()[index] rather than somearray[index]. This has the potential to break some or all the drivers. Although the fixes might be trivial, I don't have easy access to all the drivers on my windows machine. I'm therefore looking for a suggestion of how to proceed. I can transfer stuff to my Linux machine, but I'm not sure how best to do that with git. I could create a fork on github so other developers can contribute, but if I create a public fork then we should merge that into the master branch rather than rebase it which I think is not compliant with our current workflow. I think creating a branch on github (or some other public repository) is the only way that you can proceed if you want others to see what you are doing. Though not perhaps ideal, you should be able to rebase master off a public branch. If you are interested, Hez or I could add you to the PLplot organization on github. Then you could push your branch there. We haven't done anything with it yet, so you'll have to create a the repository first. https://github.com/PLplot -Hazen -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
On 2014-09-22 15:17+0100 Phil Rosenberg wrote: However the cost [of this change] is that arrays in the plstream will all become structs so will need accessing by somearray.mem[index] or maybe somearray.getMem()[index] rather than somearray[index]. This has the potential to break some or all the drivers. Although the fixes might be trivial, I don't have easy access to all the drivers on my windows machine. I'm therefore looking for a suggestion of how to proceed. I can transfer stuff to my Linux machine, but I'm not sure how best to do that with git. I could create a fork on github so other developers can contribute, but if I create a public fork then we should merge that into the master branch rather than rebase it which I think is not compliant with our current workflow. Hi Phil: Publishing your experimental development branch using github or our SF repository might be a tempting way to share it for evaluation, but the rebase steps of just keeping up with master or rebasing before you merge with master become problematic since all the commits on your branch get changed into new commits by that rebase. The result is the old set of commits for you branch eventually get garbage collected, leaving users of that old set of public commits high and dry which, of course, is a situation to be avoided. From what I have seen as a result of lurking on several development lists where git is being used, it is quite common to communicate experimental branches using patches between core developers rather than by making the branch available via public git access. That is, they use git format-patch to create a series of patches of their work which they attach to development-list e-mails, and others on the list apply those patches to some experimental unpublished branch of their own using the git am commmand. They then test that experimental branch all they like and even make changes to it which they share back with the original branch developer via git format-patch, etc. Given our rebase workflow constraints, that is the way I suggest you share your experimental branch work with the rest of the core developers here with the further understanding that nobody pushes that work to any publicly accessible git repository for the above reason. Once the branch matures (with hopefully more than one developer testing it), then your final job for that experimental branch would be to rebase your private version of that branch and merge it to master. Note that once the core developers here with a lot of C experience agree that your experimental idea is a good one, then I would encourage you to go ahead and quickly rebase your changes and then push them to origin master. This quick-push to master philosophy is possible with PLplot because of its nice architecture that keeps driver changes largely isolated from each other and also from core library changes. That is, as long as your approved changes don't interfere with building PLplot, and run-time tests for at least some of the drivers work, then we can figure out the rest using fixes to the master branch version. Note when your experimental series of patches is published to this list, I would leave to others with more C experience the judgement call about whether your changes are the right way for us to go, but I would certainly be happy to give you feedback based on how well the result builds and runs on Linux for each device driver there. Of course, to answer your last question, you can proceed on Linux by simply cloning the official SF repo, followed by using git am to apply your experimental branch patches to an experimental branch of that local repo. followed by build and run testing on Linux. In fact, it sounds like you would want to do that test before publishing your experimental patch series here. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net
Re: [Plplot-devel] Exit calls and memory management
Thanks for the thumbs up Jim.
My intention is basically to make development easier and give better code
reuse. It is much easier to do
array.resize(array, 10);
if(array.size!=10)
{
array.destroy(array);
//do some other clean up and deal with problem as needed
}
Than people writing their own reallocation code every time. It also avoids bugs
where sizes of arrays are not updated and it means that behind the scenes extra
spare memory can be allocated and tracked to avoid extra reallocations.
If drivers need to access the data to read or write fast then they can grab the
pointer and use it raw
double *arrayraw=array.mem;
size_t n = array.size;
for (i=0; in; ++I)
//do something with each element
It really comes from the fact that most exit calls come from memory
allocations, so effectively dealing with them is much easier with a more
controlled memory management system. Maybe it also comes from the fact that I
am a C++ person so I get used to the object oriented stuff.
-Original Message-
From: Jim Dishaw j...@dishaw.org
Sent: 22/09/2014 16:32
To: Phil Rosenberg philip_rosenb...@yahoo.com
Cc: PLplot development list Plplot-devel@lists.sourceforge.net
Subject: Re: [Plplot-devel] Exit calls and memory management
Unless the memory calls have changed, the raw pointers was something I
implemented 5+ years ago when I submitted a patch to transition away from
temporary files.
The design goal I had in mind was speed, to keep the implementation portable,
and to make the memory buffers agnostic to the data. Speed was important to me
because I was using plplot to display data real time. I implemented a simple
double the memory buffer when the buffer got full.
I thought about a making the code smarter about memory management, but I
decided that the compiler and OS would suffice and the benefit was (what I
thought) simpler code. I did not want to play whack-a-mole fixing memory
allocation bugs.
That said, tools have gotten better and a fresh look is a good idea.
On Sep 22, 2014, at 10:17 AM, Phil Rosenberg philip_rosenb...@yahoo.com wrote:
This one is mostly directed at Alan but probably others will be interested and
may have comments. Git might also be a big help here, but I'm not sure how it
will work so advice from all appreciated.
I have been trying to improve memory management in plplot as part of my attempt
to try to remove exit calls. To do this I am removing raw pointer arrays and
replacing them with a struct which will give us more object oriented access to
memory allocation. I think this will be a generally good thing as the struct
can have function pointers which will allow trivial resizing and adding
elements. It will also be extendible, to allow further functionality similar to
std::vector if we want. Arrays will also know their own size.
However the cost is that arrays in the plstream will all become structs so will
need accessing by somearray.mem[index] or maybe somearray.getMem()[index]
rather than somearray[index]. This has the potential to break some or all the
drivers. Although the fixes might be trivial, I don't have easy access to all
the drivers on my windows machine.
I'm therefore looking for a suggestion of how to proceed. I can transfer stuff
to my Linux machine, but I'm not sure how best to do that with git. I could
create a fork on github so other developers can contribute, but if I create a
public fork then we should merge that into the master branch rather than rebase
it which I think is not compliant with our current workflow.
Any suggestions greatly appreciated.
Phil
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
On 2014-09-22 12:08-0400 Hazen Babcock wrote: I think creating a branch on github (or some other public repository) is the only way that you can proceed if you want others to see what you are doing. Though not perhaps ideal, you should be able to rebase master off a public branch. Hi Hazen: Since our advice to Phil is completely contradictory, we should get this important question of workflow decided for the case of experimental topic development work (as opposed to the workflow for our more usual less controversial topic development which is normally pushed to origin master as soon as the changes by a developer pass his own tests). And once the workflow for this special case of experimental development topics is decided, we should document it in README.developers. You are much more experienced with git than me. However, I thought that rebasing a public branch was always a bad idea for the reasons I mentioned concerning disappearing commits. I am positive a number of resources I read when we were considering using a rebase workflow mentioned this drawback to that approach. Were those resources overstating the case? If they are right, then for rebase workflows like ours experimental branches should not be made accessible on public git repos since the bad choices afterwards are to rebase them periodically (which loses prior public commits) to keep up with master development or else become out of date with master. Which means some other alternative for sharing experimental development topics must be used such as sending patches to this list. My feeling is that working with such patches is unlikely to be much of a burden since git has such good facilities (format-patch and am) for generating and using such patches. Of course, if after we have gained some substantial experience with the patch method for the special case of sharing experimental development, we find that method is inhibiting our experimental development, we should move to a merge workflow instead like suggested by Brad King. But for now, let's give a good trial (at least 6 months) to our current rebase workflow including its constraint (if the resources I read are not overstating the case) that developers should refrain from sharing their experimental branches via public repos because it leads to bad further choices that typically come back to haunt you with a rebase workflow. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
On 9/22/2014 3:37 PM, Alan W. Irwin wrote: On 2014-09-22 12:08-0400 Hazen Babcock wrote: I think creating a branch on github (or some other public repository) is the only way that you can proceed if you want others to see what you are doing. Though not perhaps ideal, you should be able to rebase master off a public branch. Since our advice to Phil is completely contradictory, we should get this important question of workflow decided for the case of experimental topic development work (as opposed to the workflow for our more usual less controversial topic development which is normally pushed to origin master as soon as the changes by a developer pass his own tests). And once the workflow for this special case of experimental development topics is decided, we should document it in README.developers. You are much more experienced with git than me. However, I thought that rebasing a public branch was always a bad idea for the reasons I mentioned concerning disappearing commits. I am positive a number of resources I read when we were considering using a rebase workflow mentioned this drawback to that approach. Were those resources overstating the case? Probably not. However I don't really see a problem with someone pushing a private branch of PLplot to a public repo and asking others to have a look and make suggestions, as long as everyone else understands that it is read only. Actual collaboration on a branch is going to more complicated and I have no experience in this area using a rebase workflow. Exchanging patches sound like as good as an approach as any. -Hazen -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
Phil, This is a laudible plan, however fundamentally it won't remove the exit calls from plplot. The reason is that memory allocation can fail. We always need to check this and fail graceful, returning an error to the calling program and letting it decide how to procede. However we do the memory management, this is still true. The fundamental problem is that we have no mechanism for doing this at the moment since none of our functions return values. To change this is a massive API change. Another alternative would to have some global variable (like the errno variable used in the C library) and require calling programs to check this. This is not strictly an API change, but if we made it the default then it would be a big change in behaviour and would require user code changes to make the code safe. We could have it as an option though. More of a problem would be making sure this worked for all language bindings. A new function return the error flag would be one safe way. Actually have a separate function which returns the error flag probably is a clean way of doing it. We can use the global variable internally as well for higher level function to check that lower level functions which may cause memory errors have exectuted without problems. I've thought about this a lot in the past and we've had threads on list and not come up with a good way of changing past design mistakes (or features?) in a painless way. Happy to be proved wrong of course and I'm not saying we shouldn't do something dramatic. We need to think it through careful though to make sure it is the right design decision and try to make it as simple as possible for users. In the meantime tidying up the code as you suggest will help minimise the places changes are required. Andrew On Mon, Sep 22, 2014 at 03:17:36PM +0100, Phil Rosenberg wrote: This one is mostly directed at Alan but probably others will be interested and may have comments. Git might also be a big help here, but I'm not sure how it will work so advice from all appreciated. I have been trying to improve memory management in plplot as part of my attempt to try to remove exit calls. To do this I am removing raw pointer arrays and replacing them with a struct which will give us more object oriented access to memory allocation. I think this will be a generally good thing as the struct can have function pointers which will allow trivial resizing and adding elements. It will also be extendible, to allow further functionality similar to std::vector if we want. Arrays will also know their own size. However the cost is that arrays in the plstream will all become structs so will need accessing by somearray.mem[index] or maybe somearray.getMem()[index] rather than somearray[index]. This has the potential to break some or all the drivers. Although the fixes might be trivial, I don't have easy access to all the drivers on my windows machine. I'm therefore looking for a suggestion of how to proceed. I can transfer stuff to my Linux machine, but I'm not sure how best to do that with git. I could create a fork on github so other developers can contribute, but if I create a public fork then we should merge that into the master branch rather than rebase it which I think is not compliant with our current workflow. Any suggestions greatly appreciated. Phil -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Exit calls and memory management
On 2014-09-22 16:41-0400 Hazen Babcock wrote: On 9/22/2014 3:37 PM, Alan W. Irwin wrote: On 2014-09-22 12:08-0400 Hazen Babcock wrote: I think creating a branch on github (or some other public repository) is the only way that you can proceed if you want others to see what you are doing. Though not perhaps ideal, you should be able to rebase master off a public branch. Since our advice to Phil is completely contradictory, we should get this important question of workflow decided for the case of experimental topic development work (as opposed to the workflow for our more usual less controversial topic development which is normally pushed to origin master as soon as the changes by a developer pass his own tests). And once the workflow for this special case of experimental development topics is decided, we should document it in README.developers. You are much more experienced with git than me. However, I thought that rebasing a public branch was always a bad idea for the reasons I mentioned concerning disappearing commits. I am positive a number of resources I read when we were considering using a rebase workflow mentioned this drawback to that approach. Were those resources overstating the case? Probably not. However I don't really see a problem with someone pushing a private branch of PLplot to a public repo and asking others to have a look and make suggestions, as long as everyone else understands that it is read only. Actual collaboration on a branch is going to more complicated and I have no experience in this area using a rebase workflow. Exchanging patches sound like as good as an approach as any. OK. I have updated README.developers accordingly (commit fb1dcd5). @Hazen and Phil: Does the language in this new paragraph seem clear enough about our options for sharing experimental topic branch work that we are not going to push to origin master immediately? Alan -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
