Re: OpenGL Examples in D and a birth of a New Initiative
On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. On 15 May 2014 05:04, Andrej Mitrovic via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: I am starting an initiative for everyone interested in D game development by creating the github organization d-gamedev-team[1]. The first project hosted here is the the freshly created opengl-tutorials[2] project. This is a long-term project that aims to collect as many modern OpenGL examples ported to D or even provide new examples created from scratch in D. All the examples should be easy to build, provided you have a fresh D compiler and a recent version of the dub[3] dependency manager / builder tool. Currently the project contains an almost-complete port of the samples contained on the website opengl-tutorial.org[4] - which cover OpenGL v3.3x and were created by Sam Hocevar. The D examples are not straight ports, they have been D-ified and may contain more features than their original counterparts. They also use 3rd-party D OpenGL wrappers to avoid a lot of scaffolding work typical of OpenGL applications. Thanks to dub[3] you won't have to manually install the D dependencies. However you may have to install some C/C++ 3rd-party library dependencies, such as GLFW, SDL2, SDL2 Image, and potentially other libraries depending on the ones the samples require. The dependencies are documented in the project's readme[5]. Additional OpenGL example ports are planned and some are already in progress but have not yet been pushed upstream. See the opengl-tutorials[2] github repo for a list of ports which are in progress and a list of desired ports. If you wish to contribute with your own ports or with brand new D OpenGL examples don't hesitate to make a pull request. I want the d-gamedev-team organization to gradually grow and have it host as many useful projects, whether it be libraries, tools, tutorials, tips tricks, assets, or anything else related to D game development. I am strongly interested in game development in D, and although I'm at a starting stage I'm very much committed to working on this project to the best of my abilities. Hopefully in a few years we'll see some major titles made entirely in D. But before that can happen we have to make a collective push to bring D to a higher stage where it's acceptable and viable to game developers, whether it be through language support (@nogc/allocators) or library and tooling support. We'll see where this goes, but I'm very excited for the future! Cheers. :o) [1] : https://github.com/d-gamedev-team [2] : https://github.com/d-gamedev-team/opengl-tutorials [3] : http://code.dlang.org/download [4] : http://opengl-tutorial.org/ [5] : https://github.com/d-gamedev-team/opengl-tutorials/tree/master/ports/opengl-tutorial.org Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front.
Re: OpenGL Examples in D and a birth of a New Initiative
On Thursday, 15 May 2014 at 05:01:14 UTC, Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. On 15 May 2014 05:04, Andrej Mitrovic via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: I am starting an initiative for everyone interested in D game development by creating the github organization d-gamedev-team[1]. The first project hosted here is the the freshly created opengl-tutorials[2] project. This is a long-term project that aims to collect as many modern OpenGL examples ported to D or even provide new examples created from scratch in D. All the examples should be easy to build, provided you have a fresh D compiler and a recent version of the dub[3] dependency manager / builder tool. Currently the project contains an almost-complete port of the samples contained on the website opengl-tutorial.org[4] - which cover OpenGL v3.3x and were created by Sam Hocevar. The D examples are not straight ports, they have been D-ified and may contain more features than their original counterparts. They also use 3rd-party D OpenGL wrappers to avoid a lot of scaffolding work typical of OpenGL applications. Thanks to dub[3] you won't have to manually install the D dependencies. However you may have to install some C/C++ 3rd-party library dependencies, such as GLFW, SDL2, SDL2 Image, and potentially other libraries depending on the ones the samples require. The dependencies are documented in the project's readme[5]. Additional OpenGL example ports are planned and some are already in progress but have not yet been pushed upstream. See the opengl-tutorials[2] github repo for a list of ports which are in progress and a list of desired ports. If you wish to contribute with your own ports or with brand new D OpenGL examples don't hesitate to make a pull request. I want the d-gamedev-team organization to gradually grow and have it host as many useful projects, whether it be libraries, tools, tutorials, tips tricks, assets, or anything else related to D game development. I am strongly interested in game development in D, and although I'm at a starting stage I'm very much committed to working on this project to the best of my abilities. Hopefully in a few years we'll see some major titles made entirely in D. But before that can happen we have to make a collective push to bring D to a higher stage where it's acceptable and viable to game developers, whether it be through language support (@nogc/allocators) or library and tooling support. We'll see where this goes, but I'm very excited for the future! Cheers. :o) [1] : https://github.com/d-gamedev-team [2] : https://github.com/d-gamedev-team/opengl-tutorials [3] : http://code.dlang.org/download [4] : http://opengl-tutorial.org/ [5] : https://github.com/d-gamedev-team/opengl-tutorials/tree/master/ports/opengl-tutorial.org same with DirectX, more than half year passed since i published it and i only get single commit and one known user. nice, good and active community. i wish to say good luck with ur linuxez guyz i'm done here, but i always returns to see whats new, still hoping it's temporally... p.s. i admit that linux could give us some traction to have public approval, but it doesn't mean that there is only linux...
Re: OpenGL Examples in D and a birth of a New Initiative
On 5/15/14, Manu via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Sorry, but I was put off by the terrible performance. 30 FPS with regular frame drops to sub-10 FPS is unacceptable for something that displays half a dozen animated objects. It left a really bad impression on me, especially considering you always complain on the forums about other devs writing bad-performant code and them being unaware of it.
Re: OpenGL Examples in D and a birth of a New Initiative
On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm?
Re: OpenGL Examples in D and a birth of a New Initiative
On 15 May 2014 19:54, Andrej Mitrovic via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 5/15/14, Manu via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Sorry, but I was put off by the terrible performance. 30 FPS with regular frame drops to sub-10 FPS is unacceptable for something that displays half a dozen animated objects. It left a really bad impression on me, especially considering you always complain on the forums about other devs writing bad-performant code and them being unaware of it. Huh? Umm... I haven't done any work on rendering or performance. It's 100% prototype rendering code. It's rendering a crap load of procedural meshes generated on the spot each frame, and not even cacheing the text. What's there is a proof of concept. I just wanted to get a framework, and get something running and playable, so that other people could join in and hack on it with me. I was worried about synch and gameplay, ie, making it load and play songs, play them in time, receive controller events in time, read MIDI properly, match input events against notes in the songs, keep score, handle various input and output latencies. There's actually a fair bit of work in there. You never mentioned that you were deterred by performance, I can go and make it run at a thousand fps if you like :) I just didn't consider it a pressing or particularly relevant issue, since there's not a single piece of artwork or UI in there anyway. Any rendering code I write will almost certainly be replaced. I think there's all comments in there saying // HACK: horrible rendering code!, and that it will be replaced... :/
Re: OpenGL Examples in D and a birth of a New Initiative
On 16/05/2014 12:07 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm? DOOGLE[0] was my attempt at a gui toolkit. And aurora is this[1]. I would have replied earlier but it appears that there has been enough changes with dub and dmd that it just won't. But I managed to grab an old build and here is its output [2]. Not really that nice but hey, was my go at it. I was building it as pretty much the all in one toolbox for D and OpenGL and with Aurora getting more support, the issues I was having continuing just kinda was the last straw. And anyways I'm better at web development then game dev. Never really understood the 3d pipeline. [0] https://github.com/rikkimax/DOOGLE [1] https://github.com/auroragraphics [2] https://drive.google.com/file/d/0B-EiBquZktsLcGtMSmtMS1ZoZHM/edit?usp=sharing
Re: OpenGL Examples in D and a birth of a New Initiative
On 15 May 2014 23:24, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 12:07 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm? DOOGLE[0] was my attempt at a gui toolkit. And aurora is this[1]. I would have replied earlier but it appears that there has been enough changes with dub and dmd that it just won't. But I managed to grab an old build and here is its output [2]. Not really that nice but hey, was my go at it. I was building it as pretty much the all in one toolbox for D and OpenGL and with Aurora getting more support, the issues I was having continuing just kinda was the last straw. And anyways I'm better at web development then game dev. Never really understood the 3d pipeline. Those aurora repos appear to be empty :/ Is aurora intended to be a GUI toolkit? I still think there's huge need for a realtime gfx environment GUI toolkit. But if I were you, I wouldn't be marrying it to any particular gfx API. Just make sure to structure the API to allow backends (like OpenGL) to be plugged in. [0] https://github.com/rikkimax/DOOGLE [1] https://github.com/auroragraphics [2] https://drive.google.com/file/d/0B-EiBquZktsLcGtMSmtMS1ZoZHM/edit?usp=sharing
Re: OpenGL Examples in D and a birth of a New Initiative
On 16/05/2014 1:38 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 23:24, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 12:07 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm? DOOGLE[0] was my attempt at a gui toolkit. And aurora is this[1]. I would have replied earlier but it appears that there has been enough changes with dub and dmd that it just won't. But I managed to grab an old build and here is its output [2]. Not really that nice but hey, was my go at it. I was building it as pretty much the all in one toolbox for D and OpenGL and with Aurora getting more support, the issues I was having continuing just kinda was the last straw. And anyways I'm better at web development then game dev. Never really understood the 3d pipeline. Those aurora repos appear to be empty :/ Is aurora intended to be a GUI toolkit? Only DirectX has code so far from what I've heard and seen. I still think there's huge need for a realtime gfx environment GUI toolkit. I'm completely on board with that, but I really don't have the skills for it. But hey we also need a web service framework aimed at enterprise in D and that's my target work now days. But if I were you, I wouldn't be marrying it to any particular gfx API. Just make sure to structure the API to allow backends (like OpenGL) to be plugged in. It was designed to be completely independent of any 3d api. In theory you could have a different implementation of controls to a window. But I never bothered going there considering. In fact I had plans to make it like GWT does as another backend and hooking it up to Vibe. [0] https://github.com/rikkimax/DOOGLE [1] https://github.com/auroragraphics [2] https://drive.google.com/file/d/0B-EiBquZktsLcGtMSmtMS1ZoZHM/edit?usp=sharing
Re: OpenGL Examples in D and a birth of a New Initiative
On Thu, May 15, 2014 at 7:22 PM, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 1:38 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 23:24, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 12:07 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm? DOOGLE[0] was my attempt at a gui toolkit. And aurora is this[1]. I would have replied earlier but it appears that there has been enough changes with dub and dmd that it just won't. But I managed to grab an old build and here is its output [2]. Not really that nice but hey, was my go at it. I was building it as pretty much the all in one toolbox for D and OpenGL and with Aurora getting more support, the issues I was having continuing just kinda was the last straw. And anyways I'm better at web development then game dev. Never really understood the 3d pipeline. Those aurora repos appear to be empty :/ Is aurora intended to be a GUI toolkit? Only DirectX has code so far from what I've heard and seen. I still think there's huge need for a realtime gfx environment GUI toolkit. I'm completely on board with that, but I really don't have the skills for it. But hey we also need a web service framework aimed at enterprise in D and that's my target work now days. Interesting. What are you working on? Is it on github? But if I were you, I wouldn't be marrying it to any particular gfx API. Just make sure to structure the API to allow backends (like OpenGL) to be plugged in. It was designed to be completely independent of any 3d api. In theory you could have a different implementation of controls to a window. But I never bothered going there considering. In fact I had plans to make it like GWT does as another backend and hooking it up to Vibe. [0] https://github.com/rikkimax/DOOGLE [1] https://github.com/auroragraphics [2] https://drive.google.com/file/d/0B-EiBquZktsLcGtMSmtMS1ZoZHM/ edit?usp=sharing
10 Years of Derelict
I managed to let this little anniversary slip by me. My first post in the old Derelict forum at DSource[1] is dated May 6, 2004, my initial commit to svn[2] was May 7, and my announcement in the newsgroup[3] was on May 8. My attention to the project has waxed and waned, but I'm still puttering along. I expect to continue for the forseeable future. There have been a number of contributors over the years who have helped out in porting to Linux and Mac, fixing bugs, and keeping the bindings up to date. It's been very much a community project, which has made it more enjoyable to work on. It helps that it isn't difficult to maintain :) Maybe before the next decade goes by I'll actually finish a game in D, which is the reason I started Derelict in the first place. For those who don't know, the current iteration of Derelict can be found at the DerelictOrg group at github[4]. I've still got work to do on it (ahem, documentation), but it's coming along. [1] http://www.dsource.org/forums/viewtopic.php?t=105 [2] http://www.dsource.org/projects/derelict/changeset/5/ [3] http://forum.dlang.org/thread/c7hl51$2k4u$1...@digitaldaemon.com [4] https://github.com/DerelictOrg
Re: OpenGL Examples in D and a birth of a New Initiative
On Thursday, 15 May 2014 at 14:55:46 UTC, Adil Baig via Digitalmars-d-announce wrote: On Thu, May 15, 2014 at 7:22 PM, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 1:38 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 23:24, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 16/05/2014 12:07 a.m., Manu via Digitalmars-d-announce wrote: On 15 May 2014 16:30, Rikki Cattermole via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 15/05/2014 5:01 p.m., Manu via Digitalmars-d-announce wrote: I tried to gather support for a community game project (FeedBack). Lots of interest, but nobody actually joined the party when I kicked it off. Yeah I was one of those people who wanted to work on it. Unfortunately that was while I was working on DOOGLE and pre aurora commencing. That to me really killed my enthusiasm. Personally I'm waiting for aurora before starting up on the GUI front. What's doogle and aurora? And why did that kill enthusiasm? DOOGLE[0] was my attempt at a gui toolkit. And aurora is this[1]. I would have replied earlier but it appears that there has been enough changes with dub and dmd that it just won't. But I managed to grab an old build and here is its output [2]. Not really that nice but hey, was my go at it. I was building it as pretty much the all in one toolbox for D and OpenGL and with Aurora getting more support, the issues I was having continuing just kinda was the last straw. And anyways I'm better at web development then game dev. Never really understood the 3d pipeline. Those aurora repos appear to be empty :/ Is aurora intended to be a GUI toolkit? Only DirectX has code so far from what I've heard and seen. I still think there's huge need for a realtime gfx environment GUI toolkit. I'm completely on board with that, but I really don't have the skills for it. But hey we also need a web service framework aimed at enterprise in D and that's my target work now days. Interesting. What are you working on? Is it on github? It is indeed and on dub repository. https://github.com/rikkimax/Cmsed https://github.com/rikkimax/Dvorm
Re: 10 Years of Derelict
Am 15.05.2014 17:02, schrieb Mike Parker: I managed to let this little anniversary slip by me. My first post in the old Derelict forum at DSource[1] is dated May 6, 2004, my initial commit to svn[2] was May 7, and my announcement in the newsgroup[3] was on May 8. My attention to the project has waxed and waned, but I'm still puttering along. I expect to continue for the forseeable future. There have been a number of contributors over the years who have helped out in porting to Linux and Mac, fixing bugs, and keeping the bindings up to date. It's been very much a community project, which has made it more enjoyable to work on. It helps that it isn't difficult to maintain :) Maybe before the next decade goes by I'll actually finish a game in D, which is the reason I started Derelict in the first place. For those who don't know, the current iteration of Derelict can be found at the DerelictOrg group at github[4]. I've still got work to do on it (ahem, documentation), but it's coming along. [1] http://www.dsource.org/forums/viewtopic.php?t=105 [2] http://www.dsource.org/projects/derelict/changeset/5/ [3] http://forum.dlang.org/thread/c7hl51$2k4u$1...@digitaldaemon.com [4] https://github.com/DerelictOrg Congratulations! I've been using it now since about seven years and it has been a great help to get started.
Re: Livestreaming DConf?
On Saturday, 10 May 2014 at 17:06:47 UTC, Joakim wrote: I demand a telehuman stream: http://youtube.com/watch?v=06tV60K-npw Facebook has one of those, right? ;) Haha, I'm disappointed they didn't go all the way back to landline.
Re: Livestreaming DConf?
On 5/15/2014 1:33 PM, Jesse Phillips wrote: On Saturday, 10 May 2014 at 17:06:47 UTC, Joakim wrote: I demand a telehuman stream: http://youtube.com/watch?v=06tV60K-npw Facebook has one of those, right? ;) Haha, I'm disappointed they didn't go all the way back to landline. Would've ruined the joke. Landline would have just worked :)
Re: OpenGL Examples in D and a birth of a New Initiative
On 5/15/14, Manu via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: You never mentioned that you were deterred by performance, I can go and make it run at a thousand fps if you like :) Bah I feel terrible for even saying what I said. I tested it on a 9600GT (yeah it's old, but UT3 worked fine on it, so.. :) ). But I own a Radeon R7 250 now and it's running at a solid 60FPS, no framedrops at all. And it's actually displaying some graphics that weren't there before. I think I may just have had some buggy drivers on my old GPU. It's running beautifully, playing the Bohemian Rhapsody. So, apologies for that rant. Maybe I'll add some silly graphics in a pull request and you won't say nobody contributed! Stay tuned. :)
Re: Livestreaming DConf?
On 09/05/2014 8:48 PM, Andrei Alexandrescu wrote: Hi folks, We at Facebook are very excited about the upcoming DConf 2014. In fact, so excited we're considering livestreaming the event for the benefit of the many of us who can't make it to Menlo Park, CA. Livestreaming entails additional costs so we're trying to assess the size of the online audience. Please follow up here and on twitter: https://twitter.com/D_Programming/status/464854296001933312 Thanks, Andrei I'd watch a livestream, if the time difference doesn't get in the way that is A...
Re: Livestreaming DConf?
On 5/12/14, simendsjo via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 05/11/2014 11:15 PM, John Carter via Digitalmars-d-announce wrote: The timezones being so different, I'm not sure livestreaming will help me... But I do plan to watch any videos of it as soon as they are up. It's the same problem for all Europeans (and many other), including me. I'm also looking forward to the videos, but I think livestreaming is a good thing for those who are able to follow them. I don't think I follow. The conference starts at 9 AM and ends at 4:30 PM in San Francisco time. Converted to GMT this is 4 PM to 11:30 PM. That should be super-convenient for most Europeans.
Regal: An SQL relational algebra builder
Hi all, I'd like to announce the initial version of Regal, an SQL relational algebra builder for D. It's intended as a backbone for a relational database ORM, in line with how Arel works with Rails' ActiveRecord, but nearly any project that generates SQL dynamically can benefit from it. The goal of the library is to make programmatically generating queries simple (and more robust than simple string concatenation), and to be database agnostic (it's very easy to write database specific printers for a Regal AST). There are extensive examples available in the Readme about how to use the library, from generating very simple queries (SELECT * FROM users) to multiple chained joins and nested constraints. Here's an example from the documentation: finding a user with ID 1: ``` auto users = new Table( users, // table name id,// the rest are columns name, created_at, updated_at); // SELECT * FROM users WHERE users.id = 1 LIMIT 1 users .where(users.id.eq(1)) .limit(1) .project(new Sql(*)) .to_sql ``` The library can be found at: https://github.com/dymk/regal And the dub package: http://code.dlang.org/packages/regal Please let me know if you find any bugs via the Github tracker! Regards, Dylan
Re: OpenGL Examples in D and a birth of a New Initiative
On 16 May 2014 04:39, Andrej Mitrovic via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: On 5/15/14, Manu via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: You never mentioned that you were deterred by performance, I can go and make it run at a thousand fps if you like :) Bah I feel terrible for even saying what I said. I tested it on a 9600GT (yeah it's old, but UT3 worked fine on it, so.. :) ). But I own a Radeon R7 250 now and it's running at a solid 60FPS, no framedrops at all. And it's actually displaying some graphics that weren't there before. I think I may just have had some buggy drivers on my old GPU. It's running beautifully, playing the Bohemian Rhapsody. So, apologies for that rant. Maybe I'll add some silly graphics in a pull request and you won't say nobody contributed! Stay tuned. :) Okay... well, I'm sure it does perform terrible anyway, I've done literally no work on rendering. It's just hacky 2 minute throw-away rendering code to display something on the screen. Out of curiosity, are you running Linux or Windows? I barely test the linux build (I just got it to work), so I can't really comment on it's performance. If a linux user would like to hit it with a profiler and see if I've done anything stupid in the linux code, that'd be useful. The linux build also needs better input and midi code to be properly playable.
Re: hijackable/customizable keyword for solving the customized algorithm issue?
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote: On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote: UFCS only apply to the method call style, not the the function call style, so it's not a matter of priority here - `foo(myObject)` will not call the `foo.myObject()` method even if there is no `foo` function(in that case it'll just fail). Having the function call style always defer to a member function is a really bad idea - and not just because of code breakage. It'll make it impossible to call a function on an object that has a method of the same name unless you use an alias or a function variable to copy the function - but then you have to make sure the alias\variable name is also not taken! This will be a disaster for anyone who wants to build or use a D library that uses templates. Well, how common can that case be? That's exactly what the proposed @hijackable annotation on a function would do, if I understand it correctly. Right. The idea though is not to customize the behavior of the algorithm, but rather provide a more efficient implementation. It shouldn't really break any existing code. In any case, it should not cause any more than the breakage that using UFCS *already* causes... this would just (opt-in by the implementation) make sure you get the same behavior with/without UFCS. FYI, there are already a few functions in phobos that delegate to member functions if they exist, and do something generic if they don't. Take and put are some of the more trivial examples. move will also delegate to proxyMove. At one point, there were also talks of doing the same for popFrontN too. The issue though is that it's all very hackish, and it makes the implementation jump through hoops to achieve said result, and it only works if the implementation pre-emptivelly thought about doing so.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 05:51:16 UTC, Ola Fosheim Grøstad wrote: However mmap to a fixed address is pure if it throws an exception on failure because the exception bypass the call site of the pure function (pure functions should not catch side effect exceptions). This of course presumes a transactional view, that the transaction is the context of purity and that flaws in purity unwinds all changes and thus abort the transaction (the try block). If the result of a series of pure function calls can be used to flip a coin within a transaction, then those functions cannot be considered pure in any meaningful sense.
Re: DFL is really cool,Who can contact Christopher E. Miller?
On Thursday, 15 May 2014 at 04:53:01 UTC, dennis luehring wrote: Am 15.05.2014 05:58, schrieb FrankLike: 1.DFL's Memory Usage is the least than other. winsamp.exe is 2.1M,DFL's example's exe is 2.7M. 2.The size of DFL's example's exe files is the least than other, and only a single file. 3.DFL's source code is the most easy to understand. D need Christopher E. Miller. and what should happen then? he seems to lost interest long time ago and there are some forks of the project on github - so why do D need Christopher E. Miller in person? The DFL License Agreement DFL is Copyright (C) 2004-2010 Christopher E. Miller If he has some intersting to DFL,I can do something for DFL,ok? Thank you.
Re: Memory allocation purity
Ola Fosheim Grøstad: Pure in D seems pointless to me. If you start using pure in D you see it's like const: it allows you to remove certain kinds of your mistakes from the code, and it makes it more easy to reason about the code. You can use mutability inside a strongly pure function. This is a very good. Bye, bearophile
Re: Memory allocation purity
Ola Fosheim Grøstad:
If the result of a series of pure function calls can be used to
flip a coin within a transaction, then those functions cannot
be considered pure in any meaningful sense.
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
void main() {}
Bye,
bearophile
Re: More radical ideas about gc and reference counting
On 14/05/14 23:47, Dmitry Olshansky wrote: This is curious: http://burntsushi.net/rustdoc/regex/ It seems they have compile time regular expressions too. -- /Jacob Carlborg
Re: Memory allocation purity
On Thursday, 15 May 2014 at 06:24:30 UTC, bearophile wrote: If you start using pure in D you see it's like const: it allows you to remove certain kinds of your mistakes from the code, and it makes it more easy to reason about the code. As lint like functionality, yes. You can use mutability inside a strongly pure function. This is a very good. Local mutability does not affect purity, it is the pre and post conditions at the call site that matters.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with
private globals. Because memoing is no longer sound anyway.
Re: Memory allocation purity
On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis
Re: Bug in Phobos' process.d: Executable file not found is supposed to show executabloe name but fails
On Wednesday, 14 May 2014 at 13:02:27 UTC, Van de Bugger wrote: I do not see Issues tab on page https://github.com/D-Programming-Language/phobos Did you read the project's readme?
Re: Memory allocation purity
On Thursday, 15 May 2014 at 06:59:08 UTC, Jonathan M Davis via Digitalmars-d wrote: And it _definitely_ has nothing to do with functional purity. Which makes it pointless and misleading. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. No, you can't say it is functionally pure if you can flip a coin with a pure function. To do that you would need a distinction between prove pure and assume pure as well as having immutable reference types that ban identity comparison. So, no, purity does _not_ imply memoization. It should, or use a different name.
Re: radical ideas about GC and ARC : need to be time driven?
On Wednesday, 14 May 2014 at 09:39:01 UTC, Marc Schütz wrote: RC is done by the object itself, so by definition it knows its own type, while the GC needs to be told about the type on allocation. AFAIK there is ongoing work to make this information available for non-class types. If you can unify RC on binary level for any type, GC can use that unification too: when you allocate the object, you has its type and can setup necessary structures needed to call the destructor. Well, it cannot be made 100% reliable by principle. That's just an inherent property of tracing GCs. The question is, can we define which uses of destructors are safe in this sense and which ones are not, and ideally find ways to detect unsafe uses at compile time... That's very much in the spirit of D: Something that looks right, should be right. If it is not, it should be rejected by the compiler. Does this suggest that if you slip a type with destructor into your code, it will force everything to be refcounted?
Re: Memory allocation purity
And mutating through parameters does not affect functional purity in the theoretical sense if the function holds the only reference to it. It is comparable to taking one value and returning a new version of it. Rigor is important in language design. If you cannot say ALWAYS, then the compiler will have to assume NEVER.
Re: New opportunities for D = ASM.js
On Wednesday, 14 May 2014 at 20:50:47 UTC, deadalnix wrote: On Tuesday, 13 May 2014 at 17:16:18 UTC, Etienne wrote: I'd like to point out that asm.js is a very fast subset of the javascript language that allows almost native speeds (3x slowdown vs C only) which enables games to be run in the browser without external dependencies. You keep saying the browser, but what you mean is firefox (and other mozilla products). For the story, mozilla dropped out of the NaCl project so they can pull out their me too solution. Now we are back to where we were 10 years ago with the browser war. We could have one unified standard int he name of NaCl, but fuck that, now we have too. ASM.js is inferior in every possible way (slower, bigger source, more overhead, you name it) to NaCl except one: ASM.js run in a standard JS interpreter. Except that if you are using this, it is because you need the speed in the first place. To take the example of video game, can you claim that the game works when you run it at 0.5fps ? Sadly, because of mozilla moves with ASM.js, there is no industry standard to run native things in the browser. Until things settle down, these are cool, but useless technologies. That is what the desktop is for anyway.
Re: New opportunities for D = ASM.js
On Wednesday, 14 May 2014 at 20:50:47 UTC, deadalnix wrote: On Tuesday, 13 May 2014 at 17:16:18 UTC, Etienne wrote: I'd like to point out that asm.js is a very fast subset of the javascript language that allows almost native speeds (3x slowdown vs C only) which enables games to be run in the browser without external dependencies. You keep saying the browser, but what you mean is firefox (and other mozilla products). For the story, mozilla dropped out of the NaCl project so they can pull out their me too solution. Now we are back to where we were 10 years ago with the browser war. We could have one unified standard int he name of NaCl, but fuck that, now we have too. ASM.js is inferior in every possible way (slower, bigger source, more overhead, you name it) to NaCl except one: ASM.js run in a standard JS interpreter. Except that if you are using this, it is because you need the speed in the first place. To take the example of video game, can you claim that the game works when you run it at 0.5fps ? Sadly, because of mozilla moves with ASM.js, there is no industry standard to run native things in the browser. Until things settle down, these are cool, but useless technologies. With their new LLVM JIT compiler, Webkit has gained optimizations that, as side effect, also benefit asm.js code patterns. https://www.webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ However, I think the place of native applications, is on native platforms. As mentioned on another reply. -- Paulo
Re: DFL is really cool,Who can contact Christopher E. Miller?
On 5/15/2014 3:19 PM, FrankLike wrote: The DFL License Agreement DFL is Copyright (C) 2004-2010 Christopher E. Miller If he has some intersting to DFL,I can do something for DFL,ok? Thank you. You don't need him to do anything with DFL. It's released under an opens source license, so as long as you follow the terms of the license you can do what you want with it.
Re: Memory allocation purity
On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is.
Re: New opportunities for D = ASM.js
On Thursday, 15 May 2014 at 05:28:56 UTC, Nick Sabalausky wrote: While Unity3D provides a great boost in cross-platform...ability, they've recently made it very clear they have absolutely no intention of ever supporting any Chrome-only technology. (And on top of that, even their http://docs.unity3d.com/Documentation/Manual/nacl-gettingstarted.html there. So, unfortunately, deadalnix is basically right: These things are pretty much worthless for right now, until everything bakes a bit more. How much are you willing to bet? :-) You need a modern PC/mac, but gamers tend to be on that frontier...
Re: Memory allocation purity
On Thu, 15 May 2014 07:22:02 +
via Digitalmars-d digitalmars-d@puremagic.com wrote:
On Thursday, 15 May 2014 at 06:59:08 UTC, Jonathan M Davis via
Digitalmars-d wrote:
And it _definitely_ has nothing to do with functional purity.
Which makes it pointless and misleading.
Now, combined with other information, you _can_ get functional
purity out it -
e.g. if all the parameters to a function are immutable, then it
_is_
functionally pure, and optimizations requiring functional
purity can be done
with that function.
No, you can't say it is functionally pure if you can flip a coin
with a pure function. To do that you would need a distinction
between prove pure and assume pure as well as having
immutable reference types that ban identity comparison.
So, no, purity does _not_ imply memoization.
It should, or use a different name.
Originally, pure required that the function parameters be pure in addition to
disallowing the function from accessing global or static variables or calling
functions that weren't pure. It allowed for mutation within the function, and
it allowed for allocation via new, but from the outside, the function _was_
functionally pure. The problem was that it was almost useless. You just
couldn't do anything in a pure function that mattered most of the time. You
couldn't call any other functions from the pure function unless they were
pure, which meant that the arguments to them had to be immutable, which just
didn't work, because while the arguments to the first function were immutable,
what it had to do internally often involved operating on other variables which
were created within the function which were not immutable and didn't need to
be immutable, but you couldn't use them with any functions unless they were
immutable thanks to the fact that all pure functions had to have immutable
parameters, and pure functions could only call pure functions. It just didn't
work.
So, Don introduced the idea of weak purity. What it comes down to is that
it's an extension of the concept that mutation within a pure function is fine
just so long as its arguments aren't mutated. We made it so that pure
functions _didn't_ have to have immutable parameters. They just couldn't
access anything that wasn't passed to them as arguments. This meant that they
could only mutate what they were given and thus they didn't violate the
strong purity of the original pure function which had immutable parameters.
e.g.
string strongFunc(immutable string foo, int i) pure
{
auto foo = str ~ hello world ;
weak(str, i);
return str;
}
void weakFunc(ref string str, int i) pure
{
foreach(j; 0 .. i)
str ~= to!(j);
}
The strong guarantees that strongFunc has which make it functionally pure are
not violated by the fact that weakFunc is _not_ functionally pure. But by
marking it pure, it guarantees that it can safely be called from a strongly
pure function without violating the guarantees of that strongly pure function.
To do that, _all_ we need to guarantee is that the weakly pure function cannot
access anything save for what's passed into it (since if it could access
global variables, that would violate the guarantees of any other pure
functions that called it), but we do need that guarantee. The result is that
the pure attribute doesn't in and of itself mean functional purity anymore,
but it _can_ be used to build a function which is functionally pure.
You could argue that a different attribute should be used other than pure to
mark weakly pure functions, but that would just complicate things. The
compiler is capable of figuring out the difference between a weakly pure and
strongly pure function on its own from just the function signature just so
long as weak purity is detectable from the function signature. So, we only
need one attribute - one to mark the fact that the function can't access
global, mutable state and can't call any functions that can. And we were
already marking strongly pure functions with pure, so it made perfect sense to
use it on weakly pure functions as well. At that point, it was just up to the
compiler to detect whether the function was strongly pure or not and thus was
functionally pure and could be used in optimizations.
So, sorry that it offends your sensibilities that pure by itself does not
indicate functional purity, but it's a building block for functional purity,
and the evolution of things made it make perfect sense to use the pure
attribute for this. And even if pure _didn't_ enable functional purity, it
would still be highly useful just from the fact that a pure function (be it
weak or strong) cannot access global variables, and that makes it _much_
easier to reason about code, because you know that it isn't accessing anything
that wasn't passed to it.
I recommend that you read this article by David Nadlinger:
http://klickverbot.at/blog/2012/05/purity-in-d/
- Jonathan M Davis
Re: Memory allocation purity
On Thu, 15 May 2014 10:14:48 +0200 luka8088 via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. The reread the paragraph at the top of the section of the documentation that you linked to: Pure functions are functions which cannot access global or static, mutable state save through their arguments. This can enable optimizations based on the fact that a pure function is guaranteed to mutate nothing which isn't passed to it, and in cases where the compiler can guarantee that a pure function cannot alter its arguments, it can enable full, functional purity (i.e. the guarantee that the function will always return the same result for the same arguments). That outright says that pure only _can_ enable functional purity - in particular when the compiler is able to guarantee that the function cannot mutate its arguments. pure itself however means nothing of the sort. The fact that pure functions cannot access global state _is_ the basis for functional purity when combined with parameters that arguments cannot be mutated. If you get hung up on what the concept of functional purity is or what you thought pure was before using D, then you're going to have a hard time understanding what pure means in D. And yes, it's a bit weird, but it comes from the practical standpoint of how to make functional purity possible without being too restrictive to be useful. So, it really doesn't matter what other sources say about what purity means. That's not what D's pure means. D's pure is just a building block for what purity normally means. It makes it so that the compiler can detect functional purity and then optimize based on it, but it doesn't in and of itself have anything to do with functional purity. If the documentation isn't getting that across, then I guess that it isn't clear enough. But I would have thought that the part that said and in cases where the compiler can guarantee that a pure function cannot alter its arguments, it can enable full, functional purity would have made it clear that D's pure is _not_ functionally pure by itself. The first part of the paragraph says what pure really means: Pure functions are functions which cannot access global or static, mutable state save through their arguments. Everything else with regards to functional purity is derived from there, but in and of itself, that's _all_ that the pure attribute in D means. See also: http://klickverbot.at/blog/2012/05/purity-in-d/ - Jonathan M Davis
Re: Memory allocation purity
On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back.
Re: Memory allocation purity
On 15.5.2014. 11:35, Jonathan M Davis via Digitalmars-d wrote:
On Thu, 15 May 2014 10:14:48 +0200
luka8088 via Digitalmars-d digitalmars-d@puremagic.com wrote:
On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote:
On Thu, 15 May 2014 05:51:14 +
via Digitalmars-d digitalmars-d@puremagic.com wrote:
Yep, purity implies memoing.
No, it doesn't. _All_ that it means when a function is pure is that
it cannot access global or static variables unless they can't be
changed after being initialized (e.g. they're immutable, or they're
const value types), and it can't call any other functions which
aren't pure. It means _nothing_ else. And it _definitely_ has
nothing to do with functional purity.
Now, combined with other information, you _can_ get functional
purity out it - e.g. if all the parameters to a function are
immutable, then it _is_ functionally pure, and optimizations
requiring functional purity can be done with that function. But by
itself, pure means nothing of the sort.
So, no, purity does _not_ imply memoization.
- Jonathan M Davis
Um. Yes it does. http://dlang.org/function.html#pure-functions
functional purity (i.e. the guarantee that the function will always
return the same result for the same arguments)
The fact that it should not be able to effect or be effected by the
global state is not a basis for purity, but rather a consequence.
Even other sources are consistent on this matter, and this is what
purity by definition is.
The reread the paragraph at the top of the section of the documentation
that you linked to:
Pure functions are functions which cannot access global or static,
mutable state save through their arguments. This can enable
optimizations based on the fact that a pure function is guaranteed to
mutate nothing which isn't passed to it, and in cases where the
compiler can guarantee that a pure function cannot alter its arguments,
it can enable full, functional purity (i.e. the guarantee that the
function will always return the same result for the same arguments).
That outright says that pure only _can_ enable functional purity - in
particular when the compiler is able to guarantee that the function
cannot mutate its arguments. pure itself however means nothing of the
sort. The fact that pure functions cannot access global state _is_ the
basis for functional purity when combined with parameters that
arguments cannot be mutated.
I am aware of weak/strong purity. I am only talking about strong purity now.
To quote bearophile:
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
void main() {}
Pure functions are functions which cannot access global or static,
mutable state save through their arguments. - no objections here
This can enable optimizations based on the fact that a pure function is
guaranteed to mutate nothing which isn't passed to it, and in cases
where the compiler can guarantee that a pure function cannot alter its
arguments, it can enable full, functional purity (i.e. the guarantee
that the function will always return the same result for the same
arguments). - no arguments where passed to the function, it should
always return the same result
If you get hung up on what the concept of functional purity is or what
you thought pure was before using D, then you're going to have a hard
time understanding what pure means in D. And yes, it's a bit weird, but
it comes from the practical standpoint of how to make functional purity
possible without being too restrictive to be useful. So, it really
doesn't matter what other sources say about what purity means. That's
not what D's pure means. D's pure is just a building block for what
purity normally means. It makes it so that the compiler can detect
functional purity and then optimize based on it, but it doesn't in and
of itself have anything to do with functional purity.
If the documentation isn't getting that across, then I guess that it
isn't clear enough. But I would have thought that the part that said
and in cases where the compiler can guarantee that a pure function
cannot alter its arguments, it can enable full, functional purity
would have made it clear that D's pure is _not_ functionally pure by
itself. The first part of the paragraph says what pure really means:
Pure functions are functions which cannot access global or static,
mutable state save through their arguments. Everything else with
regards to functional purity is derived from there, but in and of
itself, that's _all_ that the pure attribute in D means.
See also: http://klickverbot.at/blog/2012/05/purity-in-d/
- Jonathan M Davis
Yeah, it does not seem to be clear enough.
It comes down to two simple questions:
- should you be able to make a strong pure rand() function i D?
- if not, why?
My answer would be: no, because the result should always be the same.
Of course I could be wrong, but my understanding of it fits with
Re: Memory allocation purity
On Thursday, 15 May 2014 at 09:23:00 UTC, Jonathan M Davis via Digitalmars-d wrote: functions that weren't pure. It allowed for mutation within the function, and it allowed for allocation via new, but from the outside, the function _was_ functionally pure. If it didn't return the memory allocated with new and if the call to new resulted in an exception, yes. It just didn't work. That I question. A pure function ( http://en.wikipedia.org/wiki/Pure_function ) depends on the values of the parameters, and only that. That is most useful. Those value can be very complex. You could have a pure member function look up values in a cache. Then the configuration of entire cache is the value. You need to think about this in terms of pre/post conditions in Hoare Logic (which I am not very good at btw). So, Don introduced the idea of weak purity. What it comes down to is that it's an extension of the concept that mutation within a pure function is fine just so long as its arguments aren't mutated. We made it so that pure functions _didn't_ have to have immutable parameters. They just couldn't access anything that wasn't passed to them as arguments. This meant that they could only mutate what they were given and thus they didn't violate the strong purity of the original pure function which had immutable parameters. And that's fine as long as nobody else is holding a reference to those mutable parameters. That means that you are taking version N of the mutable and returning version N+1. That's similar to x=1 a =f(x) x=x+1 b = f(x) which can be rewritten as: x0 =1 a = f(x0) x1 = x0+1 b = f(x1) If you think in terms of a context for purity such as a transaction then you can even allow access to globals as long as they remain constant until the transaction is committed (or you leave the context where purity is desired). Meaning, you can memoize within that context. functions that called it), but we do need that guarantee. The result is that the pure attribute doesn't in and of itself mean functional purity anymore, but it _can_ be used to build a function which is functionally pure. But, that can be deduced by the compiler, so what is the point of having pure for weakly pure? Clearly you only need to specify strongly pure? So, sorry that it offends your sensibilities that pure by itself does not indicate functional purity, but it's a building block for functional purity, It doesn't offend me, but it is a source of confusion and not sticking to definitions makes the language design look random. The same thing goes for lazy parameters. Why pick Call-By-Name (Algol style) over Call-By-Need (memoing)? Call-By-Name is known to be error prone. Actually, lazy parameters should be restricted to pure expressions… if correctness and safety is the goal. attribute for this. And even if pure _didn't_ enable functional purity, it would still be highly useful just from the fact that a pure function (be it weak or strong) cannot access global variables, and that makes it _much_ easier to reason about code, because you know that it isn't accessing anything that wasn't passed to it. Ok, but maybe the opposite would be better. Marking functions that access globals with @global or something. After all, most functions don't access globals.
Re: Memory allocation purity
On 15.5.2014. 11:45, Don wrote: On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back. Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct?
Re: Memory allocation purity
On Thursday, 15 May 2014 at 10:31:47 UTC, luka8088 wrote: On 15.5.2014. 11:45, Don wrote: On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back. Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct? Yes. 'strong pure' means pure in the way that the functional language crowd means 'pure'. 'weak pure' just means doesn't use globals. But note that strong purity isn't an official concept, it was just the terminology I used when explain to Walter what I meant. I don't like the term because it's rather misleading -- in reality you could define a whole range of purity strengths (more than just two). The stronger the purity, the more optimizations you can apply.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 09:45:52 UTC, Don wrote: But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Uhm. That is a pretty strong assumption. memoizable is very useful property when you do multihreading, transactions or anything that requires locking. And you can still access globals, you just need a guarantee that globals don't change until you are done. Considering that 90% of the functions I write don't do IO or globals I'd rather specify the opposite. io, global whatever. That is also easy to enforce, i.e. you don't get to access IO/globals if you don't annotate the function.
Re: hijackable/customizable keyword for solving the customized algorithm issue?
On 5/14/14, 3:05 PM, monarch_dodra wrote: A little while ago, Steven Schveighoffer started a thread called D UFCS anti-pattern. The conversation started in regards to how can a generic algorithm be replaced by a custom type. In particular (in my original use case) retro The issue just came up again in .learn for .find. It comes up regularly with .put. To put it simply, the problem is given the function std.foo, how can I implement myObject.foo, and make sure that that is called when we call foo? That's the big problem with UFCS. If you were forced to always use UFCS then this wouldn't be a problem: if the type at question provides a more specific implementation *that matches the arguments' types*, the compiler takes it. Otherwise, it checks for free functions that have that type as a first argument and the rest of the arguments. But once you have an option to invoke it as a free function you loose. The U of UFCS means uniform, so in my opinion when you do: foo(x, y); then the compiler must first check if the type of x defines foo(y). If so, then it invokes that specialized version. If not, it must check if foo(x, y) exists somewhere else. If you do: x.foo(y); the compiler would do exactly the same. This is uniformity: either way I invoke a function, the compiler's logic is the same. But if you can write a same thing in two different ways but behave differently, that's looking for trouble. If you don't want the compiler to convert foo(x, y) to x.foo(y), then use a qualified name: bar.baz.foo(x, y);
Re: Next step on reference counting topics
I've been trying to think of a solution to use over here at Remedy for making the garbage collector reference count allocations instead of the current scan method (even with Rainer Schütze's GC it still does a scan, and I'd feel much more comfortable not having to schedule a GC collection somewhere in our update loop). I've totally missed the other threads on reference counting, but should reference counting be a part of some library that you can optionally use or should the garbage collector hide all that from you? My thinking on the subject is that you only need one additional function added to the garbage collector that handles pointer assignments. The parameters are a ref to the pointer you're assigning to, and the new pointer. The GC would then decrement the reference count for the memory referenced by the pointer you're assigning to, and increment the reference count fr the new memory referenced. This, I imagine, would fit neatly in with slices and classes equally. It would also be optional - the current default GC wouldn't do anything with the function and . The downside is that a pointer assign would invoke a GC call and that it would still need to deal with circular dependencies in some fashion (however dealing with that at an allocation level rather than a class level seems potentially less problematic to me). If this is beyond the scope of this thread, I'll start up a new one to discuss all the potential pitfalls and problems that could be encountered. I haven't had any time to actually do research in to it or trial it yet.
Re: Memory allocation purity
On Thu, 15 May 2014 10:10:57 + via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thursday, 15 May 2014 at 09:23:00 UTC, Jonathan M Davis via Digitalmars-d wrote: functions that weren't pure. It allowed for mutation within the function, and it allowed for allocation via new, but from the outside, the function _was_ functionally pure. If it didn't return the memory allocated with new and if the call to new resulted in an exception, yes. It just didn't work. That I question. A pure function ( http://en.wikipedia.org/wiki/Pure_function ) depends on the values of the parameters, and only that. That is most useful. Those value can be very complex. You could have a pure member function look up values in a cache. Then the configuration of entire cache is the value. You need to think about this in terms of pre/post conditions in Hoare Logic (which I am not very good at btw). So, Don introduced the idea of weak purity. What it comes down to is that it's an extension of the concept that mutation within a pure function is fine just so long as its arguments aren't mutated. We made it so that pure functions _didn't_ have to have immutable parameters. They just couldn't access anything that wasn't passed to them as arguments. This meant that they could only mutate what they were given and thus they didn't violate the strong purity of the original pure function which had immutable parameters. And that's fine as long as nobody else is holding a reference to those mutable parameters. That would only matter if the compiler were trying to optimize based on pure functions with mutable parameters. It doesn't. And it would actually be very difficult for it to do so without doing full program optimization, which really doesn't work with the C linking model that D uses. The fact that we have thread-local by default helps, but it's not enough for more than very simple cases. The compiler doesn't care about optimizing weakly pure functions. The whole purpose of weakly pure functions is to have functions which aren't functionally pure but can still be used in functions that _are_ functionally pure. If you think in terms of a context for purity such as a transaction then you can even allow access to globals as long as they remain constant until the transaction is committed (or you leave the context where purity is desired). Meaning, you can memoize within that context. That doesn't work with D's model, because it doesn't have any concept of transactions like that. It also doesn't really have any concept of memoization either. The most that it does that is anything like memoizaton is optimize away multiple calls to a function within a single expression (or maybe within a statement - I don't remember which). So, auto y = sqrt(x) * sqrt(x); might become something more like auto temp = sqrt(x); y = x * x; but after that, the result of sqrt(x) is forgotten. So, in reality, the optimization gains from strongly pure functions are pretty minimal (almost non-existent really). If we were willing to do code flow analysis, we could probably make more optimizations (assuming that the exact same function call was made several times within a single function, which isn't all that common anyway), but Walter is generally against doing code flow analysis in the compiler due to the complications that it adds. We have some, but not a lot. The two main gains for purity are 1. being able to know for sure that a function doesn't access any global variables, which makes it easier to reason about the code. 2. being able to implicitly convert types to and from mutable, const, and immutable based on the knowledge that a particular value has to be unique. I'd say that functional purity was really the original goal of adding pure to the language, but it's really those two effects which have given us the most benefit. #2 in particular was unexpected, and the compiler devs keep finding new places that they can take advantage of it, which makes dealing with immutable a lot more pleasant - particularly when it comes to creating immutable objects that require mutation in order to be initialized properly. functions that called it), but we do need that guarantee. The result is that the pure attribute doesn't in and of itself mean functional purity anymore, but it _can_ be used to build a function which is functionally pure. But, that can be deduced by the compiler, so what is the point of having pure for weakly pure? Clearly you only need to specify strongly pure? It can't be deduced from the signature, and the compiler has to be able to know based only on the signature, because it doesn't necessarily have the source code for the function available. The only functions for which the compiler ever deduces anything from their bodies are templated functions, because it always has their bodies, and if it didn't do the attribute inference for you, you'd be forced
Re: Memory allocation purity
On Thursday, 15 May 2014 at 10:46:21 UTC, Ola Fosheim Grøstad wrote: On Thursday, 15 May 2014 at 09:45:52 UTC, Don wrote: But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Uhm. That is a pretty strong assumption. memoizable is very useful property when you do multihreading, transactions or anything that requires locking. It's useful, but it's not a deep property, and importantly, it isn't transient. The compiler can trivially work it out if it knows the function is @noglobal. And you can still access globals, you just need a guarantee that globals don't change until you are done. Sure, but how can the compiler statically check that? It's a tough problem. (That's not a rhetorical question, BTW. If you have a solution, that would be awesome). Considering that 90% of the functions I write don't do IO or globals I'd rather specify the opposite. io, global whatever. That is also easy to enforce, i.e. you don't get to access IO/globals if you don't annotate the function. I agree, I'd personally like to have an annotation '@global', and put 'pure:' at the top of every module.
Re: hijackable/customizable keyword for solving the customized algorithm issue?
On 5/15/14, 7:54 AM, Ary Borenszweig wrote: On 5/14/14, 3:05 PM, monarch_dodra wrote: A little while ago, Steven Schveighoffer started a thread called D UFCS anti-pattern. The conversation started in regards to how can a generic algorithm be replaced by a custom type. In particular (in my original use case) retro The issue just came up again in .learn for .find. It comes up regularly with .put. To put it simply, the problem is given the function std.foo, how can I implement myObject.foo, and make sure that that is called when we call foo? That's the big problem with UFCS. If you were forced to always use UFCS then this wouldn't be a problem: if the type at question provides a more specific implementation *that matches the arguments' types*, the compiler takes it. Otherwise, it checks for free functions that have that type as a first argument and the rest of the arguments. But once you have an option to invoke it as a free function you loose. The U of UFCS means uniform, so in my opinion when you do: foo(x, y); then the compiler must first check if the type of x defines foo(y). If so, then it invokes that specialized version. If not, it must check if foo(x, y) exists somewhere else. If you do: x.foo(y); the compiler would do exactly the same. This is uniformity: either way I invoke a function, the compiler's logic is the same. But if you can write a same thing in two different ways but behave differently, that's looking for trouble. If you don't want the compiler to convert foo(x, y) to x.foo(y), then use a qualified name: bar.baz.foo(x, y); Of course, the problem with this is that when you read this code: foo(x, y); to understand it, you must first check if there's a foo definition somewhere in your imported modules that matches. Otherwise, check if x's type defines it. The same is true for this: x.foo(y); To understand it, you must check if x's type defines a foo(y). If so, it's taken. Otherwise, check which of the functions in the imported modules matches foo(x, y). This makes it really hard to reason about code: everytime you see foo(x, y) you must remember which of the modules had that function foo, or if x defines foo. In OOP-like land, foo(x, y) always means: check the function foo(x, y). And x.foo(y) always means: check the member foo of x's type. No confusion at all for the reader. But foo(x, y) is almost never used (global functions), so you never have to remember where methods are located by heart. You just go to that type's code and find the method there.
Re: Memory allocation purity
Don: I'd personally like to have an annotation '@global', and put 'pure:' at the top of every module. I suggested a little more powerful @outer: https://d.puremagic.com/issues/show_bug.cgi?id=5007 Bye, bearophile
Re: Memory allocation purity
On Thu, 15 May 2014 10:48:07 + Don via Digitalmars-d digitalmars-d@puremagic.com wrote: Yes. 'strong pure' means pure in the way that the functional language crowd means 'pure'. 'weak pure' just means doesn't use globals. But note that strong purity isn't an official concept, it was just the terminology I used when explain to Walter what I meant. I don't like the term because it's rather misleading -- in reality you could define a whole range of purity strengths (more than just two). The stronger the purity, the more optimizations you can apply. Yeah, I agree. The problem is that it always seems necessary to use the terms weak pure to describe the distinction - or maybe I just suck at coming up with a better way to describe it than you did initially. Your recent post in this thread talking about @noglobal seems to be a pretty good alternate way to explain it though. Certainly, the term pure throws everyone off at first. - Jonathan M Davis
Re: Memory allocation purity
On 15.5.2014. 12:48, Don wrote: On Thursday, 15 May 2014 at 10:31:47 UTC, luka8088 wrote: On 15.5.2014. 11:45, Don wrote: On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back. Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct? Yes. 'strong pure' means pure in the way that the functional language crowd means 'pure'. 'weak pure' just means doesn't use globals. But note that strong purity isn't an official concept, it was just the terminology I used when explain to Walter what I meant. I don't like the term because it's rather misleading -- in reality you could define a whole range of purity strengths (more than just two). The stronger the purity, the more optimizations you can apply. Ok. Now it is much clearer, thanks.
Re: Memory allocation purity
On 15.5.2014. 13:04, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 10:48:07 + Don via Digitalmars-d digitalmars-d@puremagic.com wrote: Yes. 'strong pure' means pure in the way that the functional language crowd means 'pure'. 'weak pure' just means doesn't use globals. But note that strong purity isn't an official concept, it was just the terminology I used when explain to Walter what I meant. I don't like the term because it's rather misleading -- in reality you could define a whole range of purity strengths (more than just two). The stronger the purity, the more optimizations you can apply. Yeah, I agree. The problem is that it always seems necessary to use the terms weak pure to describe the distinction - or maybe I just suck at coming up with a better way to describe it than you did initially. Your recent post in this thread talking about @noglobal seems to be a pretty good alternate way to explain it though. Certainly, the term pure throws everyone off at first. - Jonathan M Davis Yeah, +1. Or @isolated, as in isolated from outer scopes.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 05:51:16 UTC, Ola Fosheim Grøstad wrote: On Thursday, 15 May 2014 at 02:49:28 UTC, Adam Sakareassen via Digitalmars-d wrote: The more D allows 'pure' functions to diverge from functional purity the less relevant the flag is for compiler optimisations. ... By the same reasoning cacheability is important. A pure function might be called within a loop with a parameter that is not altered during the loop. If the compiler knows the function is pure, it can perform the calculation before the loop and simply reuse the cached result for each iteration. Yep, purity implies memoing. Malloc and new are not pure because they return objects that can be differentiated by address. There's an important difference between malloc and new: malloc returns a pointer, but new returns a typed object. This is crucial IMO, because the returned objects are equal to each other. They aren't identical, but then different int variables with the same value aren't identical either, and a function returning int is still considered pure. So it's not identity (~ address) that matters. Pure in D seems pointless to me. Not at all: Don't think of it in terms of low-level optimization opportunities, but in terms of semantics. For example, you get the concept of uniqueness. And the optimizations can still be done, because strongly pure functions can be recognized by their signatures.
Re: Memory allocation purity
Marc Schütz: And the optimizations can still be done, because strongly pure functions can be recognized by their signatures. What optimizations do you think GDC compiler is doing (or will do) on pure functions? Bye, bearophile
Re: Memory allocation purity
On Thursday, 15 May 2014 at 11:03:35 UTC, Don wrote: And you can still access globals, you just need a guarantee that globals don't change until you are done. Sure, but how can the compiler statically check that? It's a tough problem. (That's not a rhetorical question, BTW. If you have a solution, that would be awesome). What you need is some way to formally specify what a lock covers or rather register what globals can be accessed in a transaction. So you basically need some sort of whole program analysis. With transactional memory you only take the lock if the transaction fails, so it is ok if locking is slow or to take multiple locks. The CPU reverts to regular mutex based locking and clears the cache if another thread touch the memory that has been used in the transaction. At least that is how I read the Haswell documentation. I agree, I'd personally like to have an annotation '@global', and put 'pure:' at the top of every module. It makes a lot of sense to put the annotation burden on the things you want less of, yes. Do I really need to make this @global? Maybe I should do it some other way…
Re: Memory allocation purity
On 15 May 2014 10:50, Walter Bright via Digitalmars-d digitalmars-d@puremagic.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Even if it doesn't fail, malloc called with identical arguments still returns a different result every time. You can't factor malloc outside a loop and cache the result because it's being called repeatedly with the same argument like you're supposed to be able to do with any other pure function. You shouldn't be able to do that with gcalloc either... how can gcalloc be pure?
Re: Memory allocation purity
On Thursday, 15 May 2014 at 11:35:46 UTC, bearophile wrote: Marc Schütz: And the optimizations can still be done, because strongly pure functions can be recognized by their signatures. What optimizations do you think GDC compiler is doing (or will do) on pure functions? I don't know whether it does or will do any. It is a theoretical option (can be done). It's the kind of optimizations Ole talked about that apply only to functionally pure functions. But the important point is that `new` can be pure, if you consider pure to be about equality, not identity. This applies only to typed allocators, not malloc.
Re: radical ideas about GC and ARC : need to be time driven?
On Wednesday, 14 May 2014 at 19:45:20 UTC, Marc Schütz wrote: - We have external code programmed in languages other than D, most prominently C and C++. These don't provide any type information, therefore the GC needs to handle their memory conservatively, which means there can be false pointers = no deterministic destruction. It's a very rare scenario to escape GC memory into foreign opaque data structures. Usually you don't see, where your pointer goes, as foreign API is usually completely opaque, so you have nothing to scan, even if you have a precise GC. Sometimes C API will notify your code, where it releases your data, in other cases you can store your data in a managed memory and release it after you release the foreign data structure. - Variables on the stack and in registers. In theory, the compiler could generate that information, or existing debug information might be used, but that's complicated for the GC to handle and will probably have runtime costs. I guess it's unlikely to happen. And of course, when we call a C function, we're lost again. Precise GC is needed to implement moving GC, it's not needed to implement good memory management, at least on 64-bit architecture. On 32-bit architecture false pointers are possible, when you have lots of data without pointers on 32-bit architecture. It could be treated simply by allocating data without pointers (like strings) in not scanned blocks. The more valid pointers you have in your data, the smaller is probability of false pointers. The smaller is handle wrapper, the smaller is probability of a false pointer holding it. If you manage resources well, probability of handle leak goes even smaller (in C# it doesn't pose any notable difficulty even though you don't have any mechanism of eager resource management at all, only GC, in D you have non-zero opportunity for eager resource management). All these small probabilities multiply, and you get even smaller probability of an eventual resource leak.
Re: hijackable/customizable keyword for solving the customized algorithm issue?
On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra monarchdo...@gmail.com wrote: move will also delegate to proxyMove. This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the hook function have a different name. I think the two mechanisms of overriding default behavior: 1. Use a hook, to define the basic minimum functionality. 2. Implement the same-named method, but you must implement ALL functionality (possibly deferring to the global function if necessary). -Steve
Re: Memory allocation purity
On Thursday, 15 May 2014 at 11:31:34 UTC, Marc Schütz wrote: There's an important difference between malloc and new: malloc returns a pointer, but new returns a typed object. This is crucial IMO, because the returned objects are equal to each other. I most code, but not all, so how does the compiler know if you don't have a reference type that explicitly bans identity comparison? If one requires value semantics it should also cover the reference. (Some programs allocate empty objects as enums.) optimization opportunities, but in terms of semantics. For example, you get the concept of uniqueness. And the I agree that uniqueness is an important property. I think Rust is onto something when they now want to rename mut to uniq or only. But in this case uniqueness is the problem with weakly pure, a problem that pure functions don't have.
Re: borrowed pointers vs ref
On Wednesday, 14 May 2014 at 19:03:20 UTC, Walter Bright wrote:
Also A can only control escaping of any internal references
only by completely
prohibiting access to it which is not good. You have no
means to say feel free
to use this reference as long as you don't keep it outside
of current scope.
And you effectively say make all your array members private
to keep borrowing
guarantees.
You can by only returning ref's.
Also slices and pointers (or structs with pointers inside).
The idea is that 'ref' are borrowed pointers, so if you're
returning pointers, the borrowed semantics do not apply.
Somewhat more extended example:
struct Buffer
{
private byte[] data;
this() { this.data = (cast(byte*)malloc(42))[0..42]; }
~this() { free(this.data.ptr); }
byte[] get(size_t a, size_t b) { return this.data[a..b]; }
}
void foo(ref Buffer buff)
{
// here be trouble
static int[] slice = buff.get(10, 20);
}
void foo2()
{
Buffer buff;
foo(buff);
// destructor gets called, foo now has pointer to freed memory
}
Transitivity of borrowing ensures that you can use any object as
an argument for function that takes a borrowed pointer and no
reference to its internals will persist. Whatever memory
management model of object type is.
With such borrowing implementation this example code is also
totally @safe in spirit (assignment to static var will result in
compile-time error).
Re: radical ideas about GC and ARC : need to be time driven?
On Thursday, 15 May 2014 at 07:27:41 UTC, Kagamin wrote: On Wednesday, 14 May 2014 at 09:39:01 UTC, Marc Schütz wrote: RC is done by the object itself, so by definition it knows its own type, while the GC needs to be told about the type on allocation. AFAIK there is ongoing work to make this information available for non-class types. If you can unify RC on binary level for any type, GC can use that unification too: when you allocate the object, you has its type and can setup necessary structures needed to call the destructor. Exactly. Well, it cannot be made 100% reliable by principle. That's just an inherent property of tracing GCs. The question is, can we define which uses of destructors are safe in this sense and which ones are not, and ideally find ways to detect unsafe uses at compile time... That's very much in the spirit of D: Something that looks right, should be right. If it is not, it should be rejected by the compiler. Does this suggest that if you slip a type with destructor into your code, it will force everything to be refcounted? Hmm... that's probably too strict. There are often non-critical resources that need to be released on destruction, like a hypothetical String class which owns it's data and which is itself allocated on the GC heap, because we don't need eager destruction for it. We'd want the data buffer to be released as soon the String object is destroyed. This buffer might even be allocated on the C heap, so we cannot rely on the garbage collector to clean it up later. Is that a job for a finalizer?
Re: radical ideas about GC and ARC : need to be time driven?
On Thursday, 15 May 2014 at 12:05:27 UTC, Kagamin wrote: On Wednesday, 14 May 2014 at 19:45:20 UTC, Marc Schütz wrote: - We have external code programmed in languages other than D, most prominently C and C++. These don't provide any type information, therefore the GC needs to handle their memory conservatively, which means there can be false pointers = no deterministic destruction. It's a very rare scenario to escape GC memory into foreign opaque data structures. Usually you don't see, where your pointer goes, as foreign API is usually completely opaque, so you have nothing to scan, even if you have a precise GC. Sometimes C API will notify your code, where it releases your data, in other cases you can store your data in a managed memory and release it after you release the foreign data structure. Fair point. But can this be made safer? Currently you don't get any warning if a GC pointer escapes into C land. - Variables on the stack and in registers. In theory, the compiler could generate that information, or existing debug information might be used, but that's complicated for the GC to handle and will probably have runtime costs. I guess it's unlikely to happen. And of course, when we call a C function, we're lost again. Precise GC is needed to implement moving GC, it's not needed to implement good memory management, at least on 64-bit architecture. On 32-bit architecture false pointers are possible, when you have lots of data without pointers on 32-bit architecture. It could be treated simply by allocating data without pointers (like strings) in not scanned blocks. The more valid pointers you have in your data, the smaller is probability of false pointers. The smaller is handle wrapper, the smaller is probability of a false pointer holding it. If you manage resources well, probability of handle leak goes even smaller (in C# it doesn't pose any notable difficulty even though you don't have any mechanism of eager resource management at all, only GC, in D you have non-zero opportunity for eager resource management). All these small probabilities multiply, and you get even smaller probability of an eventual resource leak. But as long as there can be false pointers, no matter how improbable, there can be no guaranteed destruction, which was my point. Maybe it becomes acceptable at very low probabilities, but it's still a gamble...
Re: Memory allocation purity
On Wed, 14 May 2014 21:11:30 -0400, Brian Schott briancsch...@gmail.com
wrote:
On Thursday, 15 May 2014 at 00:48:52 UTC, Walter Bright wrote:
On 5/14/2014 5:44 PM, Brian Schott wrote:
Can we say that Mallocator failures are not recoverable?
malloc itself does not have that property. But you could design a
wrapper for it that did.
I'm concerned specifically with this wrapper:
https://github.com/andralex/phobos/blob/allocator/std/allocator.d#L773
We need to make these functions pure if they are going to be usable.
Removing the stdlib import and adding
private extern (C)
{
void* malloc(size_t) pure nothrow @trusted;
void free(void*) pure nothrow @trusted;
void* realloc(void*, size_t) pure nothrow @trusted;
}
Be careful. The above is all correct, but as has been discussed, the
minute you start making returns immutable or parameters immutable, they
become strong-pure, which has undesirable properties for allocators. If
you wrap the above with calls to typed data, then strong-pure might be
inferred.
The compiler can specially designate things like idup to make sure they
always are considered weak-pure. But library code has to be more cautious.
-Steve
Re: Memory allocation purity
On Thu, 15 May 2014 07:52:20 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 10:50, Walter Bright via Digitalmars-d digitalmars-d@puremagic.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Even if it doesn't fail, malloc called with identical arguments still returns a different result every time. You can't factor malloc outside a loop and cache the result because it's being called repeatedly with the same argument like you're supposed to be able to do with any other pure function. You shouldn't be able to do that with gcalloc either... how can gcalloc be pure? That only applies to strong-pure functions. malloc would be weak-pure, since it returns a mutable pointer. -Steve
Re: range behaviour
On Wednesday, 14 May 2014 at 14:00:18 UTC, Ola Fosheim Grøstad
wrote:
On Wednesday, 14 May 2014 at 13:29:35 UTC, Dicebot wrote:
9 int foo1(int max)
10 {
11 int sum = 0;
12 for (auto i = 0; i max; ++i)
13 {
14 sum += i*2;
15 }
16 return sum;
17 }
This isn't really a loop. It is an expression: n*(n+1)
Try something more realistic such as doing alpha-blending to a
buffer or DSP.
I think it is your turn to make a counter-example ;) It does not
matter that much what loop actually does. It was effectively an
expression here but compiler was able to reduce range version to
quite the same expression despite all the intermediate code. Of
course there are still many ways to fool existing optimizers but
declaring fundamental inability to do so? No way.
Re: Memory allocation purity
Ola, you do not understand 'pure.' To consider the design of pure, you must first consider that you cannot add functional purity to an imperative language. It cannot be done. What you can do instead is what D offers with a 'weak purity' guarantee. That global state is not modified indirectly, that side-effects do not occur in a function, except through the function's parameters, except for memory allocation. D allows you to come pretty close to strong purity when a function is marked pure, does not allocate memory inside it or through its arguments, and has only const or immutable arguments. The only way you can get a better purity guarantee than that is to use a functional programming language like Haskell.
Re: radical ideas about GC and ARC : need to be time driven?
On Wednesday, 14 May 2014 at 20:02:08 UTC, Ola Fosheim Grøstad wrote: On Wednesday, 14 May 2014 at 19:45:20 UTC, Marc Schütz wrote: - We have external code programmed in languages other than D, most prominently C and C++. These don't provide any type information, therefore the GC needs to handle their memory conservatively, which means there can be false pointers = no deterministic destruction. Oh yes, I agree. However, you could have rules for collection and FFI (calling C). Like only allowing collection if all C parameters that point to GC memory have a shorter life span than other D pointers to the same memory (kind of like borrowed pointers in Rust). Some kind of lifetime annotation would be required for this. Not that this is a bad idea, but it will require some work... - Variables on the stack and in registers. In theory, the compiler could generate that information, or existing debug information might be used, but that's complicated for the GC to handle and will probably have runtime costs. The easy solution is to use something that is to define safe zones where you can freeze (kind of like rendezvous semaphores, but not quite). This helps with getting the registers on the stack, but we still need type information for them. - Untagged unions. The GC has no way to figure out which of the union fields is currently valid. If any of them is a pointer, it needs to treat them conservatively. So you need a function that can help the GC if the pointer fields of the union don't match up or don't point to class instances. Which of course requires type information. And existing unions need to be updated to implement this function. I guess sometimes it might not even be possible to implement it, because the state information is not present in the union itself.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 11:01:38 UTC, Jonathan M Davis via Digitalmars-d wrote: functions with mutable parameters. It doesn't. And it would actually be very difficult for it to do so without doing full program optimization, which really doesn't work with the C linking model that D uses. I think this C linking excuse is used too much :-). Clearly one can specify properties of C functions when writing the bindings. (Or; should be able to). Besides, not all call trees are full of C calls. functions. The whole purpose of weakly pure functions is to have functions which aren't functionally pure but can still be used in functions that _are_ functionally pure. I take it you mean expressions and not necessarily defined functions. So the compiler memoize values derived from weakly pure functions? source code for the function available. The only functions for which the compiler ever deduces anything from their bodies are templated functions, But with the amount of templates in phobos… maybe it is time to move beyond fully separate compilation units? about is the signature of the function. So, it's up to the programmer to tell the compiler that a function is supposed to be pure, and it's up to the compiler to verify that, and then use that knowledge to infer Ok, but based on bearophile's example it doesn't verify purity properly. So it is essentially part verified (verify noglob) and part programmer guarantee (this is pure). I don't mind programmers being able to set guarantees. For instance for a C function or machine language routines it might be useful to specify that this is pure and avoid recomputation. I agree that that would be great if we were starting over - just like it would be great if @safe were the default, but it's too late now. We have to make do with what we have. Yes, many changes over time is not good. Maybe it would be a good idea to collect all those syntax improvements and apply them all at once once the semantics are frozen. avoid it. And as nice as changing some of the attributes around would be nice, it's too big a breaking change for too little gain. It doesn't have to break anything. You could add @global and @pure (strong pure) and keep pure (weakly pure) as a deprecated feature.
Re: hijackable/customizable keyword for solving the customized algorithm issue?
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote: On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote: UFCS only apply to the method call style, not the the function call style, so it's not a matter of priority here - `foo(myObject)` will not call the `foo.myObject()` method even if there is no `foo` function(in that case it'll just fail). Having the function call style always defer to a member function is a really bad idea - and not just because of code breakage. It'll make it impossible to call a function on an object that has a method of the same name unless you use an alias or a function variable to copy the function - but then you have to make sure the alias\variable name is also not taken! This will be a disaster for anyone who wants to build or use a D library that uses templates. Well, how common can that case be? That's exactly what the proposed @hijackable annotation on a function would do, if I understand it correctly. Yes, but there is quite a difference between applying this to specific functions where this behavior is desirable and applying this to everything.
Re: More radical ideas about gc and reference counting
On Thursday, 15 May 2014 at 06:30:40 UTC, Jacob Carlborg wrote: On 14/05/14 23:47, Dmitry Olshansky wrote: This is curious: http://burntsushi.net/rustdoc/regex/ It seems they have compile time regular expressions too. Yeah looks like my knowledge about their macro system is very outdated. Looks promising.
Re: radical ideas about GC and ARC : need to be time driven?
On Thursday, 15 May 2014 at 12:44:56 UTC, Marc Schütz wrote: On Wednesday, 14 May 2014 at 20:02:08 UTC, Ola Fosheim Grøstad wrote: However, you could have rules for collection and FFI (calling C). Like only allowing collection if all C parameters that point to GC memory have a shorter life span than other D pointers to the same memory (kind of like borrowed pointers in Rust). Some kind of lifetime annotation would be required for this. Not that this is a bad idea, but it will require some work... Isn't it sufficient to let the backend always push pointers that could be to GC memory on the stack in the functions that calls C? The easy solution is to use something that is to define safe zones where you can freeze (kind of like rendezvous semaphores, but not quite). This helps with getting the registers on the stack, but we still need type information for them. Yes, but you have that in the description of the stack frame that you look up when doing precise collection? You need such stack frame identification utilities for doing exception handling too. Which of course requires type information. And existing unions need to be updated to implement this function. I guess sometimes it might not even be possible to implement it, because the state information is not present in the union itself. Then the compiler could complain or insist that you use a conservative GC.
Re: Memory allocation purity
On Thu, 15 May 2014 02:50:05 -0400, Ola Fosheim Grøstad
ola.fosheim.grostad+dl...@gmail.com wrote:
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with private
globals. Because memoing is no longer sound anyway.
This has nothing to do with allocators being pure. They must be pure as a
matter of practicality.
The issue that allows the above anomaly is using the address of something.
There is a reason functional languages do not allow these types of things.
But in functional languages, allocating is allowed.
To be honest, code that would exploit such an anomaly is only ever used in
proof exercises, and never in real code. I don't think it's an issue.
-Steve
Re: range behaviour
On Thursday, 15 May 2014 at 12:34:49 UTC, Dicebot wrote: I think it is your turn to make a counter-example ;) I've given 2 examples that no compiler can get at… because they lack contextual knowledge. ;) Basically anything that involves guards at the end is an issue, most things that involve floating point is an issue, but there are others too I think… I'll get back to counter-examples later when I have time to look closer at LDC. (not this week)
Re: Memory allocation purity
On Wed, 14 May 2014 20:50:08 -0400, Walter Bright newshou...@digitalmars.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Basically, you are saying that malloc must return the same block whenever it's called with the same parameters. This is simply nonsense. null is not special, it's just another block. I'm not saying malloc should be pure based on this, but the possibility that it returns null does not disqualify it. -Steve
Re: Next step on reference counting topics
On Tuesday, 13 May 2014 at 15:14:16 UTC, Andrei Alexandrescu wrote: On 5/13/14, 6:43 AM, Dicebot wrote: On actual topic of next step - I agree with Jacob, removing all internal allocations from Phobos is most beneficial short-term goal. That's a given. This task and improving RC don't compete (except, of course, for bearing on the same people). I ask again, who wants to champion std.refcounted? Rainer? Manu? Michel? Are you there? -- Andrei You should start with at least some sort of design document for what such module should look like. Right now it is not clear to me at all what kind of improvements are desired within current language limits.
Re: [OT] DConf - How to survive without a car?
On Wed, 14 May 2014 17:35:04 -0400, Joshua Niehus jm.nie...@gmail.com wrote: On Wednesday, 14 May 2014 at 21:31:08 UTC, Jeremy Powers via Digitalmars-d wrote: Excellent. I need a lift if possible? Of so please tell me when I should be in the aloft lobby? This seems to still be an open question - what time to gather? I'll be there between 8am and 8:15am and will have 3 seats available (want to get there in time for breakfast) Agreed. I also will have at least 3 seats. -Steve
Re: Memory allocation purity
On Thursday, 15 May 2014 at 12:37:22 UTC, w0rp wrote: To consider the design of pure, you must first consider that you cannot add functional purity to an imperative language. Of course you can. Functional languages execute in an imperative context. That's why you need monads. The term pure function is only needed in a non-functional language. Applicative/functional languages only have mathematical functions, no need for the term pure there.
Re: range behaviour
On Thursday, 15 May 2014 at 13:01:34 UTC, Ola Fosheim Grøstad wrote: On Thursday, 15 May 2014 at 12:34:49 UTC, Dicebot wrote: I think it is your turn to make a counter-example ;) I've given 2 examples that no compiler can get at… because they lack contextual knowledge. ;) If compiler lacks contextual knowledge, than only means that range is not actually semantically equivalent to a loop. Which is quite possible if you start considering something like SIMD, this is true. What is wrong is assumption that such kinds of loops are anything but tiny minority and this warrants generic ranges can never be as fast as loops statement. Btw, Floating point math is inaccurate, that means the compiler will have to guess what kind of accuracy you are happy with… AFAIK it is actually not true. Floating point standard defines basic precision guarantees and compilers are free to make any adjustments outside of those basics. In your example you need to verify generated assembly anyway, loop or range.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 12:57:43 UTC, Steven Schveighoffer wrote: To be honest, code that would exploit such an anomaly is only ever used in proof exercises, and never in real code. I don't think it's an issue. That's the wrong attitude to take when it comes to the compiler and runtime. If it verifies, it should verify. Not mostly, but fully, otherwise weakly pure becomes a programmer guarantee. Which is ok too, but either the one or the other. How can you prove that such issues don't arise in real code?
Re: Memory allocation purity
On Thursday, 15 May 2014 at 12:57:43 UTC, Steven Schveighoffer
wrote:
On Thu, 15 May 2014 02:50:05 -0400, Ola Fosheim Grøstad
ola.fosheim.grostad+dl...@gmail.com wrote:
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with
private globals. Because memoing is no longer sound anyway.
This has nothing to do with allocators being pure. They must be
pure as a matter of practicality.
The issue that allows the above anomaly is using the address of
something. There is a reason functional languages do not allow
these types of things. But in functional languages, allocating
is allowed.
Which is what makes D pure lint-like helper and not effective
optimization enabler. This part of type system was under-designed
initially, it should have been much more restrictive.
I am ok with current state though, it is just sad that it creates
lot of false expectations from those familiar with pure
functional languages.
To be honest, code that would exploit such an anomaly is only
ever used in proof exercises, and never in real code. I don't
think it's an issue.
This is not true. Because of such code you can't ever
automatically memoize strongly pure function results by compiler.
A very practical concern.
Re: Memory allocation purity
On 15 May 2014 22:30, Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thu, 15 May 2014 07:52:20 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 10:50, Walter Bright via Digitalmars-d digitalmars-d@puremagic.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Even if it doesn't fail, malloc called with identical arguments still returns a different result every time. You can't factor malloc outside a loop and cache the result because it's being called repeatedly with the same argument like you're supposed to be able to do with any other pure function. You shouldn't be able to do that with gcalloc either... how can gcalloc be pure? That only applies to strong-pure functions. malloc would be weak-pure, since it returns a mutable pointer. Why should returning a mutable pointer imply weak purity?
Re: Memory allocation purity
On Thu, 15 May 2014 09:24:54 -0400, Ola Fosheim Grøstad ola.fosheim.grostad+dl...@gmail.com wrote: On Thursday, 15 May 2014 at 12:57:43 UTC, Steven Schveighoffer wrote: To be honest, code that would exploit such an anomaly is only ever used in proof exercises, and never in real code. I don't think it's an issue. That's the wrong attitude to take when it comes to the compiler and runtime. There are always ways around the guarantees. This one isn't as detectable, since there is no red-flag cast or attribute. But I see no utility in such code. If it verifies, it should verify. Not mostly, but fully, otherwise weakly pure becomes a programmer guarantee. Which is ok too, but either the one or the other. Memory allocation is an exception to purity that is widely considered OK, because of the practical need for allocating memory from a heap to do work. In general, it's the block of data that is important, not it's address. Where it comes from is of no consequence. How can you prove that such issues don't arise in real code? Of course I can't prove it doesn't arise, but I can't think of any use cases to do something significant based on the address of heap data in relation to another unrelated heap block. Perhaps you can show a use case for it? The other thing to think about is that such code won't serve the purpose for which it was written! randomBit won't be random because it will likely be memoized. In any case, the alternative is to have D pure functions avoid using pointers. It's as drastic as that. -Steve
Re: Memory allocation purity
On Thu, 15 May 2014 09:40:03 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 22:30, Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thu, 15 May 2014 07:52:20 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 10:50, Walter Bright via Digitalmars-d digitalmars-d@puremagic.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Even if it doesn't fail, malloc called with identical arguments still returns a different result every time. You can't factor malloc outside a loop and cache the result because it's being called repeatedly with the same argument like you're supposed to be able to do with any other pure function. You shouldn't be able to do that with gcalloc either... how can gcalloc be pure? That only applies to strong-pure functions. malloc would be weak-pure, since it returns a mutable pointer. Why should returning a mutable pointer imply weak purity? Because if it were strong-pure, the compiler could optimize two sequential calls as returning the same exact thing. Imagine if a constructor to a mutable object always returned the same object because the constructor's parameters were immutable. It would be useless. -Steve
Re: Memory allocation purity
On Thu, 15 May 2014 09:28:57 -0400, Dicebot pub...@dicebot.lv wrote:
On Thursday, 15 May 2014 at 12:57:43 UTC, Steven Schveighoffer wrote:
On Thu, 15 May 2014 02:50:05 -0400, Ola Fosheim Grøstad
ola.fosheim.grostad+dl...@gmail.com wrote:
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with private
globals. Because memoing is no longer sound anyway.
This has nothing to do with allocators being pure. They must be pure as
a matter of practicality.
The issue that allows the above anomaly is using the address of
something. There is a reason functional languages do not allow these
types of things. But in functional languages, allocating is allowed.
Which is what makes D pure lint-like helper and not effective
optimization enabler. This part of type system was under-designed
initially, it should have been much more restrictive.
It's easy to say this, but the restrictions to apply would be draconian. I
think it would be nearly impossible to prevent such abuses in a language
with explicit references.
To be honest, code that would exploit such an anomaly is only ever used
in proof exercises, and never in real code. I don't think it's an
issue.
This is not true. Because of such code you can't ever automatically
memoize strongly pure function results by compiler. A very practical
concern.
I think you can still memoize these cases. There is no reason for the
language to support a pure RNG.
-Steve
Re: Memory allocation purity
On Thursday, 15 May 2014 at 06:50:06 UTC, Ola Fosheim Grøstad
wrote:
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with
private globals. Because memoing is no longer sound anyway.
No, this particular example appears to be invalid code. Under the
C memory model [1], the result of comparing two pointers to
distinct objects (allocations) is undefined/unspecified behavior
(I think it is undefined in C and unspecified in C++, but I don't
remember the details).
Of course, you could rescue the example by casting the pointers
to size_t or something along the lines, but this is something
that could be disallowed in pure code.
David
[1] Which typically applies to D, unless defined otherwise. Our
language specification is woefully incomplete in this regard,
though.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 13:40:16 UTC, Manu via Digitalmars-d wrote: Why should returning a mutable pointer imply weak purity? The argument here is that a valid mutable pointer returned from a pure function could always point to a new allocation, as new is allowed in pure code. More precisely, the only way to return a valid mutable pointer from a parameterless function is to make it point to a new allocation, so allowing memory allocations does not even preclude the inference of stronger guarantees here. David
Re: Memory allocation purity
On Thursday, 15 May 2014 at 13:42:58 UTC, Steven Schveighoffer wrote: On Thu, 15 May 2014 09:24:54 -0400, Ola Fosheim Grøstad ola.fosheim.grostad+dl...@gmail.com wrote: That's the wrong attitude to take when it comes to the compiler and runtime. There are always ways around the guarantees. This one isn't as detectable, since there is no red-flag cast or attribute. But I see no utility in such code. I have to agree with Ola here. If you write a piece of pure, @safe code, there should be no way for compiler optimizations to make your code behave differently. This is not only because implicitly allowing unsafe code would be against the design philosophy behind D, but also as attribute inference might tag functions as pure without the programmer explicitly specifying so. In any case, the alternative is to have D pure functions avoid using pointers. It's as drastic as that. I'd suspect that it is enough to disallow using the content of pointers explicitly, i.e. as a sequence of bytes instead of just a handle to an object. David
Re: Memory allocation purity
On Thu, 15 May 2014 10:42:00 -0400, David Nadlinger c...@klickverbot.at wrote: On Thursday, 15 May 2014 at 13:42:58 UTC, Steven Schveighoffer wrote: On Thu, 15 May 2014 09:24:54 -0400, Ola Fosheim Grøstad ola.fosheim.grostad+dl...@gmail.com wrote: That's the wrong attitude to take when it comes to the compiler and runtime. There are always ways around the guarantees. This one isn't as detectable, since there is no red-flag cast or attribute. But I see no utility in such code. I have to agree with Ola here. If you write a piece of pure, @safe code, there should be no way for compiler optimizations to make your code behave differently. In general, I would say any code that performs differently after optimization is not preferable. But in this case, you have ignored the rules, and the result is not exactly incorrect or unsafe. In fact, you can't really argue that it's invalid (randomBit could legitimately always return true or false, even in a non-optimized program). The question here is, can we come up with a static rule that is effective, but not cripplingly prohibitive? This is not only because implicitly allowing unsafe code would be against the design philosophy behind D, but also as attribute inference might tag functions as pure without the programmer explicitly specifying so. So far, I haven't seen code that's unsafe only if optimized. Have an example? In any case, the alternative is to have D pure functions avoid using pointers. It's as drastic as that. I'd suspect that it is enough to disallow using the content of pointers explicitly, i.e. as a sequence of bytes instead of just a handle to an object. This means format(%x, ptr) isn't allowed to be pure? What about calculating index offsets? Note that pointer math between two pointers on the same block of memory is perfectly legitimate. I would expect that someone could be able to write a type equivalent to a slice, and it should be allowed to be pure. -Steve
Re: Memory allocation purity
On 5/15/14, 3:31 AM, luka8088 wrote: Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct? Yes, as long as you don't rely on distinguishing objects by address. Purity of allocation is frequently assumed by functional languages because without it it would be difficult to get much work done. Then, most functional languages make it difficult or impossible to distinguish values by their address. In D that's easy. A D programmer needs to be aware of that, and I think that's fine. Andrei
Re: Memory allocation purity
On 5/15/14, 6:02 AM, Steven Schveighoffer wrote: On Wed, 14 May 2014 20:50:08 -0400, Walter Bright newshou...@digitalmars.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Basically, you are saying that malloc must return the same block whenever it's called with the same parameters. This is simply nonsense. null is not special, it's just another block. I'm not saying malloc should be pure based on this, but the possibility that it returns null does not disqualify it. Null is special - it's a singularity. It can't be subsequently used for constructing a proper object. That makes it different even after we discount for comparing pointers. Andrei
Re: Memory allocation purity
On 05/15/2014 02:57 PM, Steven Schveighoffer wrote:
On Thu, 15 May 2014 02:50:05 -0400, Ola Fosheim Grøstad
ola.fosheim.grostad+dl...@gmail.com wrote:
On Thursday, 15 May 2014 at 06:29:06 UTC, bearophile wrote:
A little example of D purity (this compiles):
bool randomBit() pure nothrow @safe {
return (new int[1].ptr) (new int[1].ptr);
}
Yes, and then you may as well allow a random generator with private
globals. Because memoing is no longer sound anyway.
This has nothing to do with allocators being pure. They must be pure as
a matter of practicality.
The issue that allows the above anomaly is using the address of
something. There is a reason functional languages do not allow these
types of things. But in functional languages, allocating is allowed.
...
Not really, allocation is just an implementation detail. The
computational language is meaningful independent of how you might
achieve evaluation of expressions. I can in principle evaluate an
expression of such a language on paper without managing a separate
store, even though this usually will help efficiency.
Functional programming languages are not about taking away features from
a procedural core language. They are based on the idea that the
fundamental operation is substitution of terms.
To be honest, code that would exploit such an anomaly is only ever used
in proof exercises, and never in real code.
Hashtables are quite real.
I don't think it's an issue.
-Steve
This is the issue:
On Thu, 15 May 2014 10:48:07 +
Don via Digitalmars-d digitalmars-d@puremagic.com wrote:
Yes. 'strong pure' means pure in the way that the functional
language crowd means 'pure'.
'weak pure' just means doesn't use globals.
There is no way to make that claim precise in an adequate way such that
it is correct.
Re: Memory allocation purity
On 5/15/14, 6:28 AM, Dicebot wrote: This is not true. Because of such code you can't ever automatically memoize strongly pure function results by compiler. A very practical concern. I think code that doesn't return pointers should be memoizable. Playing tricks with pointer comparisons would be appropriately penalized. -- Andrei
Re: Memory allocation purity
David Nadlinger: I'd suspect that it is enough to disallow using the content of pointers explicitly, i.e. as a sequence of bytes instead of just a handle to an object. Yes, if you allow only a referentially pure view of pointers, then you have a strong purity. Bye, bearophile
Re: Next step on reference counting topics
On 5/15/14, 6:02 AM, Dicebot wrote: On Tuesday, 13 May 2014 at 15:14:16 UTC, Andrei Alexandrescu wrote: On 5/13/14, 6:43 AM, Dicebot wrote: On actual topic of next step - I agree with Jacob, removing all internal allocations from Phobos is most beneficial short-term goal. That's a given. This task and improving RC don't compete (except, of course, for bearing on the same people). I ask again, who wants to champion std.refcounted? Rainer? Manu? Michel? Are you there? -- Andrei You should start with at least some sort of design document for what such module should look like. The design is part of the championing. But haven't heard a peep - it seems everybody wants to go to heaven, but nobody wants to die :o). Right now it is not clear to me at all what kind of improvements are desired within current language limits. The clarify it to yourself and others. Start with a library approach to making reference counting easy and convenient. Then see how it informs possible language improvements. Andrei
Re: Memory allocation purity
On 15 May 2014 23:47, Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thu, 15 May 2014 09:40:03 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 22:30, Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thu, 15 May 2014 07:52:20 -0400, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 15 May 2014 10:50, Walter Bright via Digitalmars-d digitalmars-d@puremagic.com wrote: On 5/14/2014 5:03 PM, Meta wrote: Allocating memory through new and malloc should always be pure, I think, because failure either returns null in malloc's case, malloc cannot be pure if, with the same arguments, it returns null sometimes and not other times. Even if it doesn't fail, malloc called with identical arguments still returns a different result every time. You can't factor malloc outside a loop and cache the result because it's being called repeatedly with the same argument like you're supposed to be able to do with any other pure function. You shouldn't be able to do that with gcalloc either... how can gcalloc be pure? That only applies to strong-pure functions. malloc would be weak-pure, since it returns a mutable pointer. Why should returning a mutable pointer imply weak purity? Because if it were strong-pure, the compiler could optimize two sequential calls as returning the same exact thing. Imagine if a constructor to a mutable object always returned the same object because the constructor's parameters were immutable. It would be useless. -Steve I don't follow. Forget that it's a pointer... it's just a number. A strong pure function simply doesn't modify it's inputs (which we can guarantee with const/immutable args), and returns the same output. It shouldn't matter if the output is mutable or not, it just has to be the same. Whether it's the number 10, or a pointer. I guess it logically follows that the output must be const or immutable, because it can only possibly be returning a pointer to, or to a member of, one of it's args... and 'turtles all the way down'. But that leads me back to where I started... how can it possibly be that an allocator of any sort can be pure? If you can allocate a new pointer, you can return it as a const or immutable pointer if you like, and then the function looks awfully like a strong pure function even though it returns a new pointer every time. That's not pure at all.
Re: Memory allocation purity
On 05/15/2014 05:24 PM, Andrei Alexandrescu wrote: On 5/15/14, 3:31 AM, luka8088 wrote: Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct? Yes, as long as you don't rely on distinguishing objects by address. Purity of allocation is frequently assumed by functional languages Examples? because without it it would be difficult to get much work done. Why? Then, most functional languages make it difficult or impossible to distinguish values by their address. If it's not impossible because of lack of the concept it's not a pure functional language. In D that's easy. A D programmer needs to be aware of that, and I think that's fine. Andrei It's not fine: The spec claims that this problem does not exist. http://dlang.org/function.html ... and in cases where the compiler can guarantee that a pure function cannot alter its arguments, it can enable full, functional purity (i.e. the guarantee that the function will always return the same result for the same arguments)
Re: Memory allocation purity
On 05/15/2014 11:45 AM, Don wrote: No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra Why? A memoizable function is still memoizable if it is changed internally to memoize values in global memory, for example. property which the compiler can trivially determine from @noglobal.
