Re: HibernateD and DDBC - ORM and DB abstraction layer for D
On Wednesday, 3 April 2013 at 14:28:37 UTC, Vadim Lopatin wrote: Hello! I've started implemetation of ORM in D, with annotations and interfaces similar to Hibernate. As DB abstraction layer I wrote DDBC - library with interface similar to JDBC. Only MySQL driver is implemented. PostgreSQL - in progress. Project is hosted on SourceForge: https://sourceforge.net/p/hibernated/wiki/HibernateD/ License is Boost ... Look at project wiki and unittests for more info. Project status update: PostgreSQL support is added to DDBC and HibernateD. Does anyone have an idea what to do next?
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
:) nuodb. On Fri, Apr 12, 2013 at 9:08 AM, Vadim Lopatin coolreader@gmail.comwrote: On Wednesday, 3 April 2013 at 14:28:37 UTC, Vadim Lopatin wrote: Hello! I've started implemetation of ORM in D, with annotations and interfaces similar to Hibernate. As DB abstraction layer I wrote DDBC - library with interface similar to JDBC. Only MySQL driver is implemented. PostgreSQL - in progress. Project is hosted on SourceForge: https://sourceforge.net/p/** hibernated/wiki/HibernateD/https://sourceforge.net/p/hibernated/wiki/HibernateD/License is Boost ... Look at project wiki and unittests for more info. Project status update: PostgreSQL support is added to DDBC and HibernateD. Does anyone have an idea what to do next?
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
Hi Vadim, Does anyone have an idea what to do next? It would be great if you publish the DUB package. Please refer to http://registry.vibed.org/publish. There is a hint. Page http://registry.vibed.org/ lists registered DUB packages. To see them you have to register and sign in there. Otherwise, you will see some product info page.
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
There is a hint. Page http://registry.vibed.org/ lists registered DUB packages. To see them you have to register and sign in there. Otherwise, you will see some product info page. Oops. It looks fixed. :-)
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
On Friday, 12 April 2013 at 09:03:56 UTC, kdmult wrote:
Hi Vadim,
Does anyone have an idea what to do next?
It would be great if you publish the DUB package.
Please refer to http://registry.vibed.org/publish.
There is a hint.
Page http://registry.vibed.org/ lists registered DUB packages.
To see them you have to register and sign in there. Otherwise,
you will see some product info page.
Trying to package DDBC.
https://github.com/buggins/ddbc
dub build fails.
It looks like it tries to build application instead of library,
although I specified targetType: staticLibrary,
Checking dependencies in '/home/lve/src/ddbc'
Building configuration MySQL, build type debug
Running dmd (compile)...
Linking...
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_47f_1a5.o): In
function `main':
src/rt/dmain2.d:(.text.main+0xa): undefined reference to `_Dmain'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_1a1_1b8.o): In
function `_D4core6thread6Thread6__ctorMFZC4core6thread6Thread':
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x26):
undefined reference to `_tlsend'
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x31):
undefined reference to `_tlsstart'
/usr/lib/x86_64-linux-gnu/libphobos2.a(deh2_45f_525.o): In
function `_D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable':
src/core/thread.d:(.text.thread_entryPoint+0xb0): undefined
reference to `_tlsstart'
collect2: error: ld returned 1 exit status
--- errorlevel 1
Error: no main function specified
Error: Link command failed with exit code 1
{
name: DDBC,
description: DB Connector for D language, similar to JDBC,
authors: [Vadim Lopatin],
homepage: http://sourceforge.net/projects/ddbc;,
license: Boost Software License (BSL 1.0),
dependencies: {
},
targetType: staticLibrary,
configurations: [
{
name: MySQL,
versions: [USE_MYSQL]
},
{
name: SQLite,
versions: [USE_SQLITE],
libs-posix: [sqlite3],
libs-windows: [sqlite3.lib]
},
{
name: PGSQL,
versions: [USE_PGSQL],
libs-posix: [pq],
libs-windows: [libpq.lib]
}
]
}
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
Still cannot build library using DUB.
Tried different changes in package.json
dub -v
...
[7F6FC283306F: INF] Running dmd (compile)...
[7F6FC283306F: dbg] dmd -w -g -debug -c -oftemp.o
-version=USE_MYSQL -I/home/lve/src/ddbc/source
source/drivers/sha1.d source/drivers/pgsqlddbc.d
source/drivers/pgsql.d source/drivers/mysqlddbc.d
source/drivers/mysql.d source/drivers/utils.d
source/drivers/sqliteddbc.d source/common.d source/core.d
[7F6FC283306F: INF] Linking...
[7F6FC283306F: dbg] dmd -of/tmp/dub/2946988246/libddbc.a
temp.o
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_47f_1a5.o): In
function `main':
Why does it try to link app executable instead of just link
library?
package.json:
{
name: DDBC,
description: DB Connector for D language, similar to JDBC,
authors: [Vadim Lopatin],
homepage: http://sourceforge.net/projects/ddbc;,
license: Boost Software License (BSL 1.0),
sourcePaths: [source, source/drivers],
configurations: [
{
name: MySQL,
versions: [USE_MYSQL],
targetType: staticLibrary,
targetPath: bin,
targetName: ddbc,
}
]
}
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
Am 12.04.2013 15:17, schrieb Vadim Lopatin:
Still cannot build library using DUB.
Tried different changes in package.json
dub -v
...
[7F6FC283306F: INF] Running dmd (compile)...
[7F6FC283306F: dbg] dmd -w -g -debug -c -oftemp.o
-version=USE_MYSQL -I/home/lve/src/ddbc/source source/drivers/sha1.d
source/drivers/pgsqlddbc.d source/drivers/pgsql.d
source/drivers/mysqlddbc.d source/drivers/mysql.d source/drivers/utils.d
source/drivers/sqliteddbc.d source/common.d source/core.d
[7F6FC283306F: INF] Linking...
[7F6FC283306F: dbg] dmd -of/tmp/dub/2946988246/libddbc.a temp.o
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_47f_1a5.o): In function
`main':
Why does it try to link app executable instead of just link library?
package.json:
{
name: DDBC,
description: DB Connector for D language, similar to JDBC,
authors: [Vadim Lopatin],
homepage: http://sourceforge.net/projects/ddbc;,
license: Boost Software License (BSL 1.0),
sourcePaths: [source, source/drivers],
configurations: [
{
name: MySQL,
versions: [USE_MYSQL],
targetType: staticLibrary,
targetPath: bin,
targetName: ddbc,
}
]
}
Try to move targetType one level higher?
Re: HibernateD and DDBC - ORM and DB abstraction layer for D
On Friday, 12 April 2013 at 13:22:41 UTC, David wrote: [7F6FC283306F: INF] Linking... [7F6FC283306F: dbg] dmd -of/tmp/dub/2946988246/libddbc.a temp.o Why does it try to link app executable instead of just link library? Try to move targetType one level higher? Tried. Doesn't help. Changing of targetType to library instead of staticLibrary doesn't help, too. Link command is always the same: dmd -of/tmp/dub/2946988246/libddbc.a temp.o
Vote started for std.process
Go here: http://forum.dlang.org/post/mgstnugckomjpvdgp...@forum.dlang.org
Re: DConf 2013 official car/room sharing thread
On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. I've had no trouble getting trains between SF and SJ... Ali
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room.
Re: Vote for std.process
On Fri, 12 Apr 2013 06:46:51 +0200 Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes
Re: Vote for std.process
Sorry to derail the topic, but I'd just like to raise a major gripe I've had with introducing anything into phobos recently. I see this pattern where something is designed, discussed, and then voted into phobos. At this time the design looks good on paper, but there is very little practical experience using the library. The problem then is, once accepted, people start using it, and at some point some issues are found, or ideas for improvement are made based on user experience, but the module can no longer be touched due to the general phobia of making breaking changes... Can I suggest that ALL new modules should be added to exp. rather than std.? Here they will stay for at least 1 year, and while they live in exp, it is understood that they are still in an experimental introductory phase. Users who choose to use modules in exp are accepting that the API may receive changes, and consequently, accepting the responsibility to update their code (and not complain about it breaking their program), should the library be amended in its introductory phase. At some time later when the module has been used in a decent amount of software, and the API has stabilised, it can then be moved to std. This move is obviously a breaking change its self, but anyone who has agrees to import exp modules has already accepted the responsibility to update their code as such. Thoughts? On 12 April 2013 16:12, Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Fri, 12 Apr 2013 06:46:51 +0200 Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes
Re: PHP extension in D
On Thursday, 11 April 2013 at 20:30:28 UTC, Rémy Mouëza wrote:
On Thursday, 11 April 2013 at 11:28:05 UTC, gedaiu wrote:
Hi,
I want to extend Php with an extension written in D, and wrap
some D classes in php. My questions are:
1. How I can build a static library(I am using eclipse with
ddt)
2. How I can create methods and create objects in c++ from the
D library.
Thanks,
Bogdan
I once tried to make a proof of concept PHP extension in D on an
Ubuntu Linux. I chose to make it simple by letting Swig
(www.swig.org) do the heavy wrapping work for me. It's then
becomes quite as much difficult as calling some D code from C.
Below is an outline of the steps it took me to wrap a D function
for PHP.
You'll still have to figure out how to make it work nicely with
Eclipse and on your particular platform, nonetheless it should
help you getting started.
Note that this approach could also be used to call some D code
from Java, C# or any other language supported by Swig.
1. Write / get the D code (speedup.d).
import std.stdio , std.string ; import std.conv : to;
string speedUp (string msg) {
writefln (-- %s --, msg);
return `sped up %s`.format (msg);
}
2. make a C API for that code, declaring the functions of that
API with the extern (C) qualifier.
extern (C) {
immutable (char) * d_speedUp (char * msg) {
return toStringz (speedUp (to!string (msg)));
}
}
3. Create a Swig interface file for that C API (speedup.i). Also
add a declaration to the rt_init function from the druntime.
%module speedup
char rt_init (long long);
const char * d_speedUp (char * msg);
4. run Swig: `swig -php speedup.i`. This will generate several
files:
- speedup_wrap.c
- php_speedup.h
- speedup.php
Now we've got to compile all those parts together: we want to
generate a speedup.so dynamic library object from all that we
got.
5. From what I researched, dmd (at least on Linux) needs to see
a
main function to properly generate all the code of the druntime
within the dynamic library/shared object it generates. We will
trick it by adding a fake main function (dfakemain.d):
void main () {}
6. compile dfakemain.d:
dmd -c dfakemain.d
7. compile speedup.o:
dmd -fPIC -c -L-shared dfakemain.o speedup.d
8. compile speedup_wrap.o, you'll need to have everything needed
to compile extension for PHP on your platform:
gcc `php-config --includes` -fpic -c speedup_wrap.c
9. compile speedup.so, our target:
dmd -shared speedup_wrap.o dfakemain.o speedup.o -ofspeedup.so
10. If everything went well so far, you can now try your
extension within a script (dmd_speedup.php):
?php rt_init (0); // initialize the D runtime. echo d_speedUp
(hello from php), \n;
You can launch the script with extension loading enabled with a
command line such like this one:
php -d enable_dl=1 -d extension=`pwd`/speedup.so
dmd_speedup.php
It should then output:
-- hello from php --
sped up hello from php
Now, one could imagine to get a bit further by automating all
that, like using the json output from dmd to generate a swig
interface file and a nice OOP interface on the PHP side, but
this
would be much more work.
Have fun!
@Rémy Mouëza Thanks a lot!
@Denis Shelomovskij sorry... this is my first post here, i will
try to be careful next time when i post here next time..
Re: Vote for std.process
On Friday, 12 April 2013 at 06:25:10 UTC, Manu wrote: Sorry to derail the topic, but I'd just like to raise a major gripe I've had with introducing anything into phobos recently. I see this pattern where something is designed, discussed, and then voted into phobos. At this time the design looks good on paper, but there is very little practical experience using the library. The problem then is, once accepted, people start using it, and at some point some issues are found, or ideas for improvement are made based on user experience, but the module can no longer be touched due to the general phobia of making breaking changes... Can I suggest that ALL new modules should be added to exp. rather than std.? Here they will stay for at least 1 year, and while they live in exp, it is understood that they are still in an experimental introductory phase. Users who choose to use modules in exp are accepting that the API may receive changes, and consequently, accepting the responsibility to update their code (and not complain about it breaking their program), should the library be amended in its introductory phase. At some time later when the module has been used in a decent amount of software, and the API has stabilised, it can then be moved to std. This move is obviously a breaking change its self, but anyone who has agrees to import exp modules has already accepted the responsibility to update their code as such. Thoughts? On 12 April 2013 16:12, Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Fri, 12 Apr 2013 06:46:51 +0200 Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes Fully agree
Re: Formal Review of std.process
On 2013-04-11 22:31, Lars T. Kyllingstad wrote: It's implemented using a class, but you use it like an object. I think that is more important. I see. I know there are some people here that are very picky about these things. -- /Jacob Carlborg
Re: Vote for std.process
On 2013-04-12 08:24, Manu wrote: Thoughts? I think it's a good idea. -- /Jacob Carlborg
Re: Vote for std.process
On 4/12/13, Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes.
Re: Vote for std.process
On 12 April 2013 14:46, Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. I didn't think I had much of an opinion on std.process, but I just gave it a quick once over and noticed, relating to our recent discussions about flagrant GC usage throughout phobos, that this is riddled with the exact sorts of issues I've been talking about: string[string] is used in the main API to receive environment variables; perhaps kinda convenient, but it's impossible to supply environment variables with loads of allocations. toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions. String concatenation is rampant! Look at this code to parse the env variables (which are already an AA): foreach (var, val; childEnv) envz[pos++] = (var~'='~val~'\0').ptr; And many many more of the same... This is a lib that spawns a process, I can't imagine why this library should need to allocate a single byte, but it allocates wildly just to perform a trivial pass-through to the OS. This module is no exception either, basically all of phobos is like this, and I'd like to see careful consideration made to these details in the future. I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion.
Re: Vote for std.process
On Friday, 12 April 2013 at 06:25:10 UTC, Manu wrote: Can I suggest that ALL new modules should be added to exp. rather than std.? Java tried this. As a result, they are now stuck with javax because they realized breaking backwards compatibility wasn't worth the benefit of graduating packages from javax to java. If the goal is to include new or newly-overhauled modules in official releases, I think simply putting a big red EXPERIMENTAL warning at the top of the documentation would be simpler. Then, the warning can be removed when there is a D release which has no breaking changes in said module since its last released version. However, another direction would be to make it easier to test such fresh modules. For example, it would be nice to have nightly builds of D, both for the master version, and a mob version, which includes some work-in-progress developments.
Re: Vote for std.process
... Thoughts? Right suggestion. As it was already discussed once, D community could follow Linux in this regard: each new module enters 'staging' folder. At some point in time it goes into 'mainline'. This could also streamline the process of deprecating the old library.
Re: Vote for std.process
On Friday, 12 April 2013 at 07:04:23 UTC, Manu wrote: And many many more of the same... So how many of these will produce a measurable performance increase when optimized, considering the relative cost of process creation? This is a lib that spawns a process, I can't imagine why this library should need to allocate a single byte So that the library fits in 3500 lines instead of many more, and has an API that can be invoked using one line, instead of ten.
Re: Vote for std.process
On Friday, 12 April 2013 at 07:00:25 UTC, Vladimir Panteleev wrote: On Friday, 12 April 2013 at 06:25:10 UTC, Manu wrote: Can I suggest that ALL new modules should be added to exp. rather than std.? Java tried this. As a result, they are now stuck with javax because they realized breaking backwards compatibility wasn't worth the benefit of graduating packages from javax to java. Actually they also have another issue. Java supported since the begging something similar to a System package for dirty tricks, officially it was a Sun only API. http://www.docjar.com/docs/api/sun/misc/Unsafe.html So actually all Java implementations provide a compatible API, because many do use it. Younger languages seem not to have any issue breaking backwards compatibility for experimental code. -- Paulo
Re: Vote for std.process
On 12 April 2013 17:13, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 07:04:23 UTC, Manu wrote: And many many more of the same... So how many of these will produce a measurable performance increase when optimized, considering the relative cost of process creation? I agree that spawning processes is a low-frequency operation, but it's a principle I'm trying to illustrate here. This is a lib that spawns a process, I can't imagine why this library should need to allocate a single byte So that the library fits in 3500 lines instead of many more, and has an API that can be invoked using one line, instead of ten. You can argue whatever you like. I've said my part, and I wherever it lands is not my call. I'm just illustrating the point that phobos, like STL, and to a lesser extent the CRT, are to continue to be avoided like the plague in some fields unless these details are considered in future. Most of the required changes would be self-contained, and not affect the API at all.
Re: Vote for std.process
On Friday, 12 April 2013 at 07:22:42 UTC, Manu wrote: I agree that spawning processes is a low-frequency operation, but it's a principle I'm trying to illustrate here. My point was that it is not that it's low-frequency, it's that the OS process creation operation is so expensive, that a few memory allocations will not make much of a difference in comparison. It's the same as optimizing memory allocations in a program which is intrinsically disk- or network-bound. You can argue whatever you like. I've said my part, and I wherever it lands is not my call. Well, that's just not very constructive. Your complaint in valid in general, but I was pointing out that it is not much so when specifically aimed at std.process.
Re: Vote for std.process
On Fri, 12 Apr 2013 03:04:08 -0400, Manu turkey...@gmail.com wrote: On 12 April 2013 14:46, Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. I didn't think I had much of an opinion on std.process, but I just gave it a quick once over and noticed, relating to our recent discussions about flagrant GC usage throughout phobos, that this is riddled with the exact sorts of issues I've been talking about: It's spawning a new process. What is the issue with allocating a bit of memory? The spawning of the process performance is going to dwarf that of any memory allocations. string[string] is used in the main API to receive environment variables; perhaps kinda convenient, but it's impossible to supply environment variables with loads of allocations. I think you meant without? What would be your suggestion? string[string] is the built-in map type. How do you pass an environment map without having some allocations? toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions. This would be a lot of effort for pretty much zero gain for the majority of cases. String concatenation is rampant! Look at this code to parse the env variables (which are already an AA): foreach (var, val; childEnv) envz[pos++] = (var~'='~val~'\0').ptr; This could be improved. It could also be optimized into a single allocation automatically by the compiler (it might already be). The API would not be affected by this improvement, though. And many many more of the same... This is a lib that spawns a process, I can't imagine why this library should need to allocate a single byte, but it allocates wildly just to perform a trivial pass-through to the OS. It is not a trivial pass-through. Different OSes require different parameter types. Look at the difference between the windows and posix implementations. We are writing a cross-platform library, so whatever we pick will have to be converted on at least one OS. This module is no exception either, basically all of phobos is like this, and I'd like to see careful consideration made to these details in the future. I too like to avoid memory allocations. But the level to which you require is too stringent. Not even Tango, which made low heap-allocations a priority, avoided allocations in it's process library. I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. We cannot cater to all developers. To put all developers through a lot of pain in order to attract the fringe ones is not a practical goal. I think the convenience of std.process is what it is there for. For the highest performance, you are able to call the OS functions directly. -Steve
Re: Vote for std.process
On 12 April 2013 17:30, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 07:22:42 UTC, Manu wrote: I agree that spawning processes is a low-frequency operation, but it's a principle I'm trying to illustrate here. My point was that it is not that it's low-frequency, it's that the OS process creation operation is so expensive, that a few memory allocations will not make much of a difference in comparison. It's the same as optimizing memory allocations in a program which is intrinsically disk- or network-bound. Which OS are we talking about? What OS runs on an a Nintendo Wii? There's only 24mb of system memory in that machine, can we afford to allocate it frivolously? Will I avoid phobos as a policy? Yes. You can argue whatever you like. I've said my part, and I wherever it lands is not my call. Well, that's just not very constructive. Your complaint in valid in general, but I was pointing out that it is not much so when specifically aimed at std.process. I don't necessarily disagree with your point (although I don't agree either). Perhaps in the context of std.process it's not so important... but it's still an opportunity to set a precedent. Honestly, any new module could have appeared for approval at this particular moment and I would have made the same criticisms. Not necessarily picking on std.process in particular, I'm making a point about phobos, and what is considered acceptable.
Re: Vote for std.process
Yes.
Re: Vote for std.process
On 12 April 2013 17:35, Steven Schveighoffer schvei...@yahoo.com wrote: On Fri, 12 Apr 2013 03:04:08 -0400, Manu turkey...@gmail.com wrote: string[string] is used in the main API to receive environment variables; perhaps kinda convenient, but it's impossible to supply environment variables with loads of allocations. I think you meant without? Yes. What would be your suggestion? string[string] is the built-in map type. How do you pass an environment map without having some allocations? I'd use string[]. toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions. This would be a lot of effort for pretty much zero gain for the majority of cases. A trivial ... mixin template (?) could wrap it up, I don't see it particularly less convenient than calling toStringz(). Perhaps there are other tools missing from phobos if this is somehow hard... String concatenation is rampant! Look at this code to parse the env variables (which are already an AA): foreach (var, val; childEnv) envz[pos++] = (var~'='~val~'\0').ptr; This could be improved. It could also be optimized into a single allocation automatically by the compiler (it might already be). The API would not be affected by this improvement, though. I've never seem the compiler apply that optimisation, although I often wish it would. I saw an appender appear a few pages below, that would be an improvement here too I guess. And many many more of the same... This is a lib that spawns a process, I can't imagine why this library should need to allocate a single byte, but it allocates wildly just to perform a trivial pass-through to the OS. It is not a trivial pass-through. Different OSes require different parameter types. Look at the difference between the windows and posix implementations. We are writing a cross-platform library, so whatever we pick will have to be converted on at least one OS. Writing cross-platform code is my life. They are trivial conversions. It could be done on the stack easily. Granted, ideally the compiler could theoretically perform some of those improvements, but it doesn't, and probably depends on other changes to be able to do so anyway. For instance, the compiler couldn't confidently lower the allocations to the stack unless 'scope' arguments worked, so it knew it wouldn't escape the OS call it was being handed to. This module is no exception either, basically all of phobos is like this, and I'd like to see careful consideration made to these details in the future. I too like to avoid memory allocations. But the level to which you require is too stringent. Not even Tango, which made low heap-allocations a priority, avoided allocations in it's process library. But it would be trivial to use the stack in these cases. It doesn't matter if it adds a few lines under the hood. I'm really just trying to make the point that it should ALWAYS be a key consideration, and people should make it habit to think about/criticise these things when accepting code into phobos. This is the standard library, it will be used in more D code than any other library ever. Why skimp here? I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. We cannot cater to all developers. In this case you certainly can, it would be fairly trivial to do. To put all developers through a lot of pain in order to attract the fringe ones is not a practical goal. What pain? Are you saying a stack variable is somehow hard? If that's the case, then there's clearly some other fundamental tools missing from phobos. I think the convenience of std.process is what it is there for. For the highest performance, you are able to call the OS functions directly. No, it's to save (ideally) _all_ programmers the effort of rewriting this module themselves. This is exactly the sort of thing I waste loads of my life doing, rewriting wheels because the author was a PC programmer, and simply didn't give a shit. What a waste of life _ I re-iterate, maybe my points aren't of critical importance in relation to std.process specifically, I'm trying to raise some general issues with phobos submissions (that shouldn't exclude std.process either).
Re: Vote for std.process
On Friday, 12 April 2013 at 07:53:04 UTC, Manu wrote: Which OS are we talking about? What OS runs on an a Nintendo Wii? There's only 24mb of system memory in that machine, can we afford to allocate it frivolously? I wasn't aware that writing games for the Wii involves creating processes in the thousands. I don't necessarily disagree with your point (although I don't agree either). Perhaps in the context of std.process it's not so important... but it's still an opportunity to set a precedent. Honestly, any new module could have appeared for approval at this particular moment and I would have made the same criticisms. Not necessarily picking on std.process in particular, I'm making a point about phobos, and what is considered acceptable. Don't forget that shifting focus when adjusting goals is always a question of tradeoffs. Sacrificing tersity, maintainability, usability, safety etc. for the sake of performance would be a poor move in the case of std.process.
Re: Vote for std.process
On Friday, 12 April 2013 at 08:14:25 UTC, Manu wrote: ... While I generally agree with your point, std.process is hardly the place to start from. It is not performance-critical functionality and it is not basic functionality that may be used in other critical modules. It is blaming consequences instead of reasons, not really practical.
Re: Vote for std.process
Please correct me if I'm wrong. It's not the allocation itself that is the problem it's that it will trigger the GC to stop the world at some point.
Re: Vote for std.process
On Friday, 12 April 2013 at 07:53:04 UTC, Manu wrote: Honestly, any new module could have appeared for approval at this particular moment and I would have made the same criticisms. Observation: this move wasn't something I would encourage. Jesse, the review manager for std.process, has specifically asked: Please place any further comments in the official review thread leaving only your vote and a short comment (there should be no need to reply to anyone). There was ample time to discuss std.process during the informal and formal review. However, the situation with your replies is even worse: they don't even concern std.process in particular. Please consider starting a new thread next time, and stating your criticism aside to whatever poor module comes up the review queue ;)
Re: Vote for std.process
On Friday, 12 April 2013 at 07:53:04 UTC, Manu wrote: On 12 April 2013 17:30, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 07:22:42 UTC, Manu wrote: I agree that spawning processes is a low-frequency operation, but it's a principle I'm trying to illustrate here. My point was that it is not that it's low-frequency, it's that the OS process creation operation is so expensive, that a few memory allocations will not make much of a difference in comparison. It's the same as optimizing memory allocations in a program which is intrinsically disk- or network-bound. Which OS are we talking about? What OS runs on an a Nintendo Wii? There's only 24mb of system memory in that machine, can we afford to allocate it frivolously? Will I avoid phobos as a policy? Yes. Do you create process at all? I always had the impression those environments used real time OS like libraries and multitasking is achieved by co-routines or threads, not real processes. Just a dummy comment from an industry outsider. -- Paulo
Re: Vote for std.process
On 12 April 2013 18:18, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 07:53:04 UTC, Manu wrote: Which OS are we talking about? What OS runs on an a Nintendo Wii? There's only 24mb of system memory in that machine, can we afford to allocate it frivolously? I wasn't aware that writing games for the Wii involves creating processes in the thousands. Again, I'm just suggesting possibilities, and trying to illustrate that it's a STANDARD library, you can never predict where users will want to use it. I don't necessarily disagree with your point (although I don't agree either). Perhaps in the context of std.process it's not so important... but it's still an opportunity to set a precedent. Honestly, any new module could have appeared for approval at this particular moment and I would have made the same criticisms. Not necessarily picking on std.process in particular, I'm making a point about phobos, and what is considered acceptable. Don't forget that shifting focus when adjusting goals is always a question of tradeoffs. Sacrificing tersity, maintainability, usability, safety etc. for the sake of performance would be a poor move in the case of std.process. Can you perhaps quantify how any of those things would be reduced by addressing at least the details I highlight? I'm basically advocating making a habit of using the stack where possible. It's not exactly hard, or cryptic.
Re: Vote for std.process
On 12 April 2013 18:21, Manipulator vo...@kth.se wrote: Please correct me if I'm wrong. It's not the allocation itself that is the problem it's that it will trigger the GC to stop the world at some point. It's just another source of garbage. Which means this function of off limits beneath a routine that's disabled collection, for instance. There are lots of reasons to avoid making unnecessary garbage.
Re: Vote for std.process
On 12 April 2013 18:21, Dicebot m.stras...@gmail.com wrote: On Friday, 12 April 2013 at 08:14:25 UTC, Manu wrote: ... While I generally agree with your point, std.process is hardly the place to start from. It is not performance-critical functionality and it is not basic functionality that may be used in other critical modules. It is blaming consequences instead of reasons, not really practical. And I generally agree with your point. Like I said, I would have made this criticisms of whatever came alone ;)
Re: Vote for std.process
Am Fri, 12 Apr 2013 17:04:08 +1000 schrieb Manu turkey...@gmail.com: I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. Most (GC) allocations could be fixed without breaking the API. I can see 2 places where the API forces _GC_ allocations: * string[string] * thrown Exceptions are allocated with the GC There is no simple solution for these. Maybe we'll have a hashtable in the standard library at some point which will allow to use custom allocators. Then std.process could use these. Exceptions require some thinking. You can of course allocate them using any allocator, but freeing them is difficult... If you want to get rid of all (not only GC) allocations, there's another issue: As D strings are not zero terminated we'll always have allocations passing those to C code. Maybe we should have a cstring type in phobos which would just be a string which is guaranteed to be zero terminated. http://dpaste.dzfl.pl/e76aa995 (needs some inout though)
Re: Vote for std.process
Am Fri, 12 Apr 2013 06:46:51 +0200 schrieb Jesse Phillips jessekphillip...@gmail.com: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. std.process by Lars Kyllingstad and Steven Schveighoffer is a suggested improvement to the existing std.process and is a major change to the API. The original API remains but these will be going through deprecation. In summary of the discussion there was concern of the use of Error and Exception. Lars is very interested in getting the standard hierarchy http://wiki.dlang.org/DIP33 thus leaving these at the safest level to allow for change without breaking code. Please place any further comments in the official review thread leaving only your vote and a short comment (there should be no need to reply to anyone). Docs: http://www.kyllingen.net/code/std-process2/phobos-prerelease/std_process.html Source: https://github.com/kyllingstad/phobos/blob/std-process2/std/process.d Friday April 19 PST will be the last day of voting. Yes!
Re: Vote for std.process
On Friday, 12 April 2013 at 08:26:48 UTC, Manu wrote: Again, I'm just suggesting possibilities, and trying to illustrate that it's a STANDARD library, you can never predict where users will want to use it. So you are talking about an imaginary system where memory allocation is expensive but process creation is cheap. It's impossible to design and optimize software for some intangible goals. None of the systems that D targets, or aims to target, meet that criteria. Can you perhaps quantify how any of those things would be reduced by addressing at least the details I highlight? I'm basically advocating making a habit of using the stack where possible. It's not exactly hard, or cryptic. Yes. However, I suggest the following instead: Please rewrite some part of std.process with performance in mind, and post it here for review. This way, we can analyze the benefits and drawbacks based on a concrete example, instead of vapor and hot air.
Re: Vote for std.process
Am Fri, 12 Apr 2013 17:04:08 +1000 schrieb Manu turkey...@gmail.com: toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions. Are you sure this is possible in this specific case? The execve manpage says: execve() does not return on success, and the text, data, bss, and _stack_ of the calling process are overwritten by that of the program loaded. There is no documentation if the environment variables are copied internally in execve.
Re: Vote for std.process
On Fri, 12 Apr 2013 09:38:53 +0100, Vladimir Panteleev vladi...@thecybershadow.net wrote: On Friday, 12 April 2013 at 08:26:48 UTC, Manu wrote: Again, I'm just suggesting possibilities, and trying to illustrate that it's a STANDARD library, you can never predict where users will want to use it. So you are talking about an imaginary system where memory allocation is expensive but process creation is cheap. It's impossible to design and optimize software for some intangible goals. None of the systems that D targets, or aims to target, meet that criteria. I don't understand why you're so against these comments. There is no real argument here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less. Yes, premature optimisation is generally frowned upon in user code, but library code should ideally be optimised. Should this percieved lack of optimisation prevent it's inclusion now, no because optimisation wont change the API (it may extend it however). Can you perhaps quantify how any of those things would be reduced by addressing at least the details I highlight? I'm basically advocating making a habit of using the stack where possible. It's not exactly hard, or cryptic. Yes. However, I suggest the following instead: Please rewrite some part of std.process with performance in mind, and post it here for review. This way, we can analyze the benefits and drawbacks based on a concrete example, instead of vapor and hot air. Fair point, after all someone has to do the work and it seems logical that the person with the most understand of the issue should so it. Whether he has the time however is the Q/problem. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Vote for std.process
On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote: I don't understand why you're so against these comments. There is no real argument here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less. This is a fallacy. Performance is not free. Somebody must do the work. D contributors do not have an infinity of time and motivation. Optimizing code often implies making it more complicated. Straight-forward code is self-documenting. More complicated code is harder to read and review. More complicated code more easily hides bugs - including security bugs, such as buffer overflows. Maintaining optimized code requires understanding not only the high-level logic, but also the low-level optimization details. The benefits of optimizing std.process are likely to be so small, as to be difficult to measure. Would you still say that the above costs are worth the nearly-intangible gain?
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 04:43, Manu turkey...@gmail.com wrote: On 2 April 2013 21:18, Iain Buclaw ibuc...@ubuntu.com wrote: On Monday, 25 March 2013 at 01:59:07 UTC, Manu wrote: I'm keen to share a room/ride with someone. I have no plans yet. Arrive on the 29th of April in SFO. I look set to be arriving on the 29th at SFO also. Maybe could hook up! :) Sure! I arrive at midday. I see you pondering flying to SJC? It's an easy bart ride from SFO - San Jose, I'd save the money if I were you, the train is convenient, and it sounds like there might be cars heading past. When do you land? Yeah... I decided that my pockets don't squeeze that tight. So SFO it will be! (Plus I think will only have the flights paid for, not the expensive hotels). -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Vote for std.process
On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote: ... It is not something to discuss in this topic. Yes, phobos has allocations issues. Yes, it is worth discussing. Does std.process do something special in that sense? No, possible problems are more related to lack of general configurable allocator. Please do not fill voting thread with noise if you have no _blocking_ objections.
Re: Vote for std.process
On Friday, 12 April 2013 at 08:34:15 UTC, Johannes Pfau wrote: Am Fri, 12 Apr 2013 17:04:08 +1000 schrieb Manu turkey...@gmail.com: I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. Most (GC) allocations could be fixed without breaking the API. I can see 2 places where the API forces _GC_ allocations: * string[string] * thrown Exceptions are allocated with the GC There is no simple solution for these. Maybe we'll have a hashtable in the standard library at some point which will allow to use custom allocators. Then std.process could use these. Exceptions require some thinking. You can of course allocate them using any allocator, but freeing them is difficult... If you want to get rid of all (not only GC) allocations, there's another issue: As D strings are not zero terminated we'll always have allocations passing those to C code. Maybe we should have a cstring type in phobos which would just be a string which is guaranteed to be zero terminated. http://dpaste.dzfl.pl/e76aa995 (needs some inout though) This is the Object Pascal way, although you can reach to a point where you get language native strings and C strings mixed all over the place. Not to mention that C strings should not be allowed in safe code. -- Paulo
Re: Vote for std.process
On Fri, Apr 12, 2013 at 06:46:51AM +0200, Jesse Phillips wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes! --T
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On 2013-04-10 21:30, Walter Bright wrote: I agree. The way to do it is to support both the old and the new ways for now. Anyone want to do a pull req? Also, this should be put in bugzilla as an enhancement request. Let's agree on specifics before writing enhancement request, and possibly make a single one-time breaking change instead of several short-sighted ones. With type A being -offilename and type B being -of=filename, how about: 1) prevent any newly created flags of type A 2) migrate all A flags to B flags. Here's one possible way to achieve this (say in next dmd release): dmd -offilename main.d //works but generates a warning for now, and error after a certain time passed dmd -old_flag -offilename main.d //works and doesn't generate a warning. dmd -new_flag -of=filename main.d //works. After a certain time passed, -new_flag is implied Note, A and B flags can't be mixed, eg: -offilename -Ddoc=dir will give error, in all 3 cases above (ie for flags that are currently in the A style). 2b) Alternative: use a new binary name (dmd2) instead of -newflag. I don't like this as much somehow. 3) can we deprecate the current behavior where one can pass the file name without extension (main vs main.d) as source? Consistency is better than avoiding to type those 2 characters. I created a pathological case with main.d is conflicting with main.d.d (with different contents). Which one do you think is called when we call rdmd main.d ? Note, I raised a very analogous concern here https://github.com/D-Programming-Language/dmd/pull/1871#issuecomment-16101987 regarding naming of object files in a flat hierarchy (see my example with dmd -c -oq foo/mod.d foo_mod.d) 4) The current strategy of rdmd is to treat as input arguments anything after the first source file: rdmd main.d myfirstprogramarg // a bit awkward, especially with optional extension it gets hard to parse visually. This is error prone, and inconsistent with dmd's behavior, which is: dmd src1.d -run main.d myfirstprogramarg //a bit awkward, need to split the source from the main file. I suggest instead something simpler, explicit and consistent, using -args as a dmd command line argument, that would just work as well with rdmd: dmd main.d src1.d -args myfirstprogramarg rdmd main.d -args myfirstprogramarg 5) currently we distinguish rdmd's arguments from dmd's arguments via '--' vs '-'. A better way IMO would be to have a special flag indicating the start of dmd's (or gdc/ldc...) flags: eg rdmd --chatty --dflags -version=myversion main.d Timothee Cour
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On 11 April 2013 17:57, Paulo Pinto pj...@progtools.org wrote: Am 11.04.2013 17:20, schrieb John Colvin: On Thursday, 11 April 2013 at 15:15:09 UTC, Andrej Mitrovic wrote: On 4/11/13, Jonas Drewsen nospam4...@hotmail.com wrote: By I think that would be the way to go I did not necessary refer to having both formats supported at the same time but simply to use the standard way that people know. Unix != universal standard. Next best thing. Actually I would like to know how the desktop world would look like had Apple bought BeOS instead. This would mean no UNIX on the desktop, assuming Apple would have won a similar market as of today. -- Paulo Linux would have still taken off... and UNIX would still be relevant on the server market. :) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Vote for std.process
On Friday, 12 April 2013 at 04:46:52 UTC, Jesse Phillips wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. Yes! :)
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen.
Re: Vote for std.process
On Fri, 12 Apr 2013 10:17:54 +0100, Vladimir Panteleev vladi...@thecybershadow.net wrote: On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote: I don't understand why you're so against these comments. There is no real argument here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less. This is a fallacy. Nope, you're attacking a Strawman here. At most my statement is incomplete because it doesn't address the costs (which I left out for brevity). Performance is not free. I never claimed it was (thus the Strawman). I simply claimed std.process would be better if it performed faster and allocated less - which is true. Somebody must do the work. D contributors do not have an infinity of time and motivation. I know this as well as anyone, my own free time is in short supply. Optimizing code often implies making it more complicated. Straight-forward code is self-documenting. More complicated code is harder to read and review. More complicated code more easily hides bugs - including security bugs, such as buffer overflows. Maintaining optimized code requires understanding not only the high-level logic, but also the low-level optimization details. The benefits of optimizing std.process are likely to be so small, as to be difficult to measure. All true. However, complexity can and should be packaged in such a way as to localise, and this localised complex code should be tested to death and maintained by someone who understands it. It should be bracketed by sufficient comments and warnings about how/why it does what it does. The resulting packaged complexity, with it's associated cost can be re-used many times over for all the benefit it gives. Stack allocating the environment variables need not be a localised improvement but could be a standard library function which can be reused, for example. Would you still say that the above costs are worth the nearly-intangible gain? nearly-intangible is wrong. Library code is code which is used by (hopefully) millions of people, writing millions of applications, running for millions of hours, on millions of systems, creating thousands of processes, etc.. In short, a little effort now pays massive dividends over it's lifetime. So, yes, IMO the costs shown above are worth the resulting gains. D is a systems programming language, there is hope that it will penetrate a wide range of systems and environments - sure in many cases a little bit of memory use or performance loss is unimportant, but for many it will be the decisive factor which makes D unusable there. D is constantly being compared to other languages on the basis of performance, so it's clearly an important aspect of D's success. Library code needs to work first time and work well or people will roll their own wasting time, energy and in many cases getting some aspect of it just plain wrong. Again, I'm not suggesting (nor was Manu I think) that this in any way blocks the inclusion of std.process. But I do share his desire for performance to sometimes be a little higher up the priority list. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Vote for std.process
On Fri, 12 Apr 2013 05:46:51 +0100, Jesse Phillips jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. Yes. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Vote for std.process
On Friday, 12 April 2013 at 04:46:52 UTC, Jesse Phillips wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. std.process by Lars Kyllingstad and Steven Schveighoffer is a suggested improvement to the existing std.process and is a major change to the API. The original API remains but these will be going through deprecation. Yes. A compiler warning like for deprecation should added for experimental IMO rather than use exp. vs std.
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On Friday, 12 April 2013 at 09:54:30 UTC, Iain Buclaw wrote: On 11 April 2013 17:57, Paulo Pinto pj...@progtools.org wrote: Am 11.04.2013 17:20, schrieb John Colvin: On Thursday, 11 April 2013 at 15:15:09 UTC, Andrej Mitrovic wrote: On 4/11/13, Jonas Drewsen nospam4...@hotmail.com wrote: By I think that would be the way to go I did not necessary refer to having both formats supported at the same time but simply to use the standard way that people know. Unix != universal standard. Next best thing. Actually I would like to know how the desktop world would look like had Apple bought BeOS instead. This would mean no UNIX on the desktop, assuming Apple would have won a similar market as of today. -- Paulo Linux would have still taken off... and UNIX would still be relevant on the server market. :) Not to mention supercomputers. There's a reason why they use(d) UNIX and linux and it's not because of apple.
Re: Vote for std.process
On 2013-04-12 09:04, Manu wrote: I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. Have you tried Tango? For most functions that do some array or string processing you have the option of passing your own buffer. If the buffer is too small, the function will allocate, otherwise it won't. Docs: http://www.dsource.org/projects/tango/docs/current/ Tango D2: https://github.com/SiegeLord/Tango-D2 -- /Jacob Carlborg
Re: Formal Review of std.process
On Thursday, 11 April 2013 at 18:20:02 UTC, Jacob Carlborg wrote: Does it throw an Error, that's bad. Code should basically never create and throw an Error or Exception. It should always be a derived type from either of these classes. I completely agree. However, Phobos already throws straight Exceptions all over the place (just grep for enforce), so the few places where std.process does so is just in keeping with tradition. ;) Jokes aside, using Error/Exception until we have decided on a proper exception hierarchy will help reduce the breakage when we do. Lars
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 11:01, Manu turkey...@gmail.com wrote: On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen. Okie doke, what day are you heading back? -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Vote for std.process
On Friday, 12 April 2013 at 10:14:35 UTC, Regan Heath wrote: All true. However, complexity can and should be packaged in such a way as to localise, and this localised complex code should be tested to death and maintained by someone who understands it. It should be bracketed by sufficient comments and warnings about how/why it does what it does. The resulting packaged complexity, with it's associated cost can be re-used many times over for all the benefit it gives. Stack allocating the environment variables need not be a localised improvement but could be a standard library function which can be reused, for example. Performant abstractions. Like the much-awaited allocator design? Either way, they can only diminuate, not remove the costs. Would you still say that the above costs are worth the nearly-intangible gain? nearly-intangible is wrong. Library code is code which is used by (hopefully) millions of people, writing millions of applications, running for millions of hours, on millions of systems, creating thousands of processes, etc.. In short, a little effort now pays massive dividends over it's lifetime. So, yes, IMO the costs shown above are worth the resulting gains. D is constantly being compared to other languages on the basis of performance, so it's clearly an important aspect of D's success. Library code needs to work first time and work well or people will roll their own wasting time, energy and in many cases getting some aspect of it just plain wrong. But once again, you speak in vague terms. Consider the following hypothetical decisions and outcomes: 1. std.process is left at is. One user is angry / turned away because it performs 0.1% slower than it can be. 2. std.process is rewritten to minimize allocations. Code complexity goes up, new improvements are challenging to add; bugs pop up and go unfixed for a while because fewer programmers are qualified or willing to commit the effort of making correct fixes. More people are angry / turned away from D because its standard library is buggy. Of course, the above is an exaggerated illustration. But would optimizing all code left and right really make more D users happier? There's also the question of priorities. Would you rather than effort is spent on optimizing std.process (and dealing with all the fallout from any such optimizations), or working on something that is acutely missing and hurting D? D is a systems programming language, there is hope that it will penetrate a wide range of systems and environments - sure in many cases a little bit of memory use or performance loss is unimportant, but for many it will be the decisive factor which makes D unusable there. This is surely an exaggeration. D does not attempt to please everyone out there who is choosing a programming language for their next project. There is no such language, nor can one exist. One has to accept that D has a number of goals, none of which are absolute, but merely point towards a certain, but not overly specific, point in the multidimensional matrix of trade-offs. D never was about achieving maximum performance in all possible cases.
Re: Formal Review of std.process
On Friday, 12 April 2013 at 06:36:31 UTC, Jacob Carlborg wrote: On 2013-04-11 22:31, Lars T. Kyllingstad wrote: It's implemented using a class, but you use it like an object. I think that is more important. I see. I know there are some people here that are very picky about these things. Then let me ask you/them this: Would you prefer it was implemented as a module-level @property function that returns a singleton class instance instead? In other words, would you have the same API with slightly worse performance and more complicated implementation, *just* for the sake of a strict naming convention? Note that 'environment' (with that name) was added to std.process years ago. As Jesse mentioned, the fact that it was a class was just hidden from documentation with an alias. Unfortunately, this prevented proper documentation of its members, so changed it. Lars
Release management.
[x-post from /r/programming] It's not the most detailed article, but it does have a few interesting points. Perhaps there is something we can learn from their experience. http://phonegap.com/2012/04/12/rolling-releases-how-apache-cordova-becomes-phonegap-and-why/ Enjoy :)
Re: Formal Review of std.process
On Thursday, 11 April 2013 at 15:43:18 UTC, Steven Schveighoffer wrote: A couple minor comments: 1. I have two issues with Error being used. One is that we should have a specific type that is thrown, not raw Error type. Second is that I think in situations where the error is due to an incorrect parameter, it should be an exception not an error (and not a straight Exception either!). Let's go through the places where an Error or Exception is thrown: spawnProcess() throws RangeError when args[] is empty, but this is just the normal behaviour of arrays, and with -release/-noboundscheck it just segfaults. As such, there is little point in specifying this in the documentation. I'll remove it. (Honestly, I don't know why I put it in there in the first place. It may have had something to do with me being thoroughly annoyed over other the lack of exception specifications in Phobos documentation at large. I used to like enforce(), but now I think it has given D programmers a way too lax attitude towards error handling.) kill() throws Error if the code/signal is negative. I suspect the cases where this number comes directly from user input are so few and far between that it is reasonable to expect the programmer to ensure that it is nonnegative. In principle, on POSIX we don't need the check, because POSIX kill() will return an invalid signal error if you try to give it a negative number. On Windows, however, TerminateProcess() takes an unsigned integer (which is why I added the check in the first place), and I think we should strive to have the same behaviour on all platforms to the extent possible. I would not be strongly opposed to making this an Exception of some kind, though, if you think there is a good reason to do so. pipeProcess throws Error on an invalid combination of Redirect flags, and ProcessPipes does the same on an attempt to access a non-redirected stream. Are we in agreement that these two are always programming errors? escapeShellCommand() throws Error if the input contains \0, \r or \n. Here, it is very likely that the arguments are user input, but it would be very strange application code that somehow allowed those three characters to enter the input. Even so, it may be better to make it an Exception, just to be safe. Lars
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 11:01, Manu turkey...@gmail.com wrote: On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen. Booked! Hotel is here: http://goo.gl/maps/vNGCt I'll be checking in on the 27th April until 4th May. Can hook up when you land. :) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Formal Review of std.process
On Friday, 12 April 2013 at 11:31:29 UTC, Lars T. Kyllingstad wrote: Let's go through the places where an Error or Exception is thrown: [...] I forgot environment, whose methods throw straight Exceptions. As I've said before, I am not happy about this at all, but it *is* how environment is implemented in the old std.process too. Thus, leaving it like this until we figure out the whole exception mess is, IMO, better than breaking it twice. Lars
Was: Re: Vote for std.process
I've moved this to another thread to allay complaints. Vladimir Panteleev vladi...@thecybershadow.net On Friday, 12 April 2013 at 10:14:35 UTC, Regan Heath wrote: All true. However, complexity can and should be packaged insuch a way as to localise, and this localised complex codeshould be tested to death and maintained by someone whounderstands it. It should be bracketed by sufficient commentsand warnings about how/why it does what it does. The resultingpackaged complexity, with it's associated cost can be re-usedmany times over for all the benefit it gives. Stack allocating the environment variables need not be alocalised improvement but could be a standard library functionwhich can be reused, for example. Performant abstractions. Like the much-awaited allocator design? Either way, they can only diminuate, not remove the costs. non localised improvements have a fixed cost and ever increasing benefit - is the point I was making here. Agreed, you cannot remove the cost, and in fact well written reusable code often carries a slightly higher cost by it's very nature. Would you still say that the above costs are worth the nearly-intangible gain? nearly-intangible is wrong. Library code is code which isused by (hopefully) millions of people, writing millions ofapplications, running for millions of hours, on millions ofsystems, creating thousands of processes, etc.. In short, alittle effort now pays massive dividends over it's lifetime. So, yes, IMO the costs shown above are worth the resultinggains. D is constantly being compared to other languages on the basisof performance, so it's clearly an important aspect of D'ssuccess. Library code needs to work first time and work well or peoplewill roll their own wasting time, energy and in many casesgetting some aspect of it just plain wrong. But once again, you speak in vague terms. The initial point was a vague one, not a specific one. Manu wasn't attempting to block std.process, he had a general concern which I share. Consider the following hypothetical decisions and outcomes: 1. std.process is left at is. One user is angry / turned awaybecause it performs 0.1% slower than it can be. It very much matters *who* that 1 user is. And, the count may be higher, and we might never hear from these people as they find other solutions. We're lucky that some people who try D and have issues tell us about them, they may be 5% of the total for all we know. 2. std.process is rewritten to minimize allocations. Codecomplexity goes up, new improvements are challenging to add; bugspop up and go unfixed for a while because fewer programmers arequalified or willing to commit the effort of making correctfixes. More people are angry / turned away from D because itsstandard library is buggy. Of course, the above is an exaggerated illustration. Indeed, you've downplayed point #1 and exaggerated point #2. In reality the suggested improvements would add only very minor complexity and prevent none of the current crop of contributors from working with/on std.process. But would optimizing all code left and right really make more D users happier? Yes, as well as the users of their applications. True, none of them will even realise they could have been less happy, so none of them will realise the effort that went into it, but all of them will be better off. There's also the question of priorities. Would you rather thaneffort is spent on optimizing std.process (and dealing with allthe fallout from any such optimizations), or working on somethingthat is acutely missing and hurting D? Add the missing items, without a doubt - which is why no-one is suggesting blocking std.process over this issue. D is a systems programming language, there is hope that it will penetrate a wide range of systems and environments - sure inmany cases a little bit of memory use or performance loss isunimportant, but for many it will be the decisive factor whichmakes D unusable there. This is surely an exaggeration. Why? There exist platforms and environments where memory and performance are concerns, if the D standard library code is not careful in it's use of both then it will be less suitable than C (for example) and so D will not penetrate those platforms. Manu is using D for games development on modern high-end gaming PCs and he is still concerned with memory and performance. So, there's 2 very different cases where memory and performance are still a concern, and .. if they become too much of a concern another solution will have to be sought.. and that's bad news for D. D does not attempt to please everyone out there who is choosing a programming language for their next project. There is no suchlanguage, nor can one exist. One has to accept that D has anumber of goals, none of which are absolute, but merely pointtowards a certain, but not overly specific,
Re: Vote for std.process
12-Apr-2013 12:26, Manu пишет: On 12 April 2013 18:18, Vladimir Panteleev vladi...@thecybershadow.net mailto:vladi...@thecybershadow.net wrote: On Friday, 12 April 2013 at 07:53:04 UTC, Manu wrote: Which OS are we talking about? What OS runs on an a Nintendo Wii? There's only 24mb of system memory in that machine, can we afford to allocate it frivolously? I wasn't aware that writing games for the Wii involves creating processes in the thousands. Again, I'm just suggesting possibilities, and trying to illustrate that it's a STANDARD library, you can never predict where users will want to use it. Very true. -- Dmitry Olshansky
Re: Vote for std.process
12-Apr-2013 11:04, Manu пишет: On 12 April 2013 14:46, Jesse Phillips jessekphillip...@gmail.com mailto:jessekphillip...@gmail.com wrote: It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no. I didn't think I had much of an opinion on std.process, but I just gave it a quick once over and noticed, relating to our recent discussions about flagrant GC usage throughout phobos, that this is riddled with the exact sorts of issues I've been talking about: string[string] is used in the main API to receive environment variables; perhaps kinda convenient, but it's impossible to supply environment variables with loads of allocations. toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions. Most of this is unsolvable until there is a STANDARD interface/policy/hooks for allocators in Phobos. All work to get around the cases like that at the moment is just a waste of time or specific hacks for particular bottlenecks. I did a fair amount of these in std.regex to make matching GC-free and almost heap-free - it preallocates on the first call. It's an example of a necessary hack and it's not pretty. More then that I'm sure I'd have to re-write the whole memory allocation code once we (finally!) have he standard way to do it. TL;DR - until using allocators is easy and straight-forward (and standard) there would be no end of GC-only hegeomny 'cause it's easier/simpler. Not much to discuss ;) String concatenation is rampant! Look at this code to parse the env variables (which are already an AA): foreach (var, val; childEnv) envz[pos++] = (var~'='~val~'\0').ptr; Horrible... But let's focus on the API parts. The internals could be fixed with a pull request or two. Freaking out because of a GC happy code doesn't help those waiting for *any* sane process management module in std. -- Dmitry Olshansky
A little game, Phobos and more
In this post I use D and Phobos, to show some problems and to
suggest some improvements.
This little D program is one of the tasks of the Rosettacode
site, it allows to play a basic Rock-Paper-Scissors game on the
console against the computer:
- - - - - - - - - - - - - - - - -
import std.stdio, std.random, std.string, std.array;
immutable order = [rock, paper, scissors];
uint[string] choiceFrequency;
immutable string[string] whatBeats;
nothrow pure static this() {
whatBeats = [paper:scissors,
scissors: rock,
rock: paper];
}
string checkWinner(in string a, in string b) pure nothrow {
if (b == whatBeats[a])
return b;
else if (a == whatBeats[b])
return a;
return ;
}
string getRandomChoice() {
if (choiceFrequency.length == 0)
return order[uniform(0, $)];
const choices = choiceFrequency.keys;
return whatBeats[choices[choiceFrequency.values.dice]];
}
void main() {
writeln(Rock-paper-scissors game);
while (true) {
write(Your choice: );
immutable humanChoice = readln.strip.toLower;
if (humanChoice.empty)
break;
if (humanChoice !in whatBeats) {
writeln(Wrong input: , humanChoice);
continue;
}
immutable compChoice = getRandomChoice;
write(Computer picked , compChoice, , );
choiceFrequency[humanChoice]++;
immutable winner = checkWinner(humanChoice, compChoice);
if (winner.empty)
writeln(nobody wins!);
else
writeln(winner, wins!);
}
}
The program has a choiceFrequency string-count associative array
that keeps counts of the human choices. The computer chooses to
play with a biased dice to beat the most probable (zero-order)
human choice (it's easy to add a first-order frequency table that
counts the adjacent pairs of human choices to improve a little
the computer playing skills).
That program is simple, but there are several small things to
improve or just discuss about.
- - - - - - - - - - - - - - - - -
1) Currently you can't write:
static this() nothrow pure {
You have to write:
nothrow pure static this() {
http://d.puremagic.com/issues/show_bug.cgi?id=6677
- - - - - - - - - -
2) This allocates a global immutable associative array:
immutable string[string] whatBeats;
nothrow pure static this() {
whatBeats = [paper:scissors,
scissors: rock,
rock: paper];
}
I am glad that since some time whatBeats is allowed to be
immutable. But what we want to write is more naturally a global
variable like:
immutable whatBeats = [paper:scissors,
scissors: rock,
rock: paper];
This problem is being worked on (I think there is a patch, I
don't remember what).
- - - - - - - - - -
3) Instead of this:
string getRandomChoice() {
if (choiceFrequency.length == 0)
I'd like to use std.array.empty:
if (choiceFrequency.empty)
http://d.puremagic.com/issues/show_bug.cgi?id=6409
- - - - - - - - - -
4) Choosing a single item randomly from a sequence is a very
common operation, so instead of this:
return order[uniform(0, $)];
A more natural function to use is choice as in Python:
return order.choice;
http://d.puremagic.com/issues/show_bug.cgi?id=4851
- - - - - - - - - -
5) This not efficient, and I'd like something shorter and faster:
const choices = choiceFrequency.keys;
return whatBeats[choices[choiceFrequency.values.dice]];
But let's skip this for now.
- - - - - - - - - -
6) I don't like to write strings more than once, because they are
magic constants:
immutable order = [rock, paper, scissors];
immutable string[string] whatBeats;
nothrow pure static this() {
whatBeats = [paper:scissors,
scissors: rock,
rock: paper];
}
With a little function you don't need the associative array:
immutable order = [rock, paper, scissors];
string whatBeats(in string a) pure /*nothrow*/
in {
assert(order.canFind(a));
} body {
return order[(order.countUntil(a) + 1) % order.length];
}
Unfortunately whatBeats() can't be nothrow (why is countUntil on
the array not nothrow?), so checkWinner() can't be nothrow.
I am not so sure whatBeats() is better than the whatBeats[]
associative array. The function is more DRY, but it contains code
that needs to be unittested, while (in theory) that little
associative array just need one inspection to tell if it's
correct. In the end I have chosen the little function.
- - - - - - - - - -
7) It's generally a bad idea to repeat a raw type like string
everywhere, it's better to define an alias and use it instead:
alias Choice = string;
Even better is to use a strong typedef:
alias Choice = Typedef!string;
Thankfully now this Typedef works with limited changes to the
program, like:
immutable humanChoice =
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On Friday, 12 April 2013 at 10:39:20 UTC, John Colvin wrote: On Friday, 12 April 2013 at 09:54:30 UTC, Iain Buclaw wrote: On 11 April 2013 17:57, Paulo Pinto pj...@progtools.org wrote: Am 11.04.2013 17:20, schrieb John Colvin: On Thursday, 11 April 2013 at 15:15:09 UTC, Andrej Mitrovic wrote: On 4/11/13, Jonas Drewsen nospam4...@hotmail.com wrote: By I think that would be the way to go I did not necessary refer to having both formats supported at the same time but simply to use the standard way that people know. Unix != universal standard. Next best thing. Actually I would like to know how the desktop world would look like had Apple bought BeOS instead. This would mean no UNIX on the desktop, assuming Apple would have won a similar market as of today. -- Paulo Linux would have still taken off... and UNIX would still be relevant on the server market. :) Not to mention supercomputers. There's a reason why they use(d) UNIX and linux and it's not because of apple. Sure but I was speaking about the home market, not the server market. Then people would need to bash both Apple and Microsoft for not being UNIX like. Although I like UNIX architecture, it is not the be all end all of operating system architecture. I was lucky enough to play with various OS so far. -- Paulo
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On Friday, 12 April 2013 at 09:54:30 UTC, Iain Buclaw wrote: On 11 April 2013 17:57, Paulo Pinto pj...@progtools.org wrote: Am 11.04.2013 17:20, schrieb John Colvin: On Thursday, 11 April 2013 at 15:15:09 UTC, Andrej Mitrovic wrote: On 4/11/13, Jonas Drewsen nospam4...@hotmail.com wrote: By I think that would be the way to go I did not necessary refer to having both formats supported at the same time but simply to use the standard way that people know. Unix != universal standard. Next best thing. Actually I would like to know how the desktop world would look like had Apple bought BeOS instead. This would mean no UNIX on the desktop, assuming Apple would have won a similar market as of today. -- Paulo Linux would have still taken off... and UNIX would still be relevant on the server market. :) Linux took off because it provided a way to port UNIX software at cost zero. The companies where I developed in commercial UNIX platforms they only used Linux as a way to avoid paying UNIX licenses, not because Linux was something great. The enterprise world only cares about money. -- Paulo
Re: Vote for std.process
On 12 April 2013 18:34, Johannes Pfau nos...@example.com wrote: Am Fri, 12 Apr 2013 17:04:08 +1000 schrieb Manu turkey...@gmail.com: I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. Most (GC) allocations could be fixed without breaking the API. I can see 2 places where the API forces _GC_ allocations: * string[string] * thrown Exceptions are allocated with the GC There is no simple solution for these. Maybe we'll have a hashtable in the standard library at some point which will allow to use custom allocators. Then std.process could use these. Exceptions require some thinking. You can of course allocate them using any allocator, but freeing them is difficult... Exceptions are exceptional, I don't think that's an issue. the string[string] in the api is guaranteed garbage, and will never be changed. What's the advantage of using an AA in this case? If you want to get rid of all (not only GC) allocations, there's another issue: As D strings are not zero terminated we'll always have allocations passing those to C code. They are being passed through to a C function that doesn't retain the pointer; they can be allocated on the stack. Maybe we should have a cstring type in phobos which would just be a string which is guaranteed to be zero terminated. Maybe, although the point here is to wrap up and hide the C API, so that's not useful here. A helper in phobos for allocating a (c)string on the stack may be useful...
Re: Vote for std.process
On 12 April 2013 18:34, Johannes Pfau nos...@example.com wrote: Am Fri, 12 Apr 2013 17:04:08 +1000 schrieb Manu turkey...@gmail.com: I've said before, I sadly have to avoid phobos like the plague. Some modules (like this one) that provide fundamental functionality - not just helper functions - can't be avoided. Requirements for those should be extra strict in my opinion. Most (GC) allocations could be fixed without breaking the API. I can see 2 places where the API forces _GC_ allocations: * string[string] * thrown Exceptions are allocated with the GC There is no simple solution for these. Maybe we'll have a hashtable in the standard library at some point which will allow to use custom allocators. Then std.process could use these. Exceptions require some thinking. You can of course allocate them using any allocator, but freeing them is difficult... Exceptions are exceptional, I don't think that's an issue. the string[string] in the api is guaranteed garbage, and will never be changed. What's the advantage of using an AA in this case? If you want to get rid of all (not only GC) allocations, there's another issue: As D strings are not zero terminated we'll always have allocations passing those to C code. They are being passed through to a C function that doesn't retain the pointer; they can be allocated on the stack. Maybe we should have a cstring type in phobos which would just be a string which is guaranteed to be zero terminated. Maybe, although the point here is to wrap up and hide the C API, so that's not useful here. A helper in phobos for allocating a (c)string on the stack may be useful...
Re: dmd command line options bad design: -offilename, -Ddocdir etc.
On Friday, 12 April 2013 at 12:12:18 UTC, Paulo Pinto wrote: On Friday, 12 April 2013 at 09:54:30 UTC, Iain Buclaw wrote: On 11 April 2013 17:57, Paulo Pinto pj...@progtools.org wrote: Am 11.04.2013 17:20, schrieb John Colvin: On Thursday, 11 April 2013 at 15:15:09 UTC, Andrej Mitrovic wrote: On 4/11/13, Jonas Drewsen nospam4...@hotmail.com wrote: By I think that would be the way to go I did not necessary refer to having both formats supported at the same time but simply to use the standard way that people know. Unix != universal standard. Next best thing. Actually I would like to know how the desktop world would look like had Apple bought BeOS instead. This would mean no UNIX on the desktop, assuming Apple would have won a similar market as of today. -- Paulo Linux would have still taken off... and UNIX would still be relevant on the server market. :) Linux took off because it provided a way to port UNIX software at cost zero. The companies where I developed in commercial UNIX platforms they only used Linux as a way to avoid paying UNIX licenses, not because Linux was something great. The enterprise world only cares about money. -- Paulo I think we can probably agree that linux has become something quite great, despite that not being the initial reason for adoption. Even if you don't like the basic architecture that much, the community and frameworks built on top of it have been spectacular.
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 11:37:14 UTC, Regan Heath wrote:
I've moved this to another thread to allay complaints.
Thanks!
I completely agree that if code can be made more performant
without a significant increase in complexity, then we should do
so. While it is mostly (but not entirely) irrelevant in the
context of std.process, it is a problem that should be tackled in
Phobos as a whole. Several things could/should be done:
It would be nice to have some sugar on top of alloca(), the use
of which is usually considered bad practice. Someone
(bearophile?) once suggested static arrays whose length is
determined at runtime, which would be a great addition to the
language:
void foo(int n)
{
int[n] myArr;
...
}
Furthermore, we need to get the allocator design in place. In
SciD, I use David Simcha's region allocator to allocate temporary
workspace, and it works really well. The only times I use 'new'
is when I need persistent memory (e.g. for a return value) and
the user-supplied buffer is too small. Phobos would greatly
benefit from doing this too.
Finally, an example from the new std.process which got some heavy
criticism in the other thread:
envz[pos++] = (var~'='~val~'\0').ptr;
I have been operating under the assumption that the compiler is
smart enough to make the above a single allocation. If it isn't,
I would consider it a compiler issue.
That said, I am aware that std.process could be improved in some
places.
Lars
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 12:30:09 UTC, Lars T. Kyllingstad wrote: Finally, an example from the new std.process which got some heavy criticism in the other thread: envz[pos++] = (var~'='~val~'\0').ptr; I have been operating under the assumption that the compiler is smart enough to make the above a single allocation. If it isn't, I would consider it a compiler issue. Multiple chained array concatenations are performed at once, using the _d_arraycatnT function in Druntime.
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 12:43:57 UTC, Vladimir Panteleev wrote: On Friday, 12 April 2013 at 12:30:09 UTC, Lars T. Kyllingstad wrote: Finally, an example from the new std.process which got some heavy criticism in the other thread: envz[pos++] = (var~'='~val~'\0').ptr; I have been operating under the assumption that the compiler is smart enough to make the above a single allocation. If it isn't, I would consider it a compiler issue. Multiple chained array concatenations are performed at once, using the _d_arraycatnT function in Druntime. Good to know. Lars
Allocatoin policy in Phobos - Was: Vote for std.process
On 12 April 2013 21:00, Vladimir Panteleev vladi...@thecybershadow.netwrote: Consider the following hypothetical decisions and outcomes: 1. std.process is left at is. One user is angry / turned away because it performs 0.1% slower than it can be. 2. std.process is rewritten to minimize allocations. Code complexity goes up, new improvements are challenging to add; bugs pop up and go unfixed for a while because fewer programmers are qualified or willing to commit the effort of making correct fixes. More people are angry / turned away from D because its standard library is buggy. Of course, the above is an exaggerated illustration. But would optimizing all code left and right really make more D users happier? Just to be clear, I'm not arguing optimisation for performance here, I'm arguing intolerance for __unnecessary__ allocations as a policy, or at least a habit. There's a whole separate thread on the topic of fighting unnecessary garbage, and having the ability to use D with strict control over the GC and/or allocation in general. If std functions have no reason to allocate, why should they? There's also the question of priorities. Would you rather than effort is spent on optimizing std.process (and dealing with all the fallout from any such optimizations), or working on something that is acutely missing and hurting D? If it's somehow hard to put a string on the stack, then there may be a hole in phobos. I'm not suggesting changes that are somehow hard to implement, or obscure in some way... they should be utterly trivial. D is a systems programming language, there is hope that it will penetrate a wide range of systems and environments - sure in many cases a little bit of memory use or performance loss is unimportant, but for many it will be the decisive factor which makes D unusable there. This is surely an exaggeration. D does not attempt to please everyone out there who is choosing a programming language for their next project. There is no such language, nor can one exist. One has to accept that D has a number of goals, none of which are absolute, but merely point towards a certain, but not overly specific, point in the multidimensional matrix of trade-offs. D never was about achieving maximum performance in all possible cases. And I never suggested we scrap phobos and rewrite it so it maximises performance at all costs. I highlighted, and suggested trivial changes that would make a big difference and don't hurt anyone. If it were habit of phobos devs to generally consider and try and avoid unnecessary allocations (almost all of which would be approached by using the stack wherever applicable), the situation would be much better in general. End-users can write D code however they want, but phobos should strive to be usable in as many types of software as possible, otherwise what good is a standard library?
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 21:39, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 11:01, Manu turkey...@gmail.com wrote: On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen. Booked! Hotel is here: http://goo.gl/maps/vNGCt I'll be checking in on the 27th April until 4th May. Can hook up when you land. :) Awesome! So you decided to arrive early? :) I'll head back on the morning of the 4th I expect, I have a few friends around the SF area who I might try and catch up with while I'm there. Is that hotel near Facebook? I thought they were 20 k's south or something..
Re: Allocatoin policy in Phobos - Was: Vote for std.process
I think you are right that phobos should avoid unnecessary allocations, if it does not affect maintainability/readability of code. I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. If your pull gets accepted, std.process could be referenced from a wiki page how to minimize GC usage for phobos contributions or something. Best regards, Robert
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm?
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 11:37:14 UTC, Regan Heath wrote: The initial point was a vague one, not a specific one. Manu wasn't attempting to block std.process, he had a general concern which I share. OK, but so far my interpretation and replies were mostly in the context of std.process - this module being an example where performance improvements would have a very small real-life benefit. I agree that (generally speaking) improving the performance of the code in std.algorithm/array/range would be worth the effort and complexity. It very much matters *who* that 1 user is. And, the count may be higher, and we might never hear from these people as they find other solutions. We're lucky that some people who try D and have issues tell us about them, they may be 5% of the total for all we know. The same applies to the other side of the argument. A buggy standard library probably leaves a worse impression than a slow standard library... In reality the suggested improvements would add only very minor complexity and prevent none of the current crop of contributors from working with/on std.process. Well, how do you qualify the amount of optimization that is appropriate? For example, the code in std.process would be even faster, if it was completely written in assembler. I hope we'll agree than in practice, this would be absurd. Now, what set of well-defined arguments would conclude that rewriting it in assembler is pointless, but optimizing memory allocations is not? All three versions of std.process would perform as well as far as the end-user can perceive. Yes, as well as the users of their applications. True, none of them will even realise they could have been less happy, so none of them will realise the effort that went into it, but all of them will be better off. Absolutely - if you ignore the costs. 100%-correct faster code is always better than 100%-correct slower code, but the costs are the counter-argument. Add the missing items, without a doubt - which is why no-one is suggesting blocking std.process over this issue. Blocking is one thing, but asking for faster code where it doesn't really matter - when there are areas where D could be improved at much higher gain per effort - is another. D is a systems programming language, there is hope that it Why? There exist platforms and environments where memory and performance are concerns, if the D standard library code is not careful in it's use of both then it will be less suitable than C (for example) and so D will not penetrate those platforms. OK, but once again - how does that line up with the purpose of std.process? I can see how std.algorithm can be useful in low-spec embedded/gaming systems, but std.process? Manu is using D for games development on modern high-end gaming PCs and he is still concerned with memory and performance. In Manu's case, every bit of performance counts in the code that runs in tight loops, e.g. for every game frame. However, does that include std.process? All true, but performance is one of D's top draw cards: quoteThe D programming language. Modern convenience. Modeling power. Native **efficiency**./quote (**emphasis mine**) So, it behoves us to make sure the standard library keeps that in mind. Again, I don't (generally) disagree for the general case, however I think it pays to mind the context and perspective. When the context is std.process and the perspective is the relative cost of process creation, it seems like quite a pointless argument.
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On Friday, 12 April 2013 at 13:09:19 UTC, Lars T. Kyllingstad wrote: On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm? Or probably druntime. This rise again and again - until standard global allocators are not here it is all a premature and useless optimization. It makes no sense to try to workaround a fundamental issue. That will be a lot of effort for no real gain. Allocators are a blocker for almost any memory-related issue in both druntime and phobos.
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 07:04:23 UTC, Manu wrote:
string[string] is used in the main API to receive environment
variables;
perhaps kinda convenient, but it's impossible to supply
environment
variables with loads of allocations.
Environment variables are a mapping of strings to strings. The
natural way to express such a mapping in D is with a
string[string]. It shouldn't be necessary to allocate an AA
literal, though.
toStringz is used liberally; alternatively, alloca() could
allocate the
c-string's on the stack and zero terminate them there, passing
a pointer to
the stack string to the OS functions.
It is kind of hard to use alloca() in a safe manner in D, because
DMD will happily inline functions that use it. The following
program will overflow the stack if compiled with -inline:
void doStuff()
{
auto p = alloca(100);
}
void main()
{
foreach (i; 0 .. 1_000_000) doStuff();
}
This is of course fixable, but until that happens, I would
consider alloca() a no-go for Phobos.
Re: Was: Re: Vote for std.process
On 12 April 2013 22:30, Lars T. Kyllingstad pub...@kyllingen.net wrote:
On Friday, 12 April 2013 at 11:37:14 UTC, Regan Heath wrote:
I've moved this to another thread to allay complaints.
Thanks!
I completely agree that if code can be made more performant without a
significant increase in complexity, then we should do so. While it is
mostly (but not entirely) irrelevant in the context of std.process, it is a
problem that should be tackled in Phobos as a whole. Several things
could/should be done:
It would be nice to have some sugar on top of alloca(), the use of which
is usually considered bad practice. Someone (bearophile?) once suggested
static arrays whose length is determined at runtime, which would be a great
addition to the language:
void foo(int n)
{
int[n] myArr;
...
}
That's beautiful!
Furthermore, we need to get the allocator design in place. In SciD, I use
David Simcha's region allocator to allocate temporary workspace, and it
works really well. The only times I use 'new' is when I need persistent
memory (e.g. for a return value) and the user-supplied buffer is too small.
Phobos would greatly benefit from doing this too.
Finally, an example from the new std.process which got some heavy
criticism in the other thread:
envz[pos++] = (var~'='~val~'\0').ptr;
I have been operating under the assumption that the compiler is smart
enough to make the above a single allocation. If it isn't, I would
consider it a compiler issue.
Does it? I've not seen the compiler do that, although I'd like to think it
should be possible. 1 allocation is better than 3 I guess, however, I
wonder if that code could be restructured to use the stack aswell.
alloca() is really underrated! I can't imagine why people don't like it.
Perhaps some more helpers built around it might encourage its use? It does
feel a little bit 'raw', like malloc(). It implies some annoying casts.
Re: Allocatoin policy in Phobos - Was: Vote for std.process
12-Apr-2013 17:20, Dicebot пишет: On Friday, 12 April 2013 at 13:09:19 UTC, Lars T. Kyllingstad wrote: On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm? Or probably druntime. This rise again and again - until standard global allocators are not here it is all a premature and useless optimization. It makes no sense to try to workaround a fundamental issue. That will be a lot of effort for no real gain. Allocators are a blocker for almost any memory-related issue in both druntime and phobos. Same here - trying to get out of your way to avoid GC right now is both painful and useless in the long run. More then that it's limited as you can't have the default allocation scheme that fits all nor use some scheme that would require change of API (so that later it's broken again once allocators come). -- Dmitry Olshansky
Re: A little game, Phobos and more
Impressive list, when I saw the code, my first impression looks good, thanks for opening my eyes!
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On Friday, 12 April 2013 at 12:52:39 UTC, Manu wrote: If it's somehow hard to put a string on the stack, then there may be a hole in phobos. I'm not suggesting changes that are somehow hard to implement, or obscure in some way... they should be utterly trivial. Well, ironically or not, it is not something utterly trivial. The main issue is that the stack can't hold a lot of data. This is not a problem with the heap, which is limited by the amount of memory and address space; these (usually abundant) limits are usually the user's concern, not the programmer's. Did you know that Linux does not impose a limit on the size of the environment? The default stack size seems to be 8MB... Now, what would happen if on certain machines that, for one reason or another, have an environment larger than that, and std.process did not account for it? So, to perform the task correctly, std.process would need to perform most allocations on the stack if they are up to a certain size, and on the heap otherwise. What would be a good limit for stack allocations? You may want to choose a value based on whatever's the default stack size on today's Linux versions (after all, std.process is near the leaf parts of call stacks). However, certain applications create a lot of stacks, for example for use in lightweight threads (fibers). When restricted by a small address space (32-bit architecture), the stacks need to be much smaller than usual... I highlighted, and suggested trivial changes that would make a big difference and don't hurt anyone. Well, why do you think they would make a big difference in std.process? I don't think any of the Phobos developers are against improving performance when the cost is low. So, it's not that I think you're wrong in general, but that the std.process scapegoat (for lack of better word) was not the best choice. I suggest that you file enhancement requests on Bugzilla for each specific component of Phobos / Druntime, improving the allocation behavior of which would result in a real-world benefit for you.
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 13:58, Manu turkey...@gmail.com wrote: On 12 April 2013 21:39, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 11:01, Manu turkey...@gmail.com wrote: On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen. Booked! Hotel is here: http://goo.gl/maps/vNGCt I'll be checking in on the 27th April until 4th May. Can hook up when you land. :) Awesome! So you decided to arrive early? :) I'll head back on the morning of the 4th I expect, I have a few friends around the SF area who I might try and catch up with while I'm there. Is that hotel near Facebook? I thought they were 20 k's south or something.. I decided that +£600 extra just for the convenience of being round the corner is just not worth it. My pockets don't stretch that far I'm afraid. :( Besides, tt's only a 14 minute drive away rather than a 2 minute... :) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Was: Re: Vote for std.process
On 12 April 2013 23:08, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 11:37:14 UTC, Regan Heath wrote: It very much matters *who* that 1 user is. And, the count may be higher, and we might never hear from these people as they find other solutions. We're lucky that some people who try D and have issues tell us about them, they may be 5% of the total for all we know. The same applies to the other side of the argument. A buggy standard library probably leaves a worse impression than a slow standard library... If allocating a string on the stack makes it buggy, then there is something really wrong. It should be no less convenient if appropriate helpers are available. With consideration to the string[string] argument, surely instances like that can be reconsidered? How is string[] going to produce more bugs than string[string]? You're being paranoid, or sensationalising the effect of simple optimisation. In reality the suggested improvements would add only very minor complexity and prevent none of the current crop of contributors from working with/on std.process. Well, how do you qualify the amount of optimization that is appropriate? As much is convenient without causing you to start obscuring your code? That's my personal rule. But I make it a habit to consider efficiency when designing code, I never retrofit it. I tend to choose designs that are both simple and efficient at the start. For example, the code in std.process would be even faster, if it was completely written in assembler. I hope we'll agree than in practice, this would be absurd. Now, what set of well-defined arguments would conclude that rewriting it in assembler is pointless, but optimizing memory allocations is not? All three versions of std.process would perform as well as far as the end-user can perceive. Actually, it would probably be slower if hand-written in assembler. And again, speed is not my concern here, it's inconsiderate the allocation policy. Yes, as well as the users of their applications. True, none of them will even realise they could have been less happy, so none of them will realise the effort that went into it, but all of them will be better off. Absolutely - if you ignore the costs. 100%-correct faster code is always better than 100%-correct slower code, but the costs are the counter-argument. Can you describe the 'costs'? Add the missing items, without a doubt - which is why no-one is suggesting blocking std.process over this issue. Blocking is one thing, but asking for faster code where it doesn't really matter - when there are areas where D could be improved at much higher gain per effort - is another. I'm asking for code that doesn't needlessly allocate, as a policy/habit in phobos. D is a systems programming language, there is hope that it Why? There exist platforms and environments where memory and performance are concerns, if the D standard library code is not careful in it's use of both then it will be less suitable than C (for example) and so D will not penetrate those platforms. OK, but once again - how does that line up with the purpose of std.process? I can see how std.algorithm can be useful in low-spec embedded/gaming systems, but std.process? Manu is using D for games development on modern high-end gaming PCs and he is still concerned with memory and performance. In Manu's case, every bit of performance counts in the code that runs in tight loops, e.g. for every game frame. However, does that include std.process? I'm interested in eliminating allocations. It's just another function that can't be called in a no-gc area. If it used the stack for its temporaries, no problem. All true, but performance is one of D's top draw cards: quoteThe D programming language. Modern convenience. Modeling power. Native **efficiency**./quote (**emphasis mine**) So, it behoves us to make sure the standard library keeps that in mind. Again, I don't (generally) disagree for the general case, however I think it pays to mind the context and perspective. When the context is std.process and the perspective is the relative cost of process creation, it seems like quite a pointless argument. It was the first module that appeared for consideration since the recent discussions about irresponsible GC usage. The argument applies to everything considered for acceptance into phobos. I'd like to see it applied as a systematic consideration in the future, irrespective of the module being considered. Avoiding allocation for temporaries shouldn't be hard, if some tools are missing, then that is something that needs further discussion I guess.
Re: DConf 2013 official car/room sharing thread
On 12 April 2013 13:58, Manu turkey...@gmail.com wrote: On 12 April 2013 21:39, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 11:01, Manu turkey...@gmail.com wrote: On 12 April 2013 19:43, Iain Buclaw ibuc...@ubuntu.com wrote: On 12 April 2013 07:06, Manu turkey...@gmail.com wrote: On 12 April 2013 15:59, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 10:52 PM, Manu wrote: On 12 April 2013 15:49, Ali Çehreli acehr...@yahoo.com wrote: On 04/11/2013 08:43 PM, Manu wrote: It's an easy bart ride from SFO - San Jose That is planned for the future. :) BART doesn't go to San Jose. Oh right, well it's some other train then? Oh, of course! I am slow today. :) You meant CalTrain. Pretty good train indeed. That's the one! That one... Anyway yeah, I'll be at SFO midday 29th, happy to connect with anyone in the area/driving past. I still have no plans for accommodation, nor a lot of money, if people are looking to split a room. I'm about to book. Can get a hotel with two beds if yer like! Walter mentioned that they'll cover the hotel, but no reason not to save some money all the same. So yeah, I'm happy to share a room if you're keen. Booked! Hotel is here: http://goo.gl/maps/vNGCt I'll be checking in on the 27th April until 4th May. Can hook up when you land. :) Awesome! So you decided to arrive early? :) Wrangled flight times and costs to suit my needs. Will be landing at around 9pm on the 27th, so will quite literally be a check-in + bed. :) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Was: Re: Vote for std.process
On 12 April 2013 23:21, Lars T. Kyllingstad pub...@kyllingen.net wrote:
On Friday, 12 April 2013 at 07:04:23 UTC, Manu wrote:
string[string] is used in the main API to receive environment variables;
perhaps kinda convenient, but it's impossible to supply environment
variables with loads of allocations.
Environment variables are a mapping of strings to strings. The natural
way to express such a mapping in D is with a string[string]. It shouldn't
be necessary to allocate an AA literal, though.
That's a good point, do AA's support literals that don't allocate? You
can't even produce an array literal without it needlessly allocating.
toStringz is used liberally; alternatively, alloca() could allocate the
c-string's on the stack and zero terminate them there, passing a pointer
to
the stack string to the OS functions.
It is kind of hard to use alloca() in a safe manner in D, because DMD will
happily inline functions that use it. The following program will overflow
the stack if compiled with -inline:
void doStuff()
{
auto p = alloca(100);
}
void main()
{
foreach (i; 0 .. 1_000_000) doStuff();
}
This is of course fixable, but until that happens, I would consider
alloca() a no-go for Phobos.
Very good point. This is a problem.
Hmmm...
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On 12 April 2013 23:09, Lars T. Kyllingstad pub...@kyllingen.net wrote: On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm? I think I'll have a go at this after dconf. For now, I need to stop distracting myself from my lecture preparation! ;)
Re: Vote for std.process
This is good feedback and all, but it would be good to have had this placed in the official review thread. But it is too late now and I would not want to break the discussion up.
Re: Vote for std.process
On Friday, 12 April 2013 at 06:25:10 UTC, Manu wrote: I see this pattern where something is designed, discussed, and then voted into phobos. At this time the design looks good on paper, but there is very little practical experience using the library. The problem then is, once accepted, people start using it, and at some point some issues are found, or ideas for improvement are made based on user experience, but the module can no longer be touched due to the general phobia of making breaking changes... I think this needs to happen prior to the formal review/voting. I would say it should be a precursor to starting the official review, however this would raise the bar too high for things like Jacob's Serialization library; he has a working library, but it isn't ready for Phobos and it would be silly to require the translation prior to approving it for Phobos. How we choose to add to the exp module would need some consideration.
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On Fri, 12 Apr 2013 14:25:22 +0100, Dmitry Olshansky dmitry.o...@gmail.com wrote: 12-Apr-2013 17:20, Dicebot пишет: On Friday, 12 April 2013 at 13:09:19 UTC, Lars T. Kyllingstad wrote: On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm? Or probably druntime. This rise again and again - until standard global allocators are not here it is all a premature and useless optimization. It makes no sense to try to workaround a fundamental issue. That will be a lot of effort for no real gain. Allocators are a blocker for almost any memory-related issue in both druntime and phobos. Same here - trying to get out of your way to avoid GC right now is both painful and useless in the long run. More then that it's limited as you can't have the default allocation scheme that fits all nor use some scheme that would require change of API (so that later it's broken again once allocators come). My impression is that there is some low hanging fruit here. My impression (pls correct me if I'm wrong) is that Manu is not trying to avoid the GC, but to avoid allocations (and presumably collections) at the wrong moment in time. So... If the GC were to have a hook function for allocation and for free, and if when these were in use it would not itself trigger collection. Then... Manu could supply hook functions, use the alloc function to supply pre-allocated memory, and trigger collection as/when convenient. For collection itself some upper bound time would also be desireable. These ideas are nowhere near as complex as a full blown allocator proposal/idea. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On 12 April 2013 23:32, Vladimir Panteleev vladi...@thecybershadow.netwrote: On Friday, 12 April 2013 at 12:52:39 UTC, Manu wrote: If it's somehow hard to put a string on the stack, then there may be a hole in phobos. I'm not suggesting changes that are somehow hard to implement, or obscure in some way... they should be utterly trivial. Well, ironically or not, it is not something utterly trivial. The main issue is that the stack can't hold a lot of data. This is not a problem with the heap, which is limited by the amount of memory and address space; these (usually abundant) limits are usually the user's concern, not the programmer's. Did you know that Linux does not impose a limit on the size of the environment? The default stack size seems to be 8MB... Now, what would happen if on certain machines that, for one reason or another, have an environment larger than that, and std.process did not account for it? So, to perform the task correctly, std.process would need to perform most allocations on the stack if they are up to a certain size, and on the heap otherwise. What would be a good limit for stack allocations? You may want to choose a value based on whatever's the default stack size on today's Linux versions (after all, std.process is near the leaf parts of call stacks). However, certain applications create a lot of stacks, for example for use in lightweight threads (fibers). When restricted by a small address space (32-bit architecture), the stacks need to be much smaller than usual... Yes, you're right, there's an 'if' required here to catch unreasonably large environment blocks, but I still consider that within the realm of 'trivial'. This is processed in an appending loop, just check the next bit fits, and if it overflows 1kb or so of stack string, revert to the heap and continue. I reckon helpers could be written to assist with common cases of this (which would have to be mixin template based I guess?)... And I really like the variable-length static array idea! I highlighted, and suggested trivial changes that would make a big difference and don't hurt anyone. Well, why do you think they would make a big difference in std.process? I don't think any of the Phobos developers are against improving performance when the cost is low. So, it's not that I think you're wrong in general, but that the std.process scapegoat (for lack of better word) was not the best choice. Fuck, I've repeated myself so many times now. The point I make is a general issue I have with phobos, I consider it an issue that should be made policy (irrespective of module being considered), and std.process came into question right at the moment I thought to make the point. It may not be the strongest case for the principle, it's just the one that appeared. I suggest that you file enhancement requests on Bugzilla for each specific component of Phobos / Druntime, improving the allocation behavior of which would result in a real-world benefit for you. I'll start doing it myself, but I also suggest it be made a policy, and carefully considered when considering acceptance of ANY new module. That way, new code that suffers the unpredictable/surprise! allocation problems won't be introduced.
Re: Allocatoin policy in Phobos - Was: Vote for std.process
On 13 April 2013 00:01, Regan Heath re...@netmail.co.nz wrote: On Fri, 12 Apr 2013 14:25:22 +0100, Dmitry Olshansky dmitry.o...@gmail.com wrote: 12-Apr-2013 17:20, Dicebot пишет: On Friday, 12 April 2013 at 13:09:19 UTC, Lars T. Kyllingstad wrote: On Friday, 12 April 2013 at 13:00:36 UTC, Robert wrote: I would suggest, instead of writing your own std.process, that you simply provide a pull request to the now to be approved one, this way you can prove your point that it is easy by already improving a part of the standard library. Actually, Manu (or anyone) could do this for *any* part of Phobos. And considering that std.process is probably the module that would benefit the least from it, maybe he should start somewhere else? std.string, perhaps, or std.algorithm? Or probably druntime. This rise again and again - until standard global allocators are not here it is all a premature and useless optimization. It makes no sense to try to workaround a fundamental issue. That will be a lot of effort for no real gain. Allocators are a blocker for almost any memory-related issue in both druntime and phobos. Same here - trying to get out of your way to avoid GC right now is both painful and useless in the long run. More then that it's limited as you can't have the default allocation scheme that fits all nor use some scheme that would require change of API (so that later it's broken again once allocators come). My impression is that there is some low hanging fruit here. My impression (pls correct me if I'm wrong) is that Manu is not trying to avoid the GC, but to avoid allocations (and presumably collections) at the wrong moment in time. So... If the GC were to have a hook function for allocation and for free, and if when these were in use it would not itself trigger collection. Then... Manu could supply hook functions, use the alloc function to supply pre-allocated memory, and trigger collection as/when convenient. Believe it or not, I'm not actually a fan of over-complexity. And I'm focusing here on totally unnecessary allocations. Isn't using the stack where applicable just a whole low easier? That's what it's there for. There are of course plenty of functions that do allocate by design, and in many cases, allocators would assist with those. But I'm not arguing for that right now, since allocators still seem to be a way off. For collection itself some upper bound time would also be desireable. Absolutely. Is it possible for collection to run like a coroutine? Can it run a fragmented collection process, each time picking up where it left off? These ideas are nowhere near as complex as a full blown allocator proposal/idea.
Re: Allocatoin policy in Phobos - Was: Vote for std.process
12-Apr-2013 18:12, Manu пишет: On 13 April 2013 00:01, Regan Heath re...@netmail.co.nz mailto:re...@netmail.co.nz wrote: So... If the GC were to have a hook function for allocation and for free, and if when these were in use it would not itself trigger collection. Then... Manu could supply hook functions, use the alloc function to supply pre-allocated memory, and trigger collection as/when convenient. Believe it or not, I'm not actually a fan of over-complexity. And I'm focusing here on totally unnecessary allocations. Isn't using the stack where applicable just a whole low easier? That's what it's there for. 'cause nobody can tell you how big it is. This knowledge is only available to end user and there is still no easy way to tell that to the library. The end result is utterly useless as library can't reliably use stack space. In the end if one library thinks it's fine to burn say 32K of stack with alloca but then it calls into another one that burns another chunk with alloca and so on untill happily stack overflowing (sometimes, on some systems at the right time!). Call graphs to calculate this is only available for the final app. Maybe just adding a separate thread-local growable stack for data would work - at least it wouldn't depend on sheer luck and particular OS settings. -- Dmitry Olshansky
Re: Was: Re: Vote for std.process
On Friday, 12 April 2013 at 13:39:38 UTC, Manu wrote: If allocating a string on the stack makes it buggy, then there is something really wrong. It should be no less convenient if appropriate helpers are available. Please see my reply to your other post. With consideration to the string[string] argument, surely instances like that can be reconsidered? How is string[] going to produce more bugs than string[string]? env ~= FOO=BAR; This will probably not do what you want if there was already a line starting with FOO= in env. An array of strings is a less direct representation of the environment than a string map. Certain common operations, such as finding the value of a variable, or setting / overwriting a variable, become more difficult. You're being paranoid, or sensationalising the effect of simple optimisation. Strong words... And again, speed is not my concern here, it's inconsiderate the allocation policy. I'm interested in eliminating allocations. It's just another function that can't be called in a no-gc area. If it used the stack for its temporaries, no problem. Why allocations, specifically, if not for the performance costs of allocation and garbage collection? Can you describe the 'costs'? See my previous posts in today's discussions. As much is convenient without causing you to start obscuring your code? That's my personal rule. But I make it a habit to consider efficiency when designing code, I never retrofit it. I tend to choose designs that are both simple and efficient at the start. OK, so if I understand you correctly: you would like Phobos to adopt a policy of avoiding heap allocations whenever possible, and this argument applies to std.process not because doing so would result in a tangible improvement of its performance or other metric, but for the purpose of being consistent across Phobos. Assuming that the language can provide or allow implementing suitably safe abstractions for doing so without complicating the code much, I think that's a goal worth looking forward, and we have been doing so for some time (hence the pending allocator design).
