Re: The CAPI Manifesto
Walter Bright , dans le message (digitalmars.D:146786), a écrit :
If you want to add a layer on top of the C API, that would be fine. std.zlib
is
an example of that.
But the idea of CAPI is NOT to add a layer. Not fix, extend, refactor,
improve, etc.
I definitely agree with that, no attempt should be made to fix anything.
However, if you want only direct translation, the only way I see is to
treat all defines as mixins. That mean all macro will become very
tedious to use. Even there, there are choices to make (how do you
translate multi-parameter macros ?). So I think a minimum of adaptation
should be provided.
Here is an example of how this could be made:
#define square(x) ((x)*(x))
=
// direct translation
mixin template!(string x)square
{
enum square = '((' ~ x ~ ')*(' ~ x ~ '))';
}
// adapted version
T square(T)(T x) { return x*x; }
Which version should be added ?
Both do not do the same thing ! As you know, In the first one, if x is a
function call, it is called twice, and it the other one, it is called
only once.
So if I follow the rule: no fix, no extend, etc, I must include only the
direct translation. But the CAPI library will not be as usable as the c
one. Then I miss the purpose of CAPI: make d as usable as c when using
c libraries.
So which version should be added: I think it is both:
// direct translation
mixin template!(string x)M_square
{
enum M_square = '((' ~ x ~ ')*(' ~ x ~ '))';
}
// adapted version
T square(T)(T x) { return mixin M_square!q{x}; }
So this is what I propose:
Direct translation have to be provided. Adapted version can be provided
alongside the direct translation of the d header. Adaptaded version must
be a direct forward call to the direct version (note here that the
adapted. Rules will have to be defined to avoid name clashes (for
example, here, I used a the direct name for the usable version, and M_
prefix for the mixin version, but we could decide other rules).
Macros are a big issue.
And I think abvious translating, such as const char* to string, or
pointer-length pairs to dynamic arrays is about the same problem.
double array_sum(double* a, size_t length);
=
double array_sum(double* a, size_t length);
// AND
double array_sum(double[] a) { return array_sum(a.ptr, a.length); }
That is very little work. The direct translation is mandatory, and the
adapted translation is not. But when the translation is obvious, there
is no reason for everyone to make it on it's corner. Make it in the
public header and share it!
I order to remain consistent, adaptation will have to obey to very
precise rules, that have to be set. No fix, no extend, no refactor, no
improve, etc. Just a forward call, to have, in addition to the C API,
an API that use D's power: using enums, inline functions, for defines
instead of the direct mixin translation. Using D's arrays instead of C
arrays, etc. could be nice too.
What translation should be provided ? What rules to translate defines ?
Is translation of pointer+length pair to array worth doing ? What about
stringz and strings ? Where to draw the line ?
--
Christophe
Re: The CAPI Manifesto
Sure. You're right. Weird macros are often used for handy shortcuts (like the aforementioned for loop header), which are not handy enough to invent a just-as-weird way of translating them. Everything else is either for compiler extensions (which either have built-in support in D or are simply nit translatable) or hacks for compile-time activity (can be replaced by more intuitive counterparts in D). Translating _any_ C code is impossible. The C preprocessor cannot be fully simulated in D. But translating _this_ C code is not as hard, because it's a bad practice to abuse macros in C and no good C library would do that. The translator should not be a _any_ C code translator, it should be an arbitrary _this_ C code translator, which makes a few assumptions and does not guarantee 100% convertibility of the code. The most important part of the translator should be the problem reporter, which should point you to all the suspicious and untranslated parts of the C code, so you can deal with them manually. On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise marco.le...@gmx.de wrote: Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan gor.f.gyolchan...@gmail.com: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif Maybe these cases can often be solved when, as you say, they are for compatibility with other compilers. The header converter would simply assume to be the latest and greatest of the known compilers and evaluate the code like this: - there is an #ifdef - if it a define from the list of known C compilers with quirks - jump right to the else block The #define obviously can become more than one thing in the .d file. But that is determined by the instantiation site. The above CONST would probably be ignored when it is used in a const parameter declaration, because of D's transitivity and become 'immutable __gshared' when used on a global variable. Do you get the idea?
Re: The CAPI Manifesto
In the end, I'm sure, that manual intervention won't be necessary and the CAPI repository will be replaced by std.c.translate or something like that. You'd just do this: mixin cInclude(sqlite3.h); and voila. The mixin will print out all macros, which i couldn't translate and had to evaluate in-line. If some of those macros are important to the user, then translating a C header would consist of including the header and adding the missing macros. I think this is a much better idea, then manually translating everything, because this is cheaper, covers more libraries. On Tue, Oct 25, 2011 at 12:13 PM, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: Sure. You're right. Weird macros are often used for handy shortcuts (like the aforementioned for loop header), which are not handy enough to invent a just-as-weird way of translating them. Everything else is either for compiler extensions (which either have built-in support in D or are simply nit translatable) or hacks for compile-time activity (can be replaced by more intuitive counterparts in D). Translating _any_ C code is impossible. The C preprocessor cannot be fully simulated in D. But translating _this_ C code is not as hard, because it's a bad practice to abuse macros in C and no good C library would do that. The translator should not be a _any_ C code translator, it should be an arbitrary _this_ C code translator, which makes a few assumptions and does not guarantee 100% convertibility of the code. The most important part of the translator should be the problem reporter, which should point you to all the suspicious and untranslated parts of the C code, so you can deal with them manually. On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise marco.le...@gmx.de wrote: Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan gor.f.gyolchan...@gmail.com: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif Maybe these cases can often be solved when, as you say, they are for compatibility with other compilers. The header converter would simply assume to be the latest and greatest of the known compilers and evaluate the code like this: - there is an #ifdef - if it a define from the list of known C compilers with quirks - jump right to the else block The #define obviously can become more than one thing in the .d file. But that is determined by the instantiation site. The above CONST would probably be ignored when it is used in a const parameter declaration, because of D's transitivity and become 'immutable __gshared' when used on a global variable. Do you get the idea?
Re: The CAPI Manifesto
The CAPI repository could be replaced by a package in Phobos with aforementioned tiny modules with a single cImport and some additions. They would be very small and conveniently available from Phobos. On Tue, Oct 25, 2011 at 12:26 PM, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: In the end, I'm sure, that manual intervention won't be necessary and the CAPI repository will be replaced by std.c.translate or something like that. You'd just do this: mixin cInclude(sqlite3.h); and voila. The mixin will print out all macros, which i couldn't translate and had to evaluate in-line. If some of those macros are important to the user, then translating a C header would consist of including the header and adding the missing macros. I think this is a much better idea, then manually translating everything, because this is cheaper, covers more libraries. On Tue, Oct 25, 2011 at 12:13 PM, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: Sure. You're right. Weird macros are often used for handy shortcuts (like the aforementioned for loop header), which are not handy enough to invent a just-as-weird way of translating them. Everything else is either for compiler extensions (which either have built-in support in D or are simply nit translatable) or hacks for compile-time activity (can be replaced by more intuitive counterparts in D). Translating _any_ C code is impossible. The C preprocessor cannot be fully simulated in D. But translating _this_ C code is not as hard, because it's a bad practice to abuse macros in C and no good C library would do that. The translator should not be a _any_ C code translator, it should be an arbitrary _this_ C code translator, which makes a few assumptions and does not guarantee 100% convertibility of the code. The most important part of the translator should be the problem reporter, which should point you to all the suspicious and untranslated parts of the C code, so you can deal with them manually. On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise marco.le...@gmx.de wrote: Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan gor.f.gyolchan...@gmail.com: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif Maybe these cases can often be solved when, as you say, they are for compatibility with other compilers. The header converter would simply assume to be the latest and greatest of the known compilers and evaluate the code like this: - there is an #ifdef - if it a define from the list of known C compilers with quirks - jump right to the else block The #define obviously can become more than one thing in the .d file. But that is determined by the instantiation site. The above CONST would probably be ignored when it is used in a const parameter declaration, because of D's transitivity and become 'immutable __gshared' when used on a global variable. Do you get the idea?
Re: The CAPI Manifesto
On Sat, 22 Oct 2011 04:34:59 +0200, Walter Bright newshou...@digitalmars.com wrote: On 10/21/2011 12:41 AM, so wrote: You are right, i forgot about macros, Is it only this or is there anything else? The only other thing is what does one do about 'char' - make it a byte, ubyte, or char D type? One should not loose the implicit conversion of string literals to const char pointers. So the answer might be to use char where something is used in a string context and (u)byte where it's not. In most cases this is a no-brainer. martin
Re: The CAPI Manifesto
Am 22.10.2011, 04:33 Uhr, schrieb Walter Bright newshou...@digitalmars.com: On 10/21/2011 4:32 PM, Fawzi Mohamed wrote: On Oct 21, 2011, at 4:20 PM, Fawzi Mohamed wrote: The main problem with this approach is how to support different versions of a library, or of OS. It quickly becomes difficult to support anything but the latest, or a fixed version. It works beautifully for mature libs. Since github has excellent support for branches, I don't see why this is a major problem. I still cannot avoid thinking that a C frontend automatically generating D modules with the help of recipes would be a better way. It will need some manual intervention for difficult cases, mainly giving manual translation of some macros, but it should be small. … and it seems that in the time I was offline others came up with the same idea... It's an old idea. The trouble is, as always, the C preprocessor. I'm currently converting the openssl .h files, and they are a zoo of metaprogramming using C preprocessor macros. People are going to demand perfect translation if it is automatic. The only way to do it is to work with the preprocessed output of the .h file, and just forget about the preprocessor. Doesn't that mean no #ifdef WINDOWS ?
Re: The CAPI Manifesto
Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan gor.f.gyolchan...@gmail.com: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif Maybe these cases can often be solved when, as you say, they are for compatibility with other compilers. The header converter would simply assume to be the latest and greatest of the known compilers and evaluate the code like this: - there is an #ifdef - if it a define from the list of known C compilers with quirks - jump right to the else block The #define obviously can become more than one thing in the .d file. But that is determined by the instantiation site. The above CONST would probably be ignored when it is used in a const parameter declaration, because of D's transitivity and become 'immutable __gshared' when used on a global variable. Do you get the idea?
Re: The CAPI Manifesto
On 2011-10-22 21:12, Andrej Mitrovic wrote: On 10/22/11, Jacob Carlborgd...@me.com wrote: On 2011-10-22 15:41, Andrej Mitrovic wrote: I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem? You still need to link to the C library. -- /Jacob Carlborg Yes, but *linking* isn't extending the library. Importing D files that are LGPL licensed could be different. According to GPL it is and according to LGPL it is for static linking, IIRC. -- /Jacob Carlborg
Re: The CAPI Manifesto
On 21.10.2011 13:42, Gor Gyolchanyan wrote: This will defeat the philosophy of D, which stands for core correctness, simplicity, maintainability and flexibility. A much better solution would be to implement the AST macros, which were discussed in a video-talk a long time ago by Walter and Andrei, for which the macro keyword was reserved. After that, all C macros will be translatable. Unfortunately, the AST macros described in the conference video don't work (they are *far* too weak). Nobody has ever come up with a replacement proposal.
Re: The CAPI Manifesto
On Oct 22, 2011, at 4:03 AM, Timon Gehr wrote: On 10/22/2011 04:33 AM, Walter Bright wrote: On 10/21/2011 4:32 PM, Fawzi Mohamed wrote: On Oct 21, 2011, at 4:20 PM, Fawzi Mohamed wrote: The main problem with this approach is how to support different versions of a library, or of OS. It quickly becomes difficult to support anything but the latest, or a fixed version. It works beautifully for mature libs. Since github has excellent support for branches, I don't see why this is a major problem. do you have a repo per library? if yes then indeed it is feasible. I didn't think about that. I still cannot avoid thinking that a C frontend automatically generating D modules with the help of recipes would be a better way. It will need some manual intervention for difficult cases, mainly giving manual translation of some macros, but it should be small. … and it seems that in the time I was offline others came up with the same idea... It's an old idea. The trouble is, as always, the C preprocessor. I'm currently converting the openssl .h files, and they are a zoo of metaprogramming using C preprocessor macros. People are going to demand perfect translation if it is automatic. that was the reason I talked about recipes that can add manual fixes where needed (for selected macros). The only way to do it is to work with the preprocessed output of the .h file, and just forget about the preprocessor. Another way is to replace the preprocessor with CTFE and string mixins. I think that could be automated quite easily. (modulo the possibility of some extremely heavy abuse on the C side that could make the other parts of the translation a lot harder of course) I think string mixings are an extremely ugly solution to this problem, and I would try to avoid that, especially if they are used to represent a function that should be inlined, and might be replaced by a normal function in a later version.
Re: The CAPI Manifesto
On 10/22/2011 6:12 PM, Fawzi Mohamed wrote: Another way is to replace the preprocessor with CTFE and string mixins. I think that could be automated quite easily. (modulo the possibility of some extremely heavy abuse on the C side that could make the other parts of the translation a lot harder of course) I think string mixings are an extremely ugly solution to this problem, and I would try to avoid that, especially if they are used to represent a function that should be inlined, and might be replaced by a normal function in a later version. Sure, but I've seen some pretty ugly use of preprocessor macros to paste together statements that need to be inserted. String mixins are the only hope.
Re: The CAPI Manifesto
On 10/22/2011 01:20 AM, Fawzi Mohamed wrote:
The main problem with this approach is how to support different versions of a
library, or of OS. It quickly becomes difficult to support anything but the
latest, or a fixed version.
It works beautifully for mature libs.
I still cannot avoid thinking that a C frontend automatically generating D
modules with the help of recipes would be a better way.
It will need some manual intervention for difficult cases, mainly giving
manual translation of some macros, but it should be small.
One would set if all the files correspond to modules, or there are just some
main directories/files.
Some things are easy:
#define a
enum { a=true }
#define b xyz
enum { b=xyz }
one could be tempted to replace
#ifdef x
with
static if (is(typeof(x)) x)
and treat other #if in a similar way, but in D a static if must contain a full
statement, as its content must be syntactically valid, whereas the C
preprocessor does not have this limitation.
D does not have this limitation either. Use string mixins. The only
difference between C macros and D string mixins is that D is more
explicit about that the feature is mere string manipulation. There is
nothing you cannot do with string mixins that is possible with macros.
(except hijacking existing code or making macro instantiations look like
function calls for transparent interchangeability, of course).
The way to work around this, if we create the headers on demand is simple: we
already evaluate all #if using the building definitions of the associated C
compiler (gcc -E -dD for example) and its default include paths (or directly
use it, keeping in account the # line file directives).
real macros are more tricky, for example one could do
#define isSmall(x) (x2)
isSmall(T)(T x){
return x2;
}
isSmall(x); // use macro in C code
string isSmall(string x) {
return `{return ~x~`;}`;
}
mixin(isSmall(q{x}); // use macro in D.
or, with Kenji Hara's proposal:
mixin template isSmall(string x){
enum isSmall = `{return ~x~`;}`;
}
isSmall!q{x} // use macro in D code
#define c(x) { x , #x }
string c(string x){
return `{ x , q{`~x~`} }`;
}
mixin(c(q{x})); // use macro in D code
or, again, with Kenji Hara's proposal:
mixin template c(string x){
enum c = `{ x , q{`~x~`} }`;
}
c!q{x} // use macro in D code
multiple parameters would possibly be best handled like this:
mixin template ADD(string x){
enum cc = {
string p = x.split(,);
assert(p.length == 2, expected 2 parameters);
return `( `~p[0] ~ '+' ~ p[0] ~ ` )`;
}();
}
ADD!q{x,y} // use macro in D code
Re: The CAPI Manifesto
On 10/22/2011 04:33 AM, Walter Bright wrote: On 10/21/2011 4:32 PM, Fawzi Mohamed wrote: On Oct 21, 2011, at 4:20 PM, Fawzi Mohamed wrote: The main problem with this approach is how to support different versions of a library, or of OS. It quickly becomes difficult to support anything but the latest, or a fixed version. It works beautifully for mature libs. Since github has excellent support for branches, I don't see why this is a major problem. I still cannot avoid thinking that a C frontend automatically generating D modules with the help of recipes would be a better way. It will need some manual intervention for difficult cases, mainly giving manual translation of some macros, but it should be small. … and it seems that in the time I was offline others came up with the same idea... It's an old idea. The trouble is, as always, the C preprocessor. I'm currently converting the openssl .h files, and they are a zoo of metaprogramming using C preprocessor macros. People are going to demand perfect translation if it is automatic. The only way to do it is to work with the preprocessed output of the .h file, and just forget about the preprocessor. Another way is to replace the preprocessor with CTFE and string mixins. I think that could be automated quite easily. (modulo the possibility of some extremely heavy abuse on the C side that could make the other parts of the translation a lot harder of course)
Re: The CAPI Manifesto
I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem?
Re: The CAPI Manifesto
Andrej Mitrovic wrote: I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem? I'm interested in this as well. This is especially evil as we currently have to link statically to D code. A D file consisting of only C imports probably doesn't count as code (and there's actually no need to compile it), but if you have macros which were translated into functions or templates, wouldn't statically linking against this LGPL'd 'header' code require everything to be LGPL/GPL licensed? -- Johannes Pfau
Re: The CAPI Manifesto
On 2011-10-22 15:41, Andrej Mitrovic wrote: I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem? You still need to link to the C library. -- /Jacob Carlborg
Re: The CAPI Manifesto
On 10/22/11, Jacob Carlborg d...@me.com wrote: On 2011-10-22 15:41, Andrej Mitrovic wrote: I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem? You still need to link to the C library. -- /Jacob Carlborg Yes, but *linking* isn't extending the library. Importing D files that are LGPL licensed could be different.
Re: The CAPI Manifesto
On Saturday, October 22, 2011 15:41:13 Andrej Mitrovic wrote: I have a question about licensing. If you translate a C LGPL'ed header file to D, and you keep the same license, are you still allowed to use whichever license in your user code that uses the new D files? Because I don't know whether using LGPL'ed .d files falls under using the library or extending the library. For example, CairoD has translated Cairo LGPL/MPL'ed header files in equivalent Boost-licensed .d files. I know I'm asking for free lawyer advice here, but do you think this is going to be a problem? I would _think_ that it would be okay as long as it's a direct translation, since all you're doing is making it so that you can link to the libraries as you would in C, not extending their functionality in any way. But I'm not a lawyer, and I'm far from an expert on software licensing. - Jonathan M Davis
Re: The CAPI Manifesto
On Tue, 18 Oct 2011 06:01:37 +0300, Walter Bright newshou...@digitalmars.com wrote: On 10/17/2011 5:56 PM, so wrote: On Tue, 18 Oct 2011 03:52:13 +0300, Nick Sabalausky a@a.a wrote: Thats would mean that every D compiler would have to *also* be a C compiler. Indeed, but i see nothing wrong with it, like i see nothing wrong with inline asm, C never changes, While C code can be directly translated to D, the C macros are another matter. You are right, i forgot about macros, Is it only this or is there anything else?
Re: The CAPI Manifesto
On Thu, 20 Oct 2011 00:26:58 +0300, Nick Sabalausky a@a.a wrote: so s...@so.so wrote in message news:op.v3ivsvb8mpw3zg@localhost.localdomain... On Tue, 18 Oct 2011 03:32:17 +0300, Jonathan M Davis jmdavisp...@gmx.com wrote: Translating C header files to D is a pain and time consuming, and it would benefit us all to have a place to go to find common C headers translated to D so that such work doesn't have to be duplicated. With the second compiler, there won't be any need for us to do anything, like C++ we could use C libraries seamlessly. The only reason C++ is able to do that is because C++ (realistically, even if not *technically*) is a proper superset of C. D isn't. Right, but more importantly it is ABI compatible which is what D also has, this opens some doors. For most C libraries if you exclude macros, you just fill structs, and call functions.
Re: The CAPI Manifesto
That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif An example of a syntactic mixin template is this piece of code, which i never actually saw anywhere (possible only in C99 and C++): #define N_TIMES(n) for(int i = 0; i != n; ++i) The last use case is very rare. The only legitimate example i ever saw is in libjpeg, where a macro is used to define function pointers of API functions. The use case before that is mostly used for portability reasons, which is not necessary in D. Some non-standard extension encapsulating macros are almost always used in C libraries, which can be removed altogether. The translation can go on regarding the above use cases and the last two cases can be evaluated in-line, commented out and warned about for manual translation. On Fri, Oct 21, 2011 at 11:48 AM, so s...@so.so wrote: On Thu, 20 Oct 2011 00:26:58 +0300, Nick Sabalausky a@a.a wrote: so s...@so.so wrote in message news:op.v3ivsvb8mpw3zg@localhost.localdomain... On Tue, 18 Oct 2011 03:32:17 +0300, Jonathan M Davis jmdavisp...@gmx.com wrote: Translating C header files to D is a pain and time consuming, and it would benefit us all to have a place to go to find common C headers translated to D so that such work doesn't have to be duplicated. With the second compiler, there won't be any need for us to do anything, like C++ we could use C libraries seamlessly. The only reason C++ is able to do that is because C++ (realistically, even if not *technically*) is a proper superset of C. D isn't. Right, but more importantly it is ABI compatible which is what D also has, this opens some doors. For most C libraries if you exclude macros, you just fill structs, and call functions.
Re: The CAPI Manifesto
Indeed, macros is a language in itself. Then again it all boils down to type manipulation and function calls. Not sure if it is doable but a special operator like __cmacro might be an answer. #define FUN(a, b) #define DATA could be accessed like: __cmacro(FUN, a, b); __cmacro(DATA); I am pushing this because the outcome well worths all these ugly hacks. On Fri, 21 Oct 2011 11:32:32 +0300, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif An example of a syntactic mixin template is this piece of code, which i never actually saw anywhere (possible only in C99 and C++): #define N_TIMES(n) for(int i = 0; i != n; ++i) The last use case is very rare. The only legitimate example i ever saw is in libjpeg, where a macro is used to define function pointers of API functions. The use case before that is mostly used for portability reasons, which is not necessary in D. Some non-standard extension encapsulating macros are almost always used in C libraries, which can be removed altogether. The translation can go on regarding the above use cases and the last two cases can be evaluated in-line, commented out and warned about for manual translation.
Re: The CAPI Manifesto
This will defeat the philosophy of D, which stands for core correctness, simplicity, maintainability and flexibility. A much better solution would be to implement the AST macros, which were discussed in a video-talk a long time ago by Walter and Andrei, for which the macro keyword was reserved. After that, all C macros will be translatable. On Fri, Oct 21, 2011 at 2:17 PM, so s...@so.so wrote: Indeed, macros is a language in itself. Then again it all boils down to type manipulation and function calls. Not sure if it is doable but a special operator like __cmacro might be an answer. #define FUN(a, b) #define DATA could be accessed like: __cmacro(FUN, a, b); __cmacro(DATA); I am pushing this because the outcome well worths all these ugly hacks. On Fri, 21 Oct 2011 11:32:32 +0300, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif An example of a syntactic mixin template is this piece of code, which i never actually saw anywhere (possible only in C99 and C++): #define N_TIMES(n) for(int i = 0; i != n; ++i) The last use case is very rare. The only legitimate example i ever saw is in libjpeg, where a macro is used to define function pointers of API functions. The use case before that is mostly used for portability reasons, which is not necessary in D. Some non-standard extension encapsulating macros are almost always used in C libraries, which can be removed altogether. The translation can go on regarding the above use cases and the last two cases can be evaluated in-line, commented out and warned about for manual translation.
Re: The CAPI Manifesto
On 2011-10-21 10:32, Gor Gyolchanyan wrote: That's ALL you can do in C. fill structs and call functions (fundamental type manipulation doesn't count). My personal research shows the following use cases of C macros (sorted by popularity in descending order): 1. enum 2. alias (most notably, conditionally compiled ones) 3. CTFE function 4. mixin template 5. syntactic alias 6. syntactic mixin template I guess it's quite difficult for a compiler to recognize the differences between these use cases. only the last 2 out of 6 cannot be translated to D. An example of a syntactic alias is this very common piece of C code: #ifdef __VERY_VERY_OLD_C_COMPILER__ #define CONST #else #define CONST const #endif An example of a syntactic mixin template is this piece of code, which i never actually saw anywhere (possible only in C99 and C++): #define N_TIMES(n) for(int i = 0; i != n; ++i) The last use case is very rare. The only legitimate example i ever saw is in libjpeg, where a macro is used to define function pointers of API functions. The use case before that is mostly used for portability reasons, which is not necessary in D. Some non-standard extension encapsulating macros are almost always used in C libraries, which can be removed altogether. The translation can go on regarding the above use cases and the last two cases can be evaluated in-line, commented out and warned about for manual translation. Something similar is used in the Boost library for its foreach macro. -- /Jacob Carlborg
Re: The CAPI Manifesto
I guess it's quite difficult for a compiler to recognize the differences between these use cases. Well, that's because C macros suck big-time. I don't see any other solution to the C-to-D translation problem.
Re: The CAPI Manifesto
Walter Bright wrote: On 10/17/2011 5:56 PM, so wrote: On Tue, 18 Oct 2011 03:52:13 +0300, Nick Sabalausky a@a.a wrote: Thats would mean that every D compiler would have to *also* be a C compiler. Indeed, but i see nothing wrong with it, like i see nothing wrong with inline asm, C never changes, While C code can be directly translated to D, the C macros are another matter. One way is to use probably number one C/C++ preprocessor now available in the world.: http://mcpp.sourceforge.net/ It's portable, BSD licensed and implements all of C90, C99 and C++98 specifications.
Re: The CAPI Manifesto
The main problem with this approach is how to support different versions of a
library, or of OS. It quickly becomes difficult to support anything but the
latest, or a fixed version.
It works beautifully for mature libs.
I still cannot avoid thinking that a C frontend automatically generating D
modules with the help of recipes would be a better way.
It will need some manual intervention for difficult cases, mainly giving
manual translation of some macros, but it should be small.
One would set if all the files correspond to modules, or there are just some
main directories/files.
Some things are easy:
#define a
enum { a=true }
#define b xyz
enum { b=xyz }
one could be tempted to replace
#ifdef x
with
static if (is(typeof(x)) x)
and treat other #if in a similar way, but in D a static if must contain a full
statement, as its content must be syntactically valid, whereas the C
preprocessor does not have this limitation.
The way to work around this, if we create the headers on demand is simple: we
already evaluate all #if using the building definitions of the associated C
compiler (gcc -E -dD for example) and its default include paths (or directly
use it, keeping in account the # line file directives).
real macros are more tricky, for example one could do
#define isSmall(x) (x2)
isSmall(T)(T x){
return x2;
}
#define c(x) { x , #x }
template(alias x){
{ x, x.stringof }
}
thus c(t) has to become c!(t).
and maybe one has to provide some macros definition by hand, but I guess the
cases are not so much.
In all this there is still a major pitfall: redefinitions of the same macro. It
is not common, but it happens, and when it does everything breaks.
One could give different names for the clashing symbols, but it remains ugly.
Furthermore in D one cannot define the same interface to a C function twice and
import it in the same scope through two modules, because it will clash, even if
private.
This makes the whole more complicated, but I think that a few recipes coding
the exceptions like macro translations, macros/defs to suppress or rename it
should work pretty well.
Once could analyze if different views of the same include file are
compatible, and automatically check for double definitions.
It isn't an easy project, but it would be very useful if done correctly. I
remember talking about it with Lindquist quite some time ago…
Fawzi
Re: The CAPI Manifesto
On Oct 21, 2011, at 4:20 PM, Fawzi Mohamed wrote: The main problem with this approach is how to support different versions of a library, or of OS. It quickly becomes difficult to support anything but the latest, or a fixed version. It works beautifully for mature libs. I still cannot avoid thinking that a C frontend automatically generating D modules with the help of recipes would be a better way. It will need some manual intervention for difficult cases, mainly giving manual translation of some macros, but it should be small. … and it seems that in the time I was offline others came up with the same idea...
Re: The CAPI Manifesto
On 10/21/2011 4:32 PM, Fawzi Mohamed wrote: On Oct 21, 2011, at 4:20 PM, Fawzi Mohamed wrote: The main problem with this approach is how to support different versions of a library, or of OS. It quickly becomes difficult to support anything but the latest, or a fixed version. It works beautifully for mature libs. Since github has excellent support for branches, I don't see why this is a major problem. I still cannot avoid thinking that a C frontend automatically generating D modules with the help of recipes would be a better way. It will need some manual intervention for difficult cases, mainly giving manual translation of some macros, but it should be small. … and it seems that in the time I was offline others came up with the same idea... It's an old idea. The trouble is, as always, the C preprocessor. I'm currently converting the openssl .h files, and they are a zoo of metaprogramming using C preprocessor macros. People are going to demand perfect translation if it is automatic. The only way to do it is to work with the preprocessed output of the .h file, and just forget about the preprocessor.
Re: The CAPI Manifesto
On 10/21/2011 12:41 AM, so wrote: You are right, i forgot about macros, Is it only this or is there anything else? The only other thing is what does one do about 'char' - make it a byte, ubyte, or char D type?
Re: The CAPI Manifesto
On 10/21/2011 5:29 AM, Piotr Szturmaj wrote: It's portable, BSD licensed and implements all of C90, C99 and C++98 specifications. Preprocessing the text is not the problem. The problem is determining a D translation of the macros.
Re: The CAPI Manifesto
so s...@so.so wrote in message news:op.v3ivsvb8mpw3zg@localhost.localdomain... On Tue, 18 Oct 2011 03:32:17 +0300, Jonathan M Davis jmdavisp...@gmx.com wrote: Translating C header files to D is a pain and time consuming, and it would benefit us all to have a place to go to find common C headers translated to D so that such work doesn't have to be duplicated. With the second compiler, there won't be any need for us to do anything, like C++ we could use C libraries seamlessly. The only reason C++ is able to do that is because C++ (realistically, even if not *technically*) is a proper superset of C. D isn't.
Re: The CAPI Manifesto
Am 18.10.2011, 07:26 Uhr, schrieb Walter Bright newshou...@digitalmars.com: On 10/17/2011 12:42 AM, Jacob Carlborg wrote: Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ Is it possible (and worthwhile) to layer a package manager over a github repository? All package managers have to store their packages in some sort of repository. It's more like it solves Jacob's problem where to upload the files now that you proposed CAPI :)
Re: The CAPI Manifesto
On Mon, Oct 17, 2011 at 11:46 PM, Lutger Blijdestijn lutger.blijdest...@gmail.com wrote: Walter Bright wrote: On 10/17/2011 12:42 AM, Jacob Carlborg wrote: Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ Is it possible (and worthwhile) to layer a package manager over a github repository? Yes, pip (for python) can do it. Homebrew[1] does too. It's rather slick actually. You can type brew edit package and it'll open the script it uses to install that package (typically just a configure and make install) in an editor. You make your changes and save and it saves to a local git clone of the formulas (package installation scripts). If you have a shared development environment you can share this modified repository among your developers so everyone is working with the same packages. It's design is rather clever and makes full use of DVCSes. Regards, Brad Anderson [1] http://mxcl.github.com/homebrew/
Re: The CAPI Manifesto
On 2011-10-18 07:26, Walter Bright wrote: On 10/17/2011 12:42 AM, Jacob Carlborg wrote: Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ Is it possible (and worthwhile) to layer a package manager over a github repository? I'm planning to support github repositories as packages. Have a look at Source Code Management: https://github.com/jacob-carlborg/orbit/wiki/integration -- /Jacob Carlborg
Re: new DIP12: C API headers (was The CAPI manifesto)
Martin Nowak wrote: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP12 Why using the package capi instead of just c? Jens
Re: The CAPI Manifesto
On 10/17/2011 11:40 PM, Jacob Carlborg wrote: On 2011-10-18 07:26, Walter Bright wrote: Is it possible (and worthwhile) to layer a package manager over a github repository? I'm planning to support github repositories as packages. Have a look at Source Code Management: https://github.com/jacob-carlborg/orbit/wiki/integration Great!
Re: The CAPI Manifesto
On 10/16/2011 9:28 PM, Jude Young wrote: Please excuse my ignorance, but several types in D do not currently translate well into C. For example, strings in D are not '\0' terminated, which breaks with C. This is incorrect. String literals in D are 0 terminated, as in C. C has a convention that strings are 0 terminated, but it isn't part of the core language. In D, you can terminate a string with 0 if you want to, or not. This is not usually a problem, and it's easy to wrap the function in the translated header file to automate that particular process. This would fall under attempting to fix the api, which would be beyond the scope of the library. It seems that your proposal would disallow this particular example. The translation code would have to be somewhere, what is the argument against allowing it? D access to the C API should be direct, and not include hidden costs like translation layers. In my (admittedly ignorant) opinion, it seems that allowing the automatic translation of D types to C types would fit very well, otherwise you'll have to add them yourself every time you wish to call C. C doesn't actually even have a 'string' type.
Re: The CAPI Manifesto
On 10/16/2011 10:24 PM, Gor Gyolchanyan wrote: I think there might be a few tricks to improve the C API without adding any new code. For example, replace by-pointer parameter declarations with _out_ parameters when applicable (the underlying function signature is the same), replace const parameters with in parameters, etc. This won't change the C API a single bit (won't even add new code), but will vastly improve readability and sometimes safety of the API. In other cases, some minor additions could be made, for example: libjpeg provides API to register error handlers, instead of setting errno and such. Those kind of situations could be used to throw exceptions. It only takes a static this() and a few lines of code. In case those kind of modifications/additions are made, there could be a standard way to disable them and use the original version. If you want to add a layer on top of the C API, that would be fine. std.zlib is an example of that. But the idea of CAPI is NOT to add a layer. Not fix, extend, refactor, improve, etc. Just the thinnest possible direct calls to the C API. Any improvements, fixes, whatever, should be a separate project. I know the urge to do these fixes can be overpowering, but they end badly every time. It's like trying to mix the language's lexer up with the semantic analysis :-) They really are better off being separate and distinct. For one thing, it makes the inevitable maintenance *FAR* easier, as those C APIs will change. Updating the corresponding D module becomes simple then - just a line by line comparison and tweaking. There would be no tearing of hair and rending of garments. For another it means you'll have to regenerate the C documentation, but with all the changes you made. Then, as the C guys improve their documentation, your layer falls behind, gets neglected, and finally sucks in comparison.
Re: The CAPI Manifesto
On Mon, 17 Oct 2011 08:24:26 +0300, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: For example, replace by-pointer parameter declarations with _out_ parameters when applicable (the underlying function signature is the same) Note that this would make it nearly impossible to pass null pointers. A human would need to decide if a null pointer can be specified, or the converter would need to be aware of proprietary extensions which specify such things (e.g. Microsoft's __in). -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: The CAPI Manifesto
On 10/16/2011 11:21 PM, Vladimir Panteleev wrote: On Mon, 17 Oct 2011 08:24:26 +0300, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: For example, replace by-pointer parameter declarations with _out_ parameters when applicable (the underlying function signature is the same) Note that this would make it nearly impossible to pass null pointers. A human would need to decide if a null pointer can be specified, or the converter would need to be aware of proprietary extensions which specify such things (e.g. Microsoft's __in). Right. You could also annotate C API functions with pure, @safe, etc., but you'd have to be very careful that those functions actually were that way, and would not violate those attributes in the future.
Re: The CAPI Manifesto
Fair enough, thanks for the time. I found a thin D binding to ncurses, and 'fixed' it to work with D2. If y'all get this up and running, this code will probably be better than starting from scratch. In any case, I think that CAPI is exactly the type of thing that D needs going forward. I really hope that CAPI gets good support. On Mon, Oct 17, 2011 at 12:55 AM, Walter Bright newshou...@digitalmars.comwrote: On 10/16/2011 9:28 PM, Jude Young wrote: Please excuse my ignorance, but several types in D do not currently translate well into C. For example, strings in D are not '\0' terminated, which breaks with C. This is incorrect. String literals in D are 0 terminated, as in C. C has a convention that strings are 0 terminated, but it isn't part of the core language. In D, you can terminate a string with 0 if you want to, or not. This is not usually a problem, and it's easy to wrap the function in the translated header file to automate that particular process. This would fall under attempting to fix the api, which would be beyond the scope of the library. It seems that your proposal would disallow this particular example. The translation code would have to be somewhere, what is the argument against allowing it? D access to the C API should be direct, and not include hidden costs like translation layers. In my (admittedly ignorant) opinion, it seems that allowing the automatic translation of D types to C types would fit very well, otherwise you'll have to add them yourself every time you wish to call C. C doesn't actually even have a 'string' type.
Re: The CAPI Manifesto
On 2011-10-17 04:18, Ary Manzana wrote: On 10/16/11 11:02 PM, Walter Bright wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. So you would put every interface to every possible C code there? In Ruby if you want to have very efficient code you'd implement it as C extensions. For that, you create wrappers in Ruby for C. Now, big part of the standard library has extensions for the most needed things. Everything else, like bindings to an efficient xml parser, are made by different people that public them as gems. Having a public gem repository it's really easy to find bindings for whatever you want. They don't need to be part of the standard library. And it wouldn't make sense, having so much functionality out there available as C code. So I'd suggest having D headers for the most common things in phobos and focusing on a tool like rubygems. It would give such a big boost to the language. I also can't imagine how that big repository would work. You'd copy the remote file locally? What if that file gets fixes? You'd copy it again? Or maybe you'd git checkout everything from that repository locally and synchronize it from time to time, with the chance of breaking existing code... Having gems and versioning them should make all these problems disappear. Maybe there is an openssl header in D. The problem is that there might be many, and they don't know each other, and google is a maze to find such things. Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ -- /Jacob Carlborg
Re: The CAPI Manifesto
On 2011-10-17 04:02, Walter Bright wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. I'm working on a Clang based tool for automatically converting C header files to D modules. -- /Jacob Carlborg
Re: The CAPI Manifesto
That sounds good. Now D projects will have a mother and a father. Phobos is the mother, which is always there for you and is the first one you'll go to if you need help :-) Diemos is the father, which is the backbone of your software family, who you rely on to do all the hard work for you. :-) It's generally a good idea to separate our own code from an external one, which we provide support for. And if the remote modules proposal gets implemented, there would be no problems with using Diemos if it won't get included in DMD package. On Mon, Oct 17, 2011 at 7:55 AM, Walter Bright newshou...@digitalmars.com wrote: On 10/16/2011 8:17 PM, Vladimir Panteleev wrote: I think the name's fine. I don't see 4-letter word is already taken as a valid argument, considering the huge number of users. There's a GitHub user called phobos and one called tools, etc. Also, I'm pretty sure the D standard library is named after Mars's moon, considering that D was originally named Mars Programming Language (after the company name, Digital Mars). I thought maybe Diemos.
Re: The CAPI Manifesto
Unless CAPI is trademarked, I think we're in good shape. when i hear of CAPI i instantly think of this: http://www.capi.org/pages/home.php
Re: The CAPI Manifesto
Yeah. Diemos is much better, IMO. On Mon, Oct 17, 2011 at 12:30 PM, Mirko Pilger pil...@cymotec.de wrote: Unless CAPI is trademarked, I think we're in good shape. when i hear of CAPI i instantly think of this: http://www.capi.org/pages/home.php
Re: The CAPI Manifesto
On Oct 16, 2011, at 7:02 PM, Walter Bright wrote: The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; If the C header file has a name that is a D keyword, an underscore will be appended to the D module name. If a C type name matches a C function name (stat), the type name will have a _t appended. There's also the occasional issue of something that doesn't translate into D. As one slightly weird example, some of the the Posix routines in OSX have alternates with odd suffixes like $2003 that are the versions which should be called on newer versions of the OS. I'm still not sure of the best way to handle this, since D doesn't have macros.
Re: The CAPI Manifesto
Am 17.10.2011, 10:20 Uhr, schrieb Gor Gyolchanyan gor.f.gyolchan...@gmail.com: That sounds good. Now D projects will have a mother and a father. Phobos is the mother, which is always there for you and is the first one you'll go to if you need help :-) Diemos is the father, which is the backbone of your software family, who you rely on to do all the hard work for you. :-) It's generally a good idea to separate our own code from an external one, which we provide support for. And if the remote modules proposal gets implemented, there would be no problems with using Diemos if it won't get included in DMD package. On Mon, Oct 17, 2011 at 7:55 AM, Walter Bright newshou...@digitalmars.com wrote: On 10/16/2011 8:17 PM, Vladimir Panteleev wrote: I think the name's fine. I don't see 4-letter word is already taken as a valid argument, considering the huge number of users. There's a GitHub user called phobos and one called tools, etc. Also, I'm pretty sure the D standard library is named after Mars's moon, considering that D was originally named Mars Programming Language (after the company name, Digital Mars). I thought maybe Diemos. Deimos, people, its Deimos .
Re: The CAPI Manifesto
I'm working on a Clang based tool for automatically converting C header files to D modules. Great! Does it work yet?
Re: The CAPI Manifesto
On 2011-10-17 13:27, Steve Teale wrote: I'm working on a Clang based tool for automatically converting C header files to D modules. Great! Does it work yet? Well, yes. Some parts of it. I have mostly focused on converting Objective-C headers. I have also started to rewrite the tool to use libclang instead of embed it straight into clang. https://github.com/jacob-carlborg/clang https://github.com/jacob-carlborg/dstep -- /Jacob Carlborg
Re: The CAPI Manifesto
On 2011-10-17 10:21:45 +, Sean Kelly s...@invisibleduck.org said: On Oct 16, 2011, at 7:02 PM, Walter Bright wrote: The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; If the C header file has a name that is a D keyword, an underscore will be appended to the D module name. If a C type name matches a C function name (stat), the type name will have a _t appended. Hum, but _t in C stands for typedef. Wouldn't it be better to just append an underscore like for module names, that'd make only one rule to remember. There's also the occasional issue of something that doesn't translate into D. As one slightly weird example, some of the the Posix routines in OSX have alternates with odd suffixes like $2003 that are the versions which should be called on newer versions of the OS. I'm still not sure of the best way to handle this, since D doesn't have macros. I think what D needs to handle that is some pragma to manually specify the mangled name of a given function. Why would you need macros? -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: The CAPI Manifesto
On 2011-10-17 14:01, Michel Fortin wrote: On 2011-10-17 10:21:45 +, Sean Kelly s...@invisibleduck.org said: On Oct 16, 2011, at 7:02 PM, Walter Bright wrote: The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; If the C header file has a name that is a D keyword, an underscore will be appended to the D module name. If a C type name matches a C function name (stat), the type name will have a _t appended. Hum, but _t in C stands for typedef. Wouldn't it be better to just append an underscore like for module names, that'd make only one rule to remember. There's also the occasional issue of something that doesn't translate into D. As one slightly weird example, some of the the Posix routines in OSX have alternates with odd suffixes like $2003 that are the versions which should be called on newer versions of the OS. I'm still not sure of the best way to handle this, since D doesn't have macros. I think what D needs to handle that is some pragma to manually specify the mangled name of a given function. Why would you need macros? Perhaps the macro is used to determine if foo or foo$2003 is supposed to be called, based on some condition. -- /Jacob Carlborg
Re: The CAPI Manifesto
Michel Fortin michel.for...@michelf.com wrote in message news:j7h5gp$2d7n$1...@digitalmars.com... I think what D needs to handle that is some pragma to manually specify the mangled name of a given function. Why would you need macros? I've got a patch to do this, in the pragma_mangle branch of my fork. One day I'll get around to fixing it up and making a pull request.
Re: The CAPI Manifesto
On 2011-10-17 13:41:14 +, Jacob Carlborg d...@me.com said: On 2011-10-17 14:01, Michel Fortin wrote: On 2011-10-17 10:21:45 +, Sean Kelly s...@invisibleduck.org said: There's also the occasional issue of something that doesn't translate into D. As one slightly weird example, some of the the Posix routines in OSX have alternates with odd suffixes like $2003 that are the versions which should be called on newer versions of the OS. I'm still not sure of the best way to handle this, since D doesn't have macros. I think what D needs to handle that is some pragma to manually specify the mangled name of a given function. Why would you need macros? Perhaps the macro is used to determine if foo or foo$2003 is supposed to be called, based on some condition. Indeed. The condition is which OS release you're targeting. That can be accomplished today through static ifs. Although it'd be a little more verbose since you'd have to repeat the function prototype. If we had a way to do conditional attributes in D it'd be awesome for this use case. It could work this way for instance: static if (MAC_OS_X_VERSION_MIN_REQUIRED == 10.5) @deprecated_in_os_x_10_5 = deprecated; else @deprecated_in_os_x_10_5 = /* nothing */; @deprecated_in_os_x_10_5 void some_function_deprecated_in_os_x_10_5(); Or this way for the special mangled names: static if (MAC_OS_X_VERSION_MIN_REQUIRED == 10.5) @darwin_alias(name) = pragma(symbol_name, name ~ $UNIX2003); else @darwin_alias(name) = pragma(symbol_name, name); @darwin_alias(fwrite) size_t fwrite(const void * /*__restrict*/, size_t, size_t, FILE * /*__restrict*/); Internally, when the compiler sees @darwin_alias(fwrite) it just replaces it with the attributes @darwin_alias was supposed to be. Note that I'm *not* proposing a macro system: this would work at the semantic level as a special kind of attribute. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: The CAPI Manifesto
Well then my vote goes for let's do it. Simple bindings can be started right away, probably by copying from dsource bindings and doing any modifications necessary. For non-trivial C headers we can discuss them here methinks.
Re: The CAPI Manifesto
I like it! :)
Re: The CAPI Manifesto
+1 Let's do it!
Re: The CAPI Manifesto
Am 17.10.2011 12:21, schrieb Sean Kelly: On Oct 16, 2011, at 7:02 PM, Walter Bright wrote: The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; If the C header file has a name that is a D keyword, an underscore will be appended to the D module name. If a C type name matches a C function name (stat), the type name will have a _t appended. There's also the occasional issue of something that doesn't translate into D. As one slightly weird example, some of the the Posix routines in OSX have alternates with odd suffixes like $2003 that are the versions which should be called on newer versions of the OS. I'm still not sure of the best way to handle this, since D doesn't have macros. What about function-like macros, e.g. the Linux/POSIX cmsg stuff (CMSG_FIRSTHDR(), CMSG_NXTHDR(), CMSG_LEN() etc) needed to use functions like recvmsg() and sendmsg()? Will there be a direct D translation of the functionality or will they be omitted completely? Cheers, - Daniel
Re: The CAPI Manifesto
On 10/17/2011 1:24 PM, Daniel Gibson wrote:
What about function-like macros, e.g. the Linux/POSIX cmsg stuff
(CMSG_FIRSTHDR(), CMSG_NXTHDR(), CMSG_LEN() etc) needed to use functions like
recvmsg() and sendmsg()?
Will there be a direct D translation of the functionality or will they be
omitted completely?
Consider:
#define FOO(x) bar((x) + 1)
Do this:
int FOO()(int x) { return bar(x) + 1; }
Note that it's a function template with no template parameters. This will enable
it to be header only and not require linking to some library to resolve FOO().
Re: The CAPI Manifesto
Regan Heath re...@netmail.co.nz wrote in message news:op.v3h9w20554x...@puck.auriga.bhead.co.uk... I like it! :) vote++
Re: The CAPI Manifesto
On 17/10/11 10:33 PM, Walter Bright wrote:
On 10/17/2011 1:24 PM, Daniel Gibson wrote:
What about function-like macros, e.g. the Linux/POSIX cmsg stuff
(CMSG_FIRSTHDR(), CMSG_NXTHDR(), CMSG_LEN() etc) needed to use
functions like
recvmsg() and sendmsg()?
Will there be a direct D translation of the functionality or will they be
omitted completely?
Consider:
#define FOO(x) bar((x) + 1)
Do this:
int FOO()(int x) { return bar(x) + 1; }
Note that it's a function template with no template parameters. This
will enable it to be header only and not require linking to some
library to resolve FOO().
int FOO()(int x) { return bar(x + 1); }
would probably work better :-)
+1 for CAPI btw.
Re: The CAPI Manifesto
With D being binary compatible with C, i don't know why we worry on such things. Wasn't being able to access C libraries the point? If it wasn't, what is the worthwhile point for this constraint? Wouldn't (sorry for the poor horse) separate compilers solve the most problems (if not all) we face on these issues? C never changes and every compiler vendor have an implementation. -- import anyapi; // anyapi would be a D module or a C header (anyapi.h...) in directory paths. -- Structs are pod in both languages. Matching of the standard types is something we can take care of with documentation (RTFM) and with compiler errors generated (when we call functions from the other language). Sorry once again if this should sound stupid or impossible to implement (if so, someone please enlighten me), it probably is because everytime we open this discussion i feel i am the only one seeing the big picture, the potential of D. On Mon, 17 Oct 2011 05:02:52 +0300, Walter Bright newshou...@digitalmars.com wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; Each C .h file would have a corresponding .d file. Each C directory would have a corresponding D directory, for example: #include bar/foo.h // C import bar.foo; // D The top level directory of each library will have two subdirectories: C/ D/ and there will be a one-to-one correspondence of files and directory structure between them. The D import files will be a rote translation of the corresponding C .h file. No attempt will be made to fix, improve, or extend the C api. No attempt will be made to duplicate the C documentation, or replace it in any way. There will be no unittests. Every effort will be made to avoid needing any D specific binary files. When an updated version of the C header files becomes available, those will get checked into the C subdirectory tree, and then the corresponding D files will get updated. Version tags used must match the version tags used by the C API files. The license used for the D versions should match the C ones, as they are a derived work.
Re: The CAPI Manifesto
On Monday, October 17, 2011 17:09 so wrote: With D being binary compatible with C, i don't know why we worry on such things. Wasn't being able to access C libraries the point? If it wasn't, what is the worthwhile point for this constraint? Wouldn't (sorry for the poor horse) separate compilers solve the most problems (if not all) we face on these issues? C never changes and every compiler vendor have an implementation. -- import anyapi; // anyapi would be a D module or a C header (anyapi.h...) in directory paths. -- Structs are pod in both languages. Matching of the standard types is something we can take care of with documentation (RTFM) and with compiler errors generated (when we call functions from the other language). Sorry once again if this should sound stupid or impossible to implement (if so, someone please enlighten me), it probably is because everytime we open this discussion i feel i am the only one seeing the big picture, the potential of D. The problem is that for C code to be usable in D code, the C declarations must be redone in D, since D can't just include header files. Translating C header files to D is a pain and time consuming, and it would benefit us all to have a place to go to find common C headers translated to D so that such work doesn't have to be duplicated. Hence CAPI has been proposed. And if we're going to have it, it also benefits us to be organized about how we lay it out. Programmers can name modules in their code whatever they want, but being organized about how modules are named and laid out in a large project such as this makes it much easier to maintain and find what you want in it. - Jonathan M Davis
Re: The CAPI Manifesto
so s...@so.so wrote in message news:op.v3itelnbmpw3zg@localhost.localdomain... With D being binary compatible with C, i don't know why we worry on such things. Wasn't being able to access C libraries the point? If it wasn't, what is the worthwhile point for this constraint? Wouldn't (sorry for the poor horse) separate compilers solve the most problems (if not all) we face on these issues? C never changes and every compiler vendor have an implementation. -- import anyapi; // anyapi would be a D module or a C header (anyapi.h...) in directory paths. -- Thats would mean that every D compiler would have to *also* be a C compiler.
Re: The CAPI Manifesto
On Tue, 18 Oct 2011 03:52:13 +0300, Nick Sabalausky a@a.a wrote: Thats would mean that every D compiler would have to *also* be a C compiler. Indeed, but i see nothing wrong with it, like i see nothing wrong with inline asm, C never changes,
Re: The CAPI Manifesto
On Tue, 18 Oct 2011 03:32:17 +0300, Jonathan M Davis jmdavisp...@gmx.com wrote: The problem is that for C code to be usable in D code, the C declarations must be redone in D, since D can't just include header files. I don't understand why. Translating C header files to D is a pain and time consuming, and it would benefit us all to have a place to go to find common C headers translated to D so that such work doesn't have to be duplicated. With the second compiler, there won't be any need for us to do anything, like C++ we could use C libraries seamlessly.
new DIP12: C API headers (was The CAPI manifesto)
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP12
Re: The CAPI Manifesto
On 10/17/2011 4:04 PM, Peter Alexander wrote:
int FOO()(int x) { return bar(x + 1); }
would probably work better :-)
:-)
Re: The CAPI Manifesto
On 10/17/2011 5:56 PM, so wrote: On Tue, 18 Oct 2011 03:52:13 +0300, Nick Sabalausky a@a.a wrote: Thats would mean that every D compiler would have to *also* be a C compiler. Indeed, but i see nothing wrong with it, like i see nothing wrong with inline asm, C never changes, While C code can be directly translated to D, the C macros are another matter.
Re: The CAPI Manifesto
On 10/17/2011 12:42 AM, Jacob Carlborg wrote: Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ Is it possible (and worthwhile) to layer a package manager over a github repository?
Re: The CAPI Manifesto
Walter Bright wrote: On 10/17/2011 12:42 AM, Jacob Carlborg wrote: Already working on a package manager for D: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D https://github.com/jacob-carlborg/orbit/ Is it possible (and worthwhile) to layer a package manager over a github repository? Yes, pip (for python) can do it.
The CAPI Manifesto
Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; Each C .h file would have a corresponding .d file. Each C directory would have a corresponding D directory, for example: #include bar/foo.h // C import bar.foo; // D The top level directory of each library will have two subdirectories: C/ D/ and there will be a one-to-one correspondence of files and directory structure between them. The D import files will be a rote translation of the corresponding C .h file. No attempt will be made to fix, improve, or extend the C api. No attempt will be made to duplicate the C documentation, or replace it in any way. There will be no unittests. Every effort will be made to avoid needing any D specific binary files. When an updated version of the C header files becomes available, those will get checked into the C subdirectory tree, and then the corresponding D files will get updated. Version tags used must match the version tags used by the C API files. The license used for the D versions should match the C ones, as they are a derived work.
Re: The CAPI Manifesto
On 10/16/11 11:02 PM, Walter Bright wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. So you would put every interface to every possible C code there? In Ruby if you want to have very efficient code you'd implement it as C extensions. For that, you create wrappers in Ruby for C. Now, big part of the standard library has extensions for the most needed things. Everything else, like bindings to an efficient xml parser, are made by different people that public them as gems. Having a public gem repository it's really easy to find bindings for whatever you want. They don't need to be part of the standard library. And it wouldn't make sense, having so much functionality out there available as C code. So I'd suggest having D headers for the most common things in phobos and focusing on a tool like rubygems. It would give such a big boost to the language. I also can't imagine how that big repository would work. You'd copy the remote file locally? What if that file gets fixes? You'd copy it again? Or maybe you'd git checkout everything from that repository locally and synchronize it from time to time, with the chance of breaking existing code... Having gems and versioning them should make all these problems disappear. Maybe there is an openssl header in D. The problem is that there might be many, and they don't know each other, and google is a maze to find such things.
Re: The CAPI Manifesto
Overall, I think that it sounds like a good idea, but I'd suggest that we namespace stuff. So, instead of import foo; you'd get something like import capi.foo; Otherwise, we unnecessarily increase the odds of module names conflicting with modules in other projects. Also, if we're putting it up on github, we may want to come up with a cooler name than CAPI (though it _is_ right to the point). There's already a user with the name CAPI ( https://github.com/capi ), so aside from whether CAPI is a good name in its own right or not, that might cause problems. Maybe pick something suitably Greek or Roman, given that we already have Phobos? The name of one of the Titans would be fitting given that it relates to C (on the theory that the Greek gods are the successors of the Titans and D's standard library is named after a Greek god). It also brings up the question of who is going to manage this project. Someone (or preferrably, a group of someones) is going to have to have commit access in order to merge in pull requests. Do want to just start off with that being the same group of folks with Phobos commit access and grow it from there as appropriate? We certainly wouldn't want to insist that the two groups be the same, but it does seem like a good place to start from. - Jonathan M Davis
Re: The CAPI Manifesto
On Mon, 17 Oct 2011 05:02:52 +0300, Walter Bright newshou...@digitalmars.com wrote: So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. Questions: 1) Will these be distributed with DMD? The bar to generate D import modules from C headers isn't much higher than having to find and download headers from the Internet. 2) This isn't too different to already existing projects. The bindings project on dsource ( http://dsource.org/projects/bindings ) already has bindings for various libraries. I can't say much about the project guidelines, though. 3) You suggest to place each library in its own directory, with C and D headers as subdirectories. This means that the user will still need to edit the import search path when using a new library. Is it realistic to put all D files in the same directory? (Perhaps do this only for libraries for which we don't expect name collisions?) 4) Every effort will be made to avoid needing any D specific binary files. - What about import libraries? -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: The CAPI Manifesto
On Mon, 17 Oct 2011 05:51:57 +0300, Jonathan M Davis jmdavisp...@gmx.com wrote: Also, if we're putting it up on github, we may want to come up with a cooler name than CAPI (though it _is_ right to the point). There's already a user with the name CAPI ( https://github.com/capi ), so aside from whether CAPI is a good name in its own right or not, that might cause problems. Maybe pick something suitably Greek or Roman, given that we already have Phobos? The name of one of the Titans would be fitting given that it relates to C (on the theory that the Greek gods are the successors of the Titans and D's standard library is named after a Greek god). I think the name's fine. I don't see 4-letter word is already taken as a valid argument, considering the huge number of users. There's a GitHub user called phobos and one called tools, etc. Also, I'm pretty sure the D standard library is named after Mars's moon, considering that D was originally named Mars Programming Language (after the company name, Digital Mars). -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: The CAPI Manifesto
On 10/16/2011 8:17 PM, Vladimir Panteleev wrote: I think the name's fine. I don't see 4-letter word is already taken as a valid argument, considering the huge number of users. There's a GitHub user called phobos and one called tools, etc. Also, I'm pretty sure the D standard library is named after Mars's moon, considering that D was originally named Mars Programming Language (after the company name, Digital Mars). I thought maybe Diemos.
Re: The CAPI Manifesto
Is this repo intended for use by users or library writers? I don't think encouraging writing C-style code in D is a good idea, that's why I'm asking.
Re: The CAPI Manifesto
On 10/16/2011 8:10 PM, Vladimir Panteleev wrote: 1) Will these be distributed with DMD? The bar to generate D import modules from C headers isn't much higher than having to find and download headers from the Internet. I was thinking, no at this time. I suspect it may grow to be quite large, and would become rather onerous. I also don't want to tie it to the DMD release cycle. 2) This isn't too different to already existing projects. The bindings project on dsource ( http://dsource.org/projects/bindings ) already has bindings for various libraries. I can't say much about the project guidelines, though. The bindings project is a great resource, though it seems a little disorganized. We've had great success with github, meaning it seems to be very good at encouraging community participation. 3) You suggest to place each library in its own directory, with C and D headers as subdirectories. This means that the user will still need to edit the import search path when using a new library. Yes. Is it realistic to put all D files in the same directory? (Perhaps do this only for libraries for which we don't expect name collisions?) Hmm, you're right. Perhaps openssl.whatever. 4) Every effort will be made to avoid needing any D specific binary files. - What about import libraries? What do you mean by import libraries? Do you mean Windows DLL import libraries? Those would be supplied by whoever supplied the C library. D can access those directly. CAPI should be interface source code only library; the D equivalent of #include.
Re: The CAPI Manifesto
On 10/16/2011 7:18 PM, Ary Manzana wrote: So you would put every interface to every possible C code there? No. It's simply a collection point for interfaces to publicly available C libraries. The place to go to see if someone has already written what you're looking for. Its mission is also narrowly focused on simply being a way to access existing C libraries from D in the most straightforward way possible. In Ruby if you want to have very efficient code you'd implement it as C extensions. For that, you create wrappers in Ruby for C. Now, big part of the standard library has extensions for the most needed things. Everything else, like bindings to an efficient xml parser, are made by different people that public them as gems. Having a public gem repository it's really easy to find bindings for whatever you want. They don't need to be part of the standard library. And it wouldn't make sense, having so much functionality out there available as C code. Right. So I'd suggest having D headers for the most common things in phobos Yes. and focusing on a tool like rubygems. It would give such a big boost to the language. I think github is adequate for now to be the tool. I also can't imagine how that big repository would work. You'd copy the remote file locally? That'll work, or you can use the github 'clone' feature. What if that file gets fixes? You'd copy it again? Or maybe you'd git checkout everything from that repository locally and synchronize it from time to time, with the chance of breaking existing code... Having gems and versioning them should make all these problems disappear. Github has extensive versioning and branching support. This shouldn't be an issue. Maybe there is an openssl header in D. The problem is that there might be many, and they don't know each other, and google is a maze to find such things. Exactly. And maybe there are several openssl D headers, each of which is half-assed in a different way. With a central library for them, we can improve them globally rather than piecemeal and randomly.
Re: The CAPI Manifesto
On 10/16/2011 7:51 PM, Jonathan M Davis wrote: Overall, I think that it sounds like a good idea, but I'd suggest that we namespace stuff. So, instead of import foo; you'd get something like import capi.foo; Otherwise, we unnecessarily increase the odds of module names conflicting with modules in other projects. Perhaps you're right. Also, if we're putting it up on github, we may want to come up with a cooler name than CAPI (though it _is_ right to the point). There's already a user with the name CAPI ( https://github.com/capi ), so aside from whether CAPI is a good name in its own right or not, that might cause problems. Maybe pick something suitably Greek or Roman, given that we already have Phobos? The name of one of the Titans would be fitting given that it relates to C (on the theory that the Greek gods are the successors of the Titans and D's standard library is named after a Greek god). Unless CAPI is trademarked, I think we're in good shape. It also brings up the question of who is going to manage this project. Someone (or preferrably, a group of someones) is going to have to have commit access in order to merge in pull requests. Do want to just start off with that being the same group of folks with Phobos commit access and grow it from there as appropriate? We certainly wouldn't want to insist that the two groups be the same, but it does seem like a good place to start from. I figure initially the same team members as are on the phobos team, but it would be a separate team with its own member list.
Re: The CAPI Manifesto
On 10/16/2011 9:02 PM, Andrej Mitrovic wrote: Is this repo intended for use by users or library writers? Both. I don't think encouraging writing C-style code in D is a good idea, that's why I'm asking. D's ability to directly call any C code is a huge advantage, but one we've underutilized because of a lack of interface imports to the great mass of popular C libraries. For example, are we going to reinvent openssl? libcurl? imagemajick? No way.
Re: The CAPI Manifesto
Please excuse my ignorance, but several types in D do not currently translate well into C. For example, strings in D are not '\0' terminated, which breaks with C. This is not usually a problem, and it's easy to wrap the function in the translated header file to automate that particular process. It seems that your proposal would disallow this particular example. The translation code would have to be somewhere, what is the argument against allowing it? In my (admittedly ignorant) opinion, it seems that allowing the automatic translation of D types to C types would fit very well, otherwise you'll have to add them yourself every time you wish to call C. Forgive me If I have misunderstood something.
Re: The CAPI Manifesto
I think there might be a few tricks to improve the C API without adding any new code. For example, replace by-pointer parameter declarations with _out_ parameters when applicable (the underlying function signature is the same), replace const parameters with in parameters, etc. This won't change the C API a single bit (won't even add new code), but will vastly improve readability and sometimes safety of the API. In other cases, some minor additions could be made, for example: libjpeg provides API to register error handlers, instead of setting errno and such. Those kind of situations could be used to throw exceptions. It only takes a static this() and a few lines of code. In case those kind of modifications/additions are made, there could be a standard way to disable them and use the original version. On Mon, Oct 17, 2011 at 6:02 AM, Walter Bright newshou...@digitalmars.com wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; Each C .h file would have a corresponding .d file. Each C directory would have a corresponding D directory, for example: #include bar/foo.h // C import bar.foo; // D The top level directory of each library will have two subdirectories: C/ D/ and there will be a one-to-one correspondence of files and directory structure between them. The D import files will be a rote translation of the corresponding C .h file. No attempt will be made to fix, improve, or extend the C api. No attempt will be made to duplicate the C documentation, or replace it in any way. There will be no unittests. Every effort will be made to avoid needing any D specific binary files. When an updated version of the C header files becomes available, those will get checked into the C subdirectory tree, and then the corresponding D files will get updated. Version tags used must match the version tags used by the C API files. The license used for the D versions should match the C ones, as they are a derived work.
Re: The CAPI Manifesto
OR Maybe parallel to the CAPI, there could be the D-ified version of it, that will be developed after the original CAPI. This is, of course, not as urgent a the CAPI itself, but it would be very useful to gradually help users get rid of unnecessarily dangerous code. On Mon, Oct 17, 2011 at 9:24 AM, Gor Gyolchanyan gor.f.gyolchan...@gmail.com wrote: I think there might be a few tricks to improve the C API without adding any new code. For example, replace by-pointer parameter declarations with _out_ parameters when applicable (the underlying function signature is the same), replace const parameters with in parameters, etc. This won't change the C API a single bit (won't even add new code), but will vastly improve readability and sometimes safety of the API. In other cases, some minor additions could be made, for example: libjpeg provides API to register error handlers, instead of setting errno and such. Those kind of situations could be used to throw exceptions. It only takes a static this() and a few lines of code. In case those kind of modifications/additions are made, there could be a standard way to disable them and use the original version. On Mon, Oct 17, 2011 at 6:02 AM, Walter Bright newshou...@digitalmars.com wrote: Brad and I were talking about some D code that needed openssl support, when we ran into the same old problem: No D files corresponding to the openssl C .h files. It's not that these are a big problem to create, it's just that they are not done, and it tends to turn off people from using D. D is binary API compatible with C, but only with a corresponding D import file. This, out of the box, makes D *harder* to use than C. Lots of people roll their own, but that work is hard to find and haphazard. This problem keeps coming up again and again. So I propose creating, on github.com/D-Programming-Language, a new repository called CAPI. The CAPI Manifesto -- CAPI is a collection of C header files to publicly available C libraries and their translations to D. The idea is that if, in C, to interface to a library one would write: #include foo.h then the corresponding D code would look like: import foo; Each C .h file would have a corresponding .d file. Each C directory would have a corresponding D directory, for example: #include bar/foo.h // C import bar.foo; // D The top level directory of each library will have two subdirectories: C/ D/ and there will be a one-to-one correspondence of files and directory structure between them. The D import files will be a rote translation of the corresponding C .h file. No attempt will be made to fix, improve, or extend the C api. No attempt will be made to duplicate the C documentation, or replace it in any way. There will be no unittests. Every effort will be made to avoid needing any D specific binary files. When an updated version of the C header files becomes available, those will get checked into the C subdirectory tree, and then the corresponding D files will get updated. Version tags used must match the version tags used by the C API files. The license used for the D versions should match the C ones, as they are a derived work.
