[Issue 24444] ImportC: no way to specify where header files "live"

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=2

Lance Bachmeier  changed:

   What|Removed |Added

 CC||la...@lancebachmeier.com

--- Comment #3 from Lance Bachmeier  ---
To be honest, it's not that easy to find on this page either:
https://dlang.org/spec/importc.html#manual-cpp

Under "Running the Preprocessor Automatically" it says "The -Ppreprocessorflag
switch passes preprocessorflag to the preprocessor." It's not even searchable
using that wording.

--

[Issue 24444] ImportC: no way to specify where header files "live"

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=2

--- Comment #2 from Atila Neves  ---
I see. The -P flag doesn't show up in the man page.

--

[Issue 24444] ImportC: no way to specify where header files "live"

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=2

Dennis  changed:

   What|Removed |Added

 CC||dkor...@live.nl
   Hardware|x86_64  |All
 OS|Linux   |All

--- Comment #1 from Dennis  ---
There is the `-P=` option:

```
dmd -P-I/usr/include/python3.11
```

--

[Issue 24444] ImportC: no way to specify where header files "live"

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=2

Atila Neves  changed:

   What|Removed |Added

   Keywords||ImportC

--

[Issue 24444] New: ImportC: no way to specify where header files "live"

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=2

  Issue ID: 2
   Summary: ImportC: no way to specify where header files "live"
   Product: D
   Version: D2
  Hardware: x86_64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P1
 Component: dmd
  Assignee: nob...@puremagic.com
  Reporter: atila.ne...@gmail.com

This C file fails to compile with ImportC on my system:

---
#include 
---

The reason why is that `Python.h` is in `/usr/include/python3.11`. There's no
way to tell the compiler to look for headers in specific directories.

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

--- Comment #13 from Walter Bright  ---
(In reply to Andrej Mitrovic from comment #1)
> http://d-programming-language.org/dll.html

Now https://wiki.dlang.org/Win32_DLLs_in_D

--

[Issue 23547] [REG 2.101-master] C header files have precedent over D modules in imports

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Walter Bright  changed:

   What|Removed |Added

 CC||bugzi...@digitalmars.com

--- Comment #4 from Walter Bright  ---
The config.h file is found first because config.h appears before config.d in
the import search path.

If config.h and config.d were in the same directory, the .d file would take
precedence.

--

[Issue 23547] [REG 2.101-master] C header files have precedent over D modules in imports

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

--- Comment #3 from Iain Buclaw  ---
*** Issue 23548 has been marked as a duplicate of this issue. ***

--

[Issue 23547] [REG 2.101-master] C header files have precedent over D modules in imports

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Iain Buclaw  changed:

   What|Removed |Added

Summary|[REG master] C header files |[REG 2.101-master] C header
   |have precedent over D   |files have precedent over D
   |modules in imports  |modules in imports

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P2  |P3

--

[Issue 18589] Windows header files: bcrypt and ncrypt

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18589

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P4

--

[Issue 18363] we should autogenerate duplicate “.h” header files in dmd to keep them in sync

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18363

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P4

--

[Issue 23547] [REG master] C header files have precedent over D modules in imports

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Iain Buclaw  changed:

   What|Removed |Added

Summary|[REG master] C header files |[REG master] C header files
   |have precedent over D   |have precedent over D
   |modules |modules in imports

--

[Issue 23547] [REG master] C header files have precedent over D modules

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Iain Buclaw  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=23548

--

[Issue 23547] [REG master] C header files have precedent over D modules

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Iain Buclaw  changed:

   What|Removed |Added

   Keywords||ImportC, rejects-valid,
   ||wrong-code

--

[Issue 23547] [REG master] C header files have precedent over D modules

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

--- Comment #2 from Iain Buclaw  ---
https://github.com/dlang/dmd/pull/14636

--

[Issue 23547] [REG master] C header files have precedent over D modules

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

Iain Buclaw  changed:

   What|Removed |Added

 CC||ibuc...@gdcproject.org

--- Comment #1 from Iain Buclaw  ---
Caused by fix for issue 23479

--

[Issue 23547] New: [REG master] C header files have precedent over D modules

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23547

  Issue ID: 23547
   Summary: [REG master] C header files have precedent over D
modules
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: regression
  Priority: P1
 Component: dmd
  Assignee: nob...@puremagic.com
  Reporter: ibuc...@gdcproject.org

This test fails.

$ dmd -I=imports importd.d
importd.d(2): Error: undefined identifier `Have_Qsort_R`
importd.d(2):while evaluating: `static assert(Have_Qsort_R)`


importd.d
```
import gcc.config;
static assert(Have_Qsort_R);
```

imports/gcc/config.d
```
module gcc.config;
enum Have_Qsort_R = true;
```

gcc/config.h
```
#ifndef GCC_CONFIG_H
#define GCC_CONFIG_H
#ifdef GENERATOR_FILE
#error config.h is for the host, not build, machine.
#endif
#ifdef IN_GCC
# include "ansidecl.h"
#endif
#endif /* GCC_CONFIG_H */
```

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Walter Bright  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=23535

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Walter Bright  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=9816

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Walter Bright  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=4071

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Walter Bright  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=22367

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Richard Cattermole  changed:

   What|Removed |Added

   Keywords|bootcamp|
 CC||alphaglosi...@gmail.com

--- Comment #12 from Richard Cattermole  ---
I'm removing the bootcamp keyword from this bug.

Setting a symbol as export is not enough to get this working.

DllImport while in theory implemented in dmd as of 2.102.0 does not do any
patching at runtime required to dereference the pointer.

I went into attempting to solve this in my post here:
https://forum.dlang.org/post/ohflhdaggtapeqpyn...@forum.dlang.org

--

[Issue 9285] dtoh utility - convert D files to C++ header files

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=9285

Seb  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||greeen...@gmail.com
 Resolution|--- |FIXED

--- Comment #15 from Seb  ---
This has been added in 2.091:

https://dlang.org/changelog/2.091.0.html#headers

Issues with -HC should be opened as new issues.

--

Re: DPP: Linker issue with functions implemented in C header files

[Petar via Digitalmars-d-learn]

On Tuesday, 18 February 2020 at 09:20:08 UTC, Andre Pany wrote:


Hi Petar,


Hi Andre, I'm happy to help :)


thank you very much for the explanation and the code sample.
Filling the az_span anonymous member was the tricky part,
I thought it would be not possible to do so, but you showed me
the trick.


I wouldn't call it a trick, I was using standard struct literal 
initialization (the very syntax that DIP1031 proposes to 
deprecate).


For example:

struct Inner { int x, y; }
struct Outer { Inner inner; }

// You can initialize Outer in various ways:

// 1)
auto o1 = Outer(Inner(1, 2));

// 2)
Outer o2 = { inner: Inner(1, 2) };

// 3)
Outer o3 = { Inner(1, 2) };

// 4)
Outer o4 = { inner: { x: 1, y: 2} };

// 5)
Outer o5 = { { x: 1, y: 2} };

// 6)
Outer o6;
o6.inner.x = 1;
o6.inner.y = 1;

For POD (plain old data) struct like that, all six variants are 
equivalent (of course there more possible variations).


Since there's no `private` protection modifier in C, the only 
thing C library authors can do is make it inconvenient to access 
struct fields (by prefixing them with underscores), but they 
can't really prevent it.


For example, without this syntax, in pure C you can initialize a 
span like this:

char my_string[] = "Hey";
az_span span;
span._internal.ptr = my_string;
span._internal.length = sizeof(my_string) - 1;
span._internal.capacity = sizeof(my_string) - 1;

And with almost the same syntax you can do this in D:

string my_string = "Hey";
az_span span;
span._internal.ptr = cast(ubyte*)my_string.ptr; // note: I think 
this should be safe, because of [1]

span._internal.length = my_string.length;
span._internal.capacity = my_string.length;

It's just that that author wanted to prevent accidental bugs by 
pushing you to use the inline helper functions or macros (which 
are technically not needed).


[1]: 
https://github.com/Azure/azure-sdk-for-c/blob/25f8a0228e5f250c02e389f19d88c064c93959c1/sdk/core/core/inc/az_span.h#L22




I will do it like you have proposed but had also already created
a ticket for the Azure SDK developer:
https://github.com/Azure/azure-sdk-for-c/issues/359
There should be a more convenient way to fill a az_span 
structure.


To be honest, I don't think the authors will agree to change 
this, as putting inline functions in headers files is a pretty 
common practice in both C and C++.

There are two benefits to that:
1) Potentially better performance, because the code is easier to 
inline
2) It's possible to provide header-only libraries (not the case 
here), that don't require build steps.


For reference, here is my dockerfile which does the DPP call 
and linking:


Cool, I'll check it later!


``` dockerfile
FROM dlang2/ldc-ubuntu:1.20.0 as ldc

RUN apt-get install -y git libssl-dev uuid-dev 
libcurl4-openssl-dev curl


RUN curl -OL 
https://cmake.org/files/v3.12/cmake-3.12.4-Linux-x86_64.sh \

&& mkdir /opt/cmake \
&& sh /cmake-3.12.4-Linux-x86_64.sh --prefix=/opt/cmake 
--skip-license \

&& ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake

RUN git clone https://github.com/Azure/azure-sdk-for-c.git \
&& cd azure-sdk-for-c \
&& git submodule update --init --recursive

RUN cd azure-sdk-for-c \
&& mkdir build \
&& cd build \
&& cmake ../ \
&& make

RUN apt-get install -y clang-9 libclang-9-dev
RUN ln -s /usr/bin/clang-9 /usr/bin/clang
COPY az_storage_blobs.dpp /tmp/

RUN DFLAGS="-L=-L/usr/lib/llvm-9/lib/" dub run dpp -- --help

RUN DFLAGS="-L=-L/usr/lib/llvm-9/lib/" dub run dpp -- 
/tmp/az_storage_blobs.dpp \

--include-path /azure-sdk-for-c/sdk/core/core/inc \
--include-path /azure-sdk-for-c/sdk/core/core/internal \
--include-path /azure-sdk-for-c/sdk/storage/blobs/inc \
--include-path 
/azure-sdk-for-c/sdk/transport_policies/curl/inc \

--preprocess-only

ADD blobs_client_example.d /tmp/blobs_client_example.d
RUN  ldc2 /tmp/blobs_client_example.d /tmp/az_storage_blobs.d \
/azure-sdk-for-c/build/sdk/core/core/libaz_core.a \

/azure-sdk-for-c/build/sdk/storage/blobs/libaz_storage_blobs.a \

/azure-sdk-for-c/build/sdk/transport_policies/curl/libaz_curl.a 
\

-of=/tmp/app
```

Kind regards
André


Cheers,
Petar

Re: DPP: Linker issue with functions implemented in C header files

[Andre Pany via Digitalmars-d-learn]

On Tuesday, 18 February 2020 at 08:32:47 UTC, Petar Kirov 
[ZombineDev] wrote:

On Tuesday, 18 February 2020 at 05:41:38 UTC, Andre Pany wrote:

Hi,

I try to get wrap the "Azure SDK for C" using DPP and have 
following issue.
Functions, which are actually implemented in C header files 
will cause

linker errors:

https://github.com/Azure/azure-sdk-for-c/blob/master/sdk/core/core/inc/az_span.h#L91

Example:
AZ_NODISCARD AZ_INLINE az_span az_span_init(uint8_t * ptr, 
int32_t length, int32_t capacity) {
  return (az_span){ ._internal = { .ptr = ptr, .length = 
length, .capacity = capacity, }, };

}

Error message:
/tmp/app.o:az_storage_blobs.d:function 
_D20blobs_client_example__T19AZ_SPAN_FROM_BUFFERTG4096hZQBdFNbQnZS16az_storage_blobs7az_span: error: undefined reference to 'az_span_init'


I do not know C well, is this the expected behavior and should 
I ask the Azure SDK developers to not implement functions 
within C header files?


Kind regards
André


I think the problem is that you haven't actually linked in the 
Azure SDK C library.


Dpp translates the header declarations from C to D, but the 
actual definitions (function bodies) are not part of the 
process. The executable code for the function definitions 
should be inside either a static or dynamic library provided by 
the SDK.



From the the project's readme file, it looks like they're using 
CMake as the build system generator (afterwards both make and 
ninja should be valid choices for building):


mkdir build
cd build
cmake ../
make

In cases like this, it's best to checkout the CMakeLists.txt 
files of the individual sub project, like this one:


https://github.com/Azure/azure-sdk-for-c/blob/master/sdk/core/core/CMakeLists.txt

As you can see, there are several outputs of the build process, 
among which:


- add_library(az_core ...)
This defines a library named az_core which can produce either a 
static (.a on Linux, .lib on Windows) or dynamic library file 
(.so on Linux, .dll on Windows). (If no configuration is 
specified, I think it's static by default).

So the final file name would be libaz_core.{a,so} on Linux.
For the .c files to be built, a list of include directories 
must be specified, where the various .h would located 
(containing function and type declarations). This done like so:

target_include_directories (az_core PUBLIC ...)
The 'PUBLIC' argument to the target_include_directories 
specifies that if you want to use the library, you need to use 
the same include directories, as those needed for building it.


- add_executable(az_core_test ..)
This defines an executable build output, which looks is only 
used for testing, so it's not interesting to us, except that it 
can serve as an example app using the az_core library.


---

So in summary, if you want to use the az_core library, you need 
to:

1. Build it
2. Run Dpp like so:

d++ \
  --include-path target_include_directories>

  


You will need to repeat the same steps for any other part of 
the Azure C SDK.





TL;DR
After I went through all those steps I got a similar linker 
error for az_http_response_init. After looking where is the 
actual function definition, it turned out that it's not defined 
in a .c file, but it is an inline function part of a header 
file.
Searching for az_span_init revealed the same (I could have 
saved myself some time by reading your message more carefully 
:D).


So, to answer your original question, the problem is that dpp 
translates only declarations, not function definitions (such as 
inline functions like that).


For now, your best course of action is to translate all inline 
function definition by hand. Since in C inline functions are 
mostly short and simple functions (a better alternative to 
macros), hopefully that won't be too much work.


Also, looking at macros like AZ_SPAN_FROM_STR, there's really 
very little chance that they could be correctly translated 
automatically. As the things they do are likely not valid even 
in @system D code (without additional casts), so it's better to 
write your own D functions by hand anyway.



Here's what I tried:

test.dpp:

#include 
#include 

import std.stdio;

void main()
{
char[] resp =
"HTTP/1.2 404 We removed the\tpage!\r\n" ~
"\r\n" ~
"But there is somebody. :-)".dup;
az_span response_span =
{{
ptr: cast(ubyte*)resp.ptr,
length: cast(int)resp.length,
capacity: cast(int)resp.length
}};
az_http_response response;
az_result result = az_http_response_init(
, response_span);

writeln(result);
}

d++ --compiler ldmd2 --include-path ./inc test.dpp 
./build/libaz_core.a


Hi Petar,

thank you very much for the explanation and the code sample.
Filling the az_span anonymous member was the tricky part,
I thought it would be not possible to do so, but you showed me
the trick.

I will do it like you have proposed but had also already created
a ticket for 

Re: DPP: Linker issue with functions implemented in C header files

[Petar via Digitalmars-d-learn]

On Tuesday, 18 February 2020 at 05:41:38 UTC, Andre Pany wrote:

Hi,

I try to get wrap the "Azure SDK for C" using DPP and have 
following issue.
Functions, which are actually implemented in C header files 
will cause

linker errors:

https://github.com/Azure/azure-sdk-for-c/blob/master/sdk/core/core/inc/az_span.h#L91

Example:
AZ_NODISCARD AZ_INLINE az_span az_span_init(uint8_t * ptr, 
int32_t length, int32_t capacity) {
  return (az_span){ ._internal = { .ptr = ptr, .length = 
length, .capacity = capacity, }, };

}

Error message:
/tmp/app.o:az_storage_blobs.d:function 
_D20blobs_client_example__T19AZ_SPAN_FROM_BUFFERTG4096hZQBdFNbQnZS16az_storage_blobs7az_span: error: undefined reference to 'az_span_init'


I do not know C well, is this the expected behavior and should 
I ask the Azure SDK developers to not implement functions 
within C header files?


Kind regards
André


I think the problem is that you haven't actually linked in the 
Azure SDK C library.


Dpp translates the header declarations from C to D, but the 
actual definitions (function bodies) are not part of the process. 
The executable code for the function definitions should be inside 
either a static or dynamic library provided by the SDK.



From the the project's readme file, it looks like they're using 
CMake as the build system generator (afterwards both make and 
ninja should be valid choices for building):


mkdir build
cd build
cmake ../
make

In cases like this, it's best to checkout the CMakeLists.txt 
files of the individual sub project, like this one:


https://github.com/Azure/azure-sdk-for-c/blob/master/sdk/core/core/CMakeLists.txt

As you can see, there are several outputs of the build process, 
among which:


- add_library(az_core ...)
This defines a library named az_core which can produce either a 
static (.a on Linux, .lib on Windows) or dynamic library file 
(.so on Linux, .dll on Windows). (If no configuration is 
specified, I think it's static by default).

So the final file name would be libaz_core.{a,so} on Linux.
For the .c files to be built, a list of include directories must 
be specified, where the various .h would located (containing 
function and type declarations). This done like so:

target_include_directories (az_core PUBLIC ...)
The 'PUBLIC' argument to the target_include_directories specifies 
that if you want to use the library, you need to use the same 
include directories, as those needed for building it.


- add_executable(az_core_test ..)
This defines an executable build output, which looks is only used 
for testing, so it's not interesting to us, except that it can 
serve as an example app using the az_core library.


---

So in summary, if you want to use the az_core library, you need 
to:

1. Build it
2. Run Dpp like so:

d++ \
  --include-path target_include_directories>

  


You will need to repeat the same steps for any other part of the 
Azure C SDK.





TL;DR
After I went through all those steps I got a similar linker error 
for az_http_response_init. After looking where is the actual 
function definition, it turned out that it's not defined in a .c 
file, but it is an inline function part of a header file.
Searching for az_span_init revealed the same (I could have saved 
myself some time by reading your message more carefully :D).


So, to answer your original question, the problem is that dpp 
translates only declarations, not function definitions (such as 
inline functions like that).


For now, your best course of action is to translate all inline 
function definition by hand. Since in C inline functions are 
mostly short and simple functions (a better alternative to 
macros), hopefully that won't be too much work.


Also, looking at macros like AZ_SPAN_FROM_STR, there's really 
very little chance that they could be correctly translated 
automatically. As the things they do are likely not valid even in 
@system D code (without additional casts), so it's better to 
write your own D functions by hand anyway.



Here's what I tried:

test.dpp:

#include 
#include 

import std.stdio;

void main()
{
char[] resp =
"HTTP/1.2 404 We removed the\tpage!\r\n" ~
"\r\n" ~
"But there is somebody. :-)".dup;
az_span response_span =
{{
ptr: cast(ubyte*)resp.ptr,
length: cast(int)resp.length,
capacity: cast(int)resp.length
}};
az_http_response response;
az_result result = az_http_response_init(
, response_span);

writeln(result);
}

d++ --compiler ldmd2 --include-path ./inc test.dpp 
./build/libaz_core.a

DPP: Linker issue with functions implemented in C header files

[Andre Pany via Digitalmars-d-learn]

Hi,

I try to get wrap the "Azure SDK for C" using DPP and have 
following issue.
Functions, which are actually implemented in C header files will 
cause

linker errors:

https://github.com/Azure/azure-sdk-for-c/blob/master/sdk/core/core/inc/az_span.h#L91

Example:
AZ_NODISCARD AZ_INLINE az_span az_span_init(uint8_t * ptr, 
int32_t length, int32_t capacity) {
  return (az_span){ ._internal = { .ptr = ptr, .length = length, 
.capacity = capacity, }, };

}

Error message:
/tmp/app.o:az_storage_blobs.d:function 
_D20blobs_client_example__T19AZ_SPAN_FROM_BUFFERTG4096hZQBdFNbQnZS16az_storage_blobs7az_span: error: undefined reference to 'az_span_init'


I do not know C well, is this the expected behavior and should I 
ask the Azure SDK developers to not implement functions within C 
header files?


Kind regards
André

Re: Building GDC with auto-generated header files

[Johannes Pfau via Digitalmars-d-learn]

Am Tue, 30 Jul 2019 15:19:44 +1200 schrieb rikki cattermole:

> On 30/07/2019 4:11 AM, Eduard Staniloiu wrote:
>> Cheers, everybody
>> 
>> I'm working on this as part of my GSoC project [0].
>> 
>> I'm working on building gdc with the auto-generated `frontend.h` [1],
>> but I'm having some issues
>> 
>> There are functions in dmd that don't have an `extern (C)` or `extern
>> (C++)` but they are used by gdc (are exposed in `.h` files)
>> 
>> An example of such a function is `checkNonAssignmentArrayOp`[2] from
>> `dmd/arrayop.d` which is can be found in `gcc/d/dmd/expression.h` [3]
> 
> It may have previously been extern(C) or its a gdc specific patch.
> Either way PR please.

Actually the code at https://github.com/gcc-mirror/gcc/blob/master/gcc/d/
dmd is still the C++ frontend. The DMD frontend in upstream master 
(https://github.com/dlang/dmd/blob/master/) and in GCC master are very 
different versions, so mismatches are expected.

The latest DDMD GDC is here: https://github.com/gcc-mirror/gcc/commits/
ibuclaw/gdc However, it's still not a good idea to mix and match files 
from DMD upstream master and that GDC branch, as they will not be 100% in 
sync. It's best to simply use only files from the gcc/d repo, as that's 
what's used when compiling GDC.

You could also have a look at the gcc/d/dmd/MERGE file, which will tell 
you what upstream DMD commit has been used in the respective GDC tree.

-- 
Johannes

Re: Building GDC with auto-generated header files

[rikki cattermole via Digitalmars-d-learn]

On 30/07/2019 4:11 AM, Eduard Staniloiu wrote:

Cheers, everybody

I'm working on this as part of my GSoC project [0].

I'm working on building gdc with the auto-generated `frontend.h` [1], 
but I'm having some issues


There are functions in dmd that don't have an `extern (C)` or `extern 
(C++)` but they are used by gdc (are exposed in `.h` files)


An example of such a function is `checkNonAssignmentArrayOp`[2] from 
`dmd/arrayop.d` which is can be found in `gcc/d/dmd/expression.h` [3]


It may have previously been extern(C) or its a gdc specific patch.
Either way PR please.

Building GDC with auto-generated header files

[Eduard Staniloiu via Digitalmars-d-learn]

Cheers, everybody

I'm working on this as part of my GSoC project [0].

I'm working on building gdc with the auto-generated `frontend.h` 
[1], but I'm having some issues


There are functions in dmd that don't have an `extern (C)` or 
`extern (C++)` but they are used by gdc (are exposed in `.h` 
files)


An example of such a function is `checkNonAssignmentArrayOp`[2] 
from `dmd/arrayop.d` which is can be found in 
`gcc/d/dmd/expression.h` [3]


How does the linker find the right match in dmd? Since the 
function is `extern (D)`, isn't it mangled differently than C++?


[0] - 
https://forum.dlang.org/thread/djurwumzfrrttvtdg...@forum.dlang.org

[1] - https://github.com/dlang/dmd/pull/9971
[2] - 
https://github.com/dlang/dmd/blob/master/src/dmd/arrayop.d#L85
[3] - 
https://github.com/gcc-mirror/gcc/blob/master/gcc/d/dmd/expression.h#L84

Re: Interfacing with C libs: weeding through C/C++ macros and such in header files

[Mike Parker via Digitalmars-d-learn]

On Sunday, 13 January 2019 at 22:40:57 UTC, Alec Stewart wrote:



Example without code; for some reason a macro is defined for 
the stdlib functions `malloc`, `realloc`, and `free`. Maybe 
it's just because I don't have any pro experience with C or 
C++, but that seems a bit excessive. Or I could just be dumb.


Generally this is done to allow the compile-time configuration of 
memory allcoators. Back in my C days I had a little memory module 
that tracked total allocated and unallocated bytes and logged a 
few stats to a file at shutdown. I used macros to switch between 
it and the default stdlib calls.



I understand what it's doing, but do I really any of this with 
D?


No.


And then there's this inline function

#define RS_DATA_SIZE(f, s, input)
  \
do {
   \
if (rs_is_heap(input))  \
f(s, input->heap.buffer, rs_heap_len(input));   \
else\
f(s, input->stack.buffer, rs_stack_len(input)); \
} while (0)



Generally, this sort of thing can be moved into a D function or 
template if it's repeated in several places. Without the 
do...while, of course. And in case you're unfamiliar with its 
purpose here:


https://stackoverflow.com/questions/154136/why-use-apparently-meaningless-do-while-and-if-else-statements-in-macros

Re: Interfacing with C libs: weeding through C/C++ macros and such in header files

[Alec Stewart via Digitalmars-d-learn]

On Sunday, 13 January 2019 at 23:23:50 UTC, Alex wrote:

These three are members of the standard library in D.
https://dlang.org/phobos/core_memory.html




Ah, yea that's way easier.

At first, I would suggest to try out some automatic converters, 
which are written by the community:

https://wiki.dlang.org/Bindings#Binding_generators
especially dstep and dpp




That would make it easier because there's a lot of preprocessor 
stuff that ends up being circular references if I use `enum` or 
`const`.


Also, if there is a possibility to compile the C/C++ library, 
you could provide the interface only to the functions you need. 
Then, you reuse the existent code directly and interface the 
library by declaring the interfaces as extern in your D sources.

https://dlang.org/spec/attribute.html#linkage



That would also be easier. :P Thanks!

Re: Interfacing with C libs: weeding through C/C++ macros and such in header files

[Alex via Digitalmars-d-learn]

On Sunday, 13 January 2019 at 22:40:57 UTC, Alec Stewart wrote:
Example without code; for some reason a macro is defined for 
the stdlib functions `malloc`, `realloc`, and `free`. Maybe 
it's just because I don't have any pro experience with C or 
C++, but that seems a bit excessive. Or I could just be dumb.




These three are members of the standard library in D.
https://dlang.org/phobos/core_memory.html

Example with code (because I need help figuring out how whether 
I even need this or not):


#ifndef RS_API
#ifdef RS_NOINLINE
/* GCC version 3.1 required for the no inline attribute. */
#if RS_GCC_VERSION > 30100
#define RS_API static __attribute__((noinline))
#elif defined(_MSC_VER)
#define RS_API static __declspec(noinline)
#else
#define RS_API static
#endif
#elif RS_C99
#define RS_API static inline
#elif defined(__GNUC__)
#define RS_API static __inline__
#elif defined(_MSC_VER)
#define RS_API static __forceinline
#else
#define RS_API static
#endif
#endif

I understand what it's doing, but do I really any of this with 
D? And then there's this inline function


#define RS_DATA_SIZE(f, s, input)
  \
do {
   \
if (rs_is_heap(input))  \
f(s, input->heap.buffer, rs_heap_len(input));   \
else\
f(s, input->stack.buffer, rs_stack_len(input)); \
} while (0)

so yea. There's a little over 300 lines of preprocessor stuff. 
My question is how you all determine what to carry over from C 
libs in terms of preprocessor stuff. I imagine most useful 
values everyone just makes an enum for, and structs and unions 
are kept.


Thanks for any help you give!


At first, I would suggest to try out some automatic converters, 
which are written by the community:

https://wiki.dlang.org/Bindings#Binding_generators
especially dstep and dpp

I had some ambivalent experience with the procedure of converting 
C --> D, but there is C code out there, which behaves very well 
in this respect... So, generally, I pursued the tactics of

automatic conversion
trying to compile
rewriting missing parts.

Also, if there is a possibility to compile the C/C++ library, you 
could provide the interface only to the functions you need. Then, 
you reuse the existent code directly and interface the library by 
declaring the interfaces as extern in your D sources.

https://dlang.org/spec/attribute.html#linkage

Interfacing with C libs: weeding through C/C++ macros and such in header files

[Alec Stewart via Digitalmars-d-learn]

Hello all!

So while I have a decent grasp on D, I've been having trouble 
figuring out specific projects that I could do in D, so I thought 
I'd maybe find a little C or C++ library I could transfer over to 
D. I decided to make my life easier and look for something that's 
just a single header file, and while that does make things easier 
there are a TON of macros and inline functions that it almost 
seems ridiculous and it's somewhat overwhelming.


Example without code; for some reason a macro is defined for the 
stdlib functions `malloc`, `realloc`, and `free`. Maybe it's just 
because I don't have any pro experience with C or C++, but that 
seems a bit excessive. Or I could just be dumb.


Example with code (because I need help figuring out how whether I 
even need this or not):


#ifndef RS_API
#ifdef RS_NOINLINE
/* GCC version 3.1 required for the no inline attribute. */
#if RS_GCC_VERSION > 30100
#define RS_API static __attribute__((noinline))
#elif defined(_MSC_VER)
#define RS_API static __declspec(noinline)
#else
#define RS_API static
#endif
#elif RS_C99
#define RS_API static inline
#elif defined(__GNUC__)
#define RS_API static __inline__
#elif defined(_MSC_VER)
#define RS_API static __forceinline
#else
#define RS_API static
#endif
#endif

I understand what it's doing, but do I really any of this with D? 
And then there's this inline function


#define RS_DATA_SIZE(f, s, input) 
  \
	do { 
   \

if (rs_is_heap(input))  \
f(s, input->heap.buffer, rs_heap_len(input));   \
else\
f(s, input->stack.buffer, rs_stack_len(input)); \
} while (0)

so yea. There's a little over 300 lines of preprocessor stuff. My 
question is how you all determine what to carry over from C libs 
in terms of preprocessor stuff. I imagine most useful values 
everyone just makes an enum for, and structs and unions are kept.


Thanks for any help you give!

[Issue 18589] New: Windows header files: bcrypt and ncrypt

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18589

  Issue ID: 18589
   Summary: Windows header files: bcrypt and ncrypt
   Product: D
   Version: D2
  Hardware: All
OS: Windows
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: druntime
  Assignee: nob...@puremagic.com
  Reporter: an...@s-e-a-p.de

Could you add the windows header files for bcrypt and ncrypt to
src\druntime\src\core\sys\windows?

Thanks a lot.

--

[Issue 18363] New: we should autogenerate duplicate “.h” header files in dmd to keep them in sync

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18363

  Issue ID: 18363
   Summary: we should autogenerate duplicate “.h” header files in
dmd to keep them in sync
   Product: D
   Version: D2
  Hardware: x86
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: dmd
  Assignee: nob...@puremagic.com
  Reporter: timothee.co...@gmail.com

eg src/dmd/init.d vs src/dmd/init.h

* dtoh was reverted (https://github.com/dlang/tools/pull/125)

related: https://issues.dlang.org/show_bug.cgi?id=9285 Issue 9285 - dtoh
utility - convert D files to C++ header files (but that was not specific to
header files in dmd sources)

* https://github.com/dlang/dmd/pull/5082 [WIP] Generate C++ header files from
public extern (C++) declarations 

that was started on sept 2015

--

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=6019

Andrei Alexandrescu  changed:

   What|Removed |Added

   Keywords||bootcamp
 CC||and...@erdani.com

--

Re: A case for -H header files, avoid repeating expensive compile-time calculations

[rikki cattermole via Digitalmars-d]

On 28/08/2016 6:34 AM, cy wrote:

I made a module that looped for a while in compile time (since you can't
sleep during compile time), to see if I could pre-compile the module,
and thus save time compiling the main application. It didn't work at
first, because any file that depended on the module would import it, and
importing it would execute the compile time code again, thus taking a
long time to compile... twice.

But then I remembered about the rarely used -H option for dmd. -H
outputs a stripped down version of the source file being compiled, with
all the function definitions replaced with declarations. So if your
expensive to compile code is encapsulated within a function... it
doesn't appear in the -H output.

So uh... git://critter.cloudns.org/test-dlang-encapsulation I guess. It
might be a neat trick to speed up compilation, if you have a lot of
complicated code that can be hidden behind a simple interface.


Sounds like a use case for creating a source file in a pregeneration step.

A case for -H header files, avoid repeating expensive compile-time calculations

[cy via Digitalmars-d]

I made a module that looped for a while in compile time (since 
you can't sleep during compile time), to see if I could 
pre-compile the module, and thus save time compiling the main 
application. It didn't work at first, because any file that 
depended on the module would import it, and importing it would 
execute the compile time code again, thus taking a long time to 
compile... twice.


But then I remembered about the rarely used -H option for dmd. -H 
outputs a stripped down version of the source file being 
compiled, with all the function definitions replaced with 
declarations. So if your expensive to compile code is 
encapsulated within a function... it doesn't appear in the -H 
output.


So uh... git://critter.cloudns.org/test-dlang-encapsulation I 
guess. It might be a neat trick to speed up compilation, if you 
have a lot of complicated code that can be hidden behind a simple 
interface.

Re: noob in c macro preprocessor hell converting gsl library header files

[John Colvin via Digitalmars-d-learn]

On Wednesday, 6 January 2016 at 13:36:03 UTC, data pulverizer 
wrote:
I have been converting C numeric libraries and depositing them 
here: https://github.com/dataPulverizer. So far I have glpk and 
nlopt converted on a like for like c function basics. I am now 
stuck on the gsl library, primarily because of the preprocessor 
c code which I am very new to. The following few are 
particularly baffling to me:


#define INLINE_FUN extern inline // used in gsl_pow_int.h: 
INLINE_FUN double gsl_pow_2(const double x) { return x*x;   }


Could I just ignore the INLINE_FUN and use alias for function 
pointer declaration? For example ...


alias gsl_pow_2 = double gsl_pow_2(const(double) x);


Yes, you should be able to ignore INLINE_FUN

double gsl_pow_2(const double x);

is the correct declaration.

#define INLINE_DECL // used in interpolation/gsl_interp.h: 
INLINE_DECL size_t gsl_interp_bsearch(const double x_array[], 
double x, size_t index_lo, size_t index_hi);


I would guess the same as for INLINE_FUN?


yes

#define GSL_VAR extern // used in rng/gsl_rng.h:GSL_VAR const 
gsl_rng_type *gsl_rng_borosh13;


perhaps GSL_VAR can be ignored and I could use:

gsl_rng_borosh13 const(gsl_rng_type)*;


It should be

extern gsl_rng_type* gsl_rng_borosh13;

I have been using these kind of fixes and have not been able to 
get the rng module to recognise the ported functions, meaning 
that something has been lost in translation.


I am currently getting the following error:

$ gsl_rng_print_state

rng_example.o: In function `_Dmain':
rng_example.d:(.text._Dmain+0x13): undefined reference to 
`gsl_rng_print_state'

collect2: error: ld returned 1 exit status

I can't seem to call any of the functions but the types are 
recognized.


Thanks in advance


I think you might have some confusion between function 
declarations:


T myFunction(Q myArg);

function pointer type declarations:

alias MyFunctionPointerType = T function(Q myArg);

and function pointer declarations:

MyFunctionPointerType myFunctionPointer;

noob in c macro preprocessor hell converting gsl library header files

[data pulverizer via Digitalmars-d-learn]

I have been converting C numeric libraries and depositing them 
here: https://github.com/dataPulverizer. So far I have glpk and 
nlopt converted on a like for like c function basics. I am now 
stuck on the gsl library, primarily because of the preprocessor c 
code which I am very new to. The following few are particularly 
baffling to me:


#define INLINE_FUN extern inline // used in gsl_pow_int.h: 
INLINE_FUN double gsl_pow_2(const double x) { return x*x;   }


Could I just ignore the INLINE_FUN and use alias for function 
pointer declaration? For example ...


alias gsl_pow_2 = double gsl_pow_2(const(double) x);

#define INLINE_DECL // used in interpolation/gsl_interp.h: 
INLINE_DECL size_t gsl_interp_bsearch(const double x_array[], 
double x, size_t index_lo, size_t index_hi);


I would guess the same as for INLINE_FUN?

#define GSL_VAR extern // used in rng/gsl_rng.h:GSL_VAR const 
gsl_rng_type *gsl_rng_borosh13;


perhaps GSL_VAR can be ignored and I could use:

gsl_rng_borosh13 const(gsl_rng_type)*;

I have been using these kind of fixes and have not been able to 
get the rng module to recognise the ported functions, meaning 
that something has been lost in translation.


I am currently getting the following error:

$ gsl_rng_print_state

rng_example.o: In function `_Dmain':
rng_example.d:(.text._Dmain+0x13): undefined reference to 
`gsl_rng_print_state'

collect2: error: ld returned 1 exit status

I can't seem to call any of the functions but the types are 
recognized.


Thanks in advance

Re: noob in c macro preprocessor hell converting gsl library header files

[data pulverizer via Digitalmars-d-learn]

On Wednesday, 6 January 2016 at 13:59:44 UTC, John Colvin wrote:
#define INLINE_FUN extern inline // used in gsl_pow_int.h: 
INLINE_FUN double gsl_pow_2(const double x) { return x*x;   }


Could I just ignore the INLINE_FUN and use alias for function 
pointer declaration? For example ...


alias gsl_pow_2 = double gsl_pow_2(const(double) x);


Yes, you should be able to ignore INLINE_FUN

double gsl_pow_2(const double x);

is the correct declaration.



#define GSL_VAR extern // used in rng/gsl_rng.h:GSL_VAR const 
gsl_rng_type *gsl_rng_borosh13;


perhaps GSL_VAR can be ignored and I could use:

gsl_rng_borosh13 const(gsl_rng_type)*;


It should be

extern gsl_rng_type* gsl_rng_borosh13;


I see. Thanks.

I think you might have some confusion between function 
declarations:


T myFunction(Q myArg);

function pointer type declarations:

alias MyFunctionPointerType = T function(Q myArg);

and function pointer declarations:

MyFunctionPointerType myFunctionPointer;


Sorry in my haste to describe what I was doing I wrote down a 
function pointer instead of a function - my original code was a 
function. Your suggestion of looking at the 
https://github.com/abrown25/gsld library is a good call. I'll 
probably end up sending a pull request to that library after 
using it as a basic outline of how to deal with these 
preprocessors.

[Issue 3306] bad function/delegate literal generated into header files

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=3306

Andrei Alexandrescu and...@erdani.com changed:

   What|Removed |Added

Version|2.032   |D2

--

Re: Header Files

[Marco Leise via Digitalmars-d]

Am Sun, 10 May 2015 10:37:13 -0400
schrieb bitwise bitwise@gmail.com:

 I'm not sure I understand what you're trying to say.
 
Bit

I'm trying to hijack your thread and communicate that .di
files are important for D since they allow the compiler to
stop recursively importing modules at some point rather than
eagerly importing until the world is in compiler memory.

-- 
Marco

Re: Header Files

[weaselcat via Digitalmars-d]

On Monday, 11 May 2015 at 12:46:59 UTC, Marco Leise wrote:

Am Sun, 10 May 2015 10:37:13 -0400
schrieb bitwise bitwise@gmail.com:


I'm not sure I understand what you're trying to say.

   Bit


I'm trying to hijack your thread and communicate that .di
files are important for D since they allow the compiler to
stop recursively importing modules at some point rather than
eagerly importing until the world is in compiler memory.


Is this something that already works?

Re: Header Files

[Marco Leise via Digitalmars-d]

Am Mon, 11 May 2015 12:50:48 +
schrieb weaselcat weasel...@gmail.com:

 On Monday, 11 May 2015 at 12:46:59 UTC, Marco Leise wrote:
  Am Sun, 10 May 2015 10:37:13 -0400
  schrieb bitwise bitwise@gmail.com:
 
  I'm not sure I understand what you're trying to say.
  
 Bit
 
  I'm trying to hijack your thread and communicate that .di
  files are important for D since they allow the compiler to
  stop recursively importing modules at some point rather than
  eagerly importing until the world is in compiler memory.
 
 Is this something that already works?

The idea is very simple. You have some .d file like this:

import a;
import internal;
import std.algorithm;

SomeStruct foo()
{
return SomeStruct(min(1, 2));
}

The compiler loads and analyzes 'a', 'internal',
'std.algorithm' and publically imported modules. If there are
multiple declarations of SomeStruct it errors out. If
SomeStruct is only defined in 'a', a .di file would reduce to
this:

import a;

SomeStruct foo();

So we remove the dependency on that Phobos module
'std.algorithm' for 'min' and module 'internal'.

Ideally only the public API of a module remains in a .di file,
but we need to know more, like what size it is and if it has a
dtor. There's also compile-time reflection to iterate all the
(private and public) fields in a struct or class making it
harder to completely remove internal modules from the public
API:

import internal_types;

struct IUseInternalStructures
{
private:
SomeInternalType data;
uint handle;
float area;
}

Although for most purposes it is enough to know the .init of
this struct and that it has no dtor (not even one from
SomeInternalData), we cannot really remove the import here in
the .di file.

-- 
Marco

Re: Header Files

[Marco Leise via Digitalmars-d]

Am Sat, 09 May 2015 10:01:25 -0400
schrieb bitwise bitwise@gmail.com:

 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d
 
 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H
 
 This dumps all the *.di files into the output directory ignoring the  
 directory structure.
 
 Is there some rational for it being this way?
 Wouldn't it be much more useful if directories were preserved?
 
Bit

I agree, D modules are hierarchical and sometimes share the
same base name. Header files are very important to short
circuit the transitive import chain. We've already seen it in
Phobos. Many modules ends up pulling in most of Phobos, and
those are library modules.

Now imagine there was an operating system like Linux/GNU with
all libraries written in D and no .di files: The program entry
point in a project the size of OpenOffice would pull in so many
imports that no super computer in the world could compile it
given CTFE memory use: all of the project files, font library,
math library, image library, entire GUI toolkit, Java binding,
accessibility library, RPC library, configuration library,
OpenGL, audio, GhostScript and PDF, Python, XML, MS Office
file type readers, ICU, SSL, Phobos, X11 or similar, color
management, printing, zip, databases, ...

We often shun .di files, thinking of them as C++ relics, but
the big difference is that we don't write them. The compiler
generates them and they are still a key to a successful
eco-system with huge projects, just the way they are in C/C++.

One thing I'm missing is a way to tell the compiler or a build
tool which symbols or modules are part of the library API.
There is no sense in generating an internal.di and similar
files that are never supposed to be used by the outside.
(They'd also be listed unnecessarily in import
auto-completions of Mono-D and similar IDEs.)

-- 
Marco

Re: Header Files

[bitwise via Digitalmars-d]

On Sunday, 10 May 2015 at 01:00:31 UTC, bitwise wrote:
Is there really no way to preserve the directory structure when 
creating .di files?

  Bit


Why hello, Bitwise! I believe the '-op' flag is what you're 
looking for!


  Bitwiser

Re: Header Files

[bitwise via Digitalmars-d]

On Sun, 10 May 2015 04:20:45 -0400, Marco Leise marco.le...@gmx.de wrote:

I agree, D modules are hierarchical and sometimes share the
same base name. Header files are very important to short
circuit the transitive import chain. We've already seen it in
Phobos. Many modules ends up pulling in most of Phobos, and
those are library modules.

Now imagine there was an operating system like Linux/GNU with
all libraries written in D and no .di files: The program entry
point in a project the size of OpenOffice would pull in so many
imports that no super computer in the world could compile it
given CTFE memory use: all of the project files, font library,
math library, image library, entire GUI toolkit, Java binding,
accessibility library, RPC library, configuration library,
OpenGL, audio, GhostScript and PDF, Python, XML, MS Office
file type readers, ICU, SSL, Phobos, X11 or similar, color
management, printing, zip, databases, ...

We often shun .di files, thinking of them as C++ relics, but
the big difference is that we don't write them. The compiler
generates them and they are still a key to a successful
eco-system with huge projects, just the way they are in C/C++.

One thing I'm missing is a way to tell the compiler or a build
tool which symbols or modules are part of the library API.
There is no sense in generating an internal.di and similar
files that are never supposed to be used by the outside.
(They'd also be listed unnecessarily in import
auto-completions of Mono-D and similar IDEs.)


I'm not sure I understand what you're trying to say.

  Bit

Re: Header Files

[Ali Çehreli via Digitalmars-d]

On 05/10/2015 09:11 AM, bitwise wrote:

 On Sunday, 10 May 2015 at 01:00:31 UTC, bitwise wrote:
 Is there really no way to preserve the directory structure when
 creating .di files?
   Bit

 Why hello, Bitwise! I believe the '-op' flag is what you're looking for!

Bitwiser

Wow! Walter has already implemented it by going back in time. :)

Ali

Re: Header Files

[bitwise via Digitalmars-d]

On Sat, 09 May 2015 21:09:33 -0400, Ali Çehreli acehr...@yahoo.com wrote:


On 05/09/2015 07:01 AM, bitwise wrote:

./main.d
./pack/foo.d
./pack/sub/bar.d

dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

This dumps all the *.di files into the output directory ignoring the
directory structure.

Is there some rational for it being this way?
Wouldn't it be much more useful if directories were preserved?

   Bit


As far as I know, every other similar tool works the same way:

   $ gcc -c foo/foo.c

foo.o is outputted to the current directory.

The common solution that I know is to use makefiles to manage file  
dependencies. For example, the rule for .di specifies the file that it  
depends on (the .d corresponding to that .di) and 'make' handles it  
automatically.


Ali



I'm not sure I understand what you mean, but my point is, that as is, this  
feature of DMD is broken.


If people are successfully using dmd -H right now, they must not be using  
packages, or their 'package.di' files would be overwritten. So I'm  
wondering if I should do a PR to fix this.


  Bit

Re: Header Files

[Ali Çehreli via Digitalmars-d]

On 05/09/2015 07:01 AM, bitwise wrote:

./main.d
./pack/foo.d
./pack/sub/bar.d

dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

This dumps all the *.di files into the output directory ignoring the
directory structure.

Is there some rational for it being this way?
Wouldn't it be much more useful if directories were preserved?

   Bit


As far as I know, every other similar tool works the same way:

  $ gcc -c foo/foo.c

foo.o is outputted to the current directory.

The common solution that I know is to use makefiles to manage file 
dependencies. For example, the rule for .di specifies the file that it 
depends on (the .d corresponding to that .di) and 'make' handles it 
automatically.


Ali

Re: Header Files

[Ali Çehreli via Digitalmars-d]

On 05/09/2015 06:18 PM, bitwise wrote:

 I'm not sure I understand what you mean, but my point is, that as is,
 this feature of DMD is broken.

Arguably broken but compared to other tools it behaves in the same way.

 If people are successfully using dmd -H right now, they must not be
 using packages, or their 'package.di' files would be overwritten. So I'm
 wondering if I should do a PR to fix this.

The same can be said for gcc and other tools as well and it is not 
limited to package.d: any two files with the same name would have the 
same problem, meaning that the particular build system that puts those 
unrelated files in the same directory is in error, not the compiler.


The way I see it, the right solution is to provide correct paths to dmd 
for each file.


Ali

Re: Header Files

[bitwise via Digitalmars-d]

On Sat, 09 May 2015 21:31:20 -0400, Ali Çehreli acehr...@yahoo.com wrote:


On 05/09/2015 06:18 PM, bitwise wrote:

  I'm not sure I understand what you mean, but my point is, that as is,
  this feature of DMD is broken.

Arguably broken but compared to other tools it behaves in the same way.


dmd is the reference compiler, so the other compilers _should_ behave the  
same way. I suppose this could be considered arguable, because there may  
exist a build system(I don't know of any) that will plan for this defect  
and generate the headers one by one while building out the directories on  
it's own. So my options are to be burdened by some build tool I don't need  
just for this one feature, or waste my time writing custom build scripts  
when this is something that can trivially be implemented in the compiler.


In the interest of not breaking existing build systems, an additional flag  
could be added to dmd. Something like -Hp for preserve paths when  
outputting headers.


The way I see it, the right solution is to provide correct paths to dmd  
for each file.


Given a build command like the one below, I don't understand how you would  
achieve this without building the files one by one.



./main.d
./pack/foo.d
./pack/sub/bar.d
 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H


The output is:

./main.di
./foo.di
./bar.di

And it should be(or could be, if -Hp was passed)

./main.di
./pack/foo.di
./pack/sub/bar.di

  Bit

Re: Header Files

[bitwise via Digitalmars-d]

On Sat, 09 May 2015 10:01:25 -0400, bitwise bitwise@gmail.com wrote:


./main.d
./pack/foo.d
./pack/sub/bar.d

dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

This dumps all the *.di files into the output directory ignoring the  
directory structure.


Is there some rational for it being this way?
Wouldn't it be much more useful if directories were preserved?

   Bit



At this point, I'm thinking I should file a bug report for this.

You can't build a codebase with multiple packages because package.d or any  
other files that share the same filename will be overwritten.


Is there really no way to preserve the directory structure when creating  
.di files?


  Bit

Header Files

[bitwise via Digitalmars-d]

./main.d
./pack/foo.d
./pack/sub/bar.d

dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

This dumps all the *.di files into the output directory ignoring the  
directory structure.


Is there some rational for it being this way?
Wouldn't it be much more useful if directories were preserved?

  Bit

[Issue 9285] dtoh utility - convert D files to C++ header files

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=9285
Issue 9285 depends on issue 11620, which changed state.

Issue 11620 Summary: dmd json output should output enum values
https://issues.dlang.org/show_bug.cgi?id=11620

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--

[Issue 9285] dtoh utility - convert D files to C++ header files

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=9285

--- Comment #14 from Andrej Mitrovic andrej.mitrov...@gmail.com ---
https://issues.dlang.org/show_bug.cgi?id=11620 is now fixed, are there any
other fixes needed for this tool?

--

Some issues (bugs?) with generated header files

[Andre]

Hi,

i want to generate header (di) files for a library I developed
and faced some issues.
The project structure is:
source
-wba
--com
---defintions.d
---...
--dispatcher.d
--...
--package.d


MonoDevelop generated following statement for me:
C:\D\dmd2\windows\bin\dmd.exe -debug -gc source\wba\com\definitions.d 
source\wba\com\docHostUIHandler.d source\wba\com\oleClientSite.d 
source\wba\com\oleInPlaceFrame.d source\wba\com\oleInPlaceSite.d 
source\wba\com\storage.d source\wba\com\webBrowserEvents2.d 
source\wba\dispatcher.d source\wba\main.d 
source\wba\objectForScripting.d source\wba\package.d 
source\wba\wbWrapper.d source\wba\webbrowser.d 
source\wba\webBrowserForm.d -IC:\D\dmd2\src\druntime\import 
-IC:\D\dmd2\src\phobos -IC:\D\lib\WindowsAPI -lib -odobj\Header 
-ofJ:\Workspace\Libraries\WebBrowserApplication\bin\Header\wba.lib -H


3 issues:

1) While using the header files in another project, the package.di file 
does not work as expected. I cannot use statement: import wba;


main.d(3): Error: module wba is in file 'wba.d' which cannot be read
It only works if I rename package.di to package.d


2) property methods doesn't work with header files.
For this coding:
@property docHostUIHandler()
{
return this._docHostUIHandler;
}
This di coding was created:
@property docHostUIHandler();

While using in another project these errors are raised for
the line in the di coding:

Error: function declaration without return type. (Note that constructors 
are always named 'this')

Error: no identifier for declarator docHostUIHandler()


3) How can I specify the output folder for the di files
and also specify that the same folder structure is used as
described above (wba, wba/com)

Kind regards
André

Re: Some issues (bugs?) with generated header files

[evilrat]

On Saturday, 5 April 2014 at 10:00:13 UTC, Andre wrote:

Hi,

i want to generate header (di) files for a library I developed
and faced some issues.
The project structure is:
source
-wba
--com
---defintions.d
---...
--dispatcher.d
--...
--package.d


MonoDevelop generated following statement for me:
C:\D\dmd2\windows\bin\dmd.exe -debug -gc 
source\wba\com\definitions.d 
source\wba\com\docHostUIHandler.d 
source\wba\com\oleClientSite.d 
source\wba\com\oleInPlaceFrame.d 
source\wba\com\oleInPlaceSite.d source\wba\com\storage.d 
source\wba\com\webBrowserEvents2.d source\wba\dispatcher.d 
source\wba\main.d source\wba\objectForScripting.d 
source\wba\package.d source\wba\wbWrapper.d 
source\wba\webbrowser.d source\wba\webBrowserForm.d 
-IC:\D\dmd2\src\druntime\import -IC:\D\dmd2\src\phobos 
-IC:\D\lib\WindowsAPI -lib -odobj\Header 
-ofJ:\Workspace\Libraries\WebBrowserApplication\bin\Header\wba.lib 
-H


3 issues:

1) While using the header files in another project, the 
package.di file does not work as expected. I cannot use 
statement: import wba;


main.d(3): Error: module wba is in file 'wba.d' which cannot be 
read

It only works if I rename package.di to package.d


2) property methods doesn't work with header files.
For this coding:
@property docHostUIHandler()
{
return this._docHostUIHandler;
}
This di coding was created:
@property docHostUIHandler();

While using in another project these errors are raised for
the line in the di coding:

Error: function declaration without return type. (Note that 
constructors are always named 'this')

Error: no identifier for declarator docHostUIHandler()


3) How can I specify the output folder for the di files
and also specify that the same folder structure is used as
described above (wba, wba/com)

Kind regards
André


i have little info about this, but let me clear something for you.

- interface generation is outdated/broken
- .di files should be avoided now, this is why import looking for 
.d only, but idk why.
- to avoid @property issue you can try adding export 
specifier(export @property someProperty())


-Hd flag allows specify header location, -Hf specify name for 
example if you want to write them to directory named imports 
you call dmd myfile.d -Hdimports which save myfile.d to 
imports/myfile.di

Re: Some issues (bugs?) with generated header files

[evilrat]

On Saturday, 5 April 2014 at 10:00:13 UTC, Andre wrote:


2) property methods doesn't work with header files.
For this coding:
@property docHostUIHandler()
{
return this._docHostUIHandler;
}
This di coding was created:
@property docHostUIHandler();




the problem with this that your code doesnt specify neither auto 
nor other return type.


// not recommended, actual type unknown to users
@property auto docHostUIHandler()
{
 return this._docHostUIHandler;
}

// recommended, manual type
@property HostUIHandlerType()
{
 return this._docHostUIHandler;
}

p.s. language reference recommends also put underscore at end 
instead front, prepending underscore may be used by compiler 
generated stuff resulting in name clashes.

Re: Some issues (bugs?) with generated header files

[Andre]

Am 05.04.2014 12:49, schrieb evilrat:


i have little info about this, but let me clear something for you.

- interface generation is outdated/broken
- .di files should be avoided now, this is why import looking for .d
only, but idk why.
- to avoid @property issue you can try adding export specifier(export
@property someProperty())

-Hd flag allows specify header location, -Hf specify name for example if
you want to write them to directory named imports you call dmd
myfile.d -Hdimports which save myfile.d to imports/myfile.di



Thanks a lot for clarification.

Kind regards
André

Re: Some issues (bugs?) with generated header files

[evilrat]

On Saturday, 5 April 2014 at 10:55:19 UTC, evilrat wrote:

// recommended, manual type
@property HostUIHandlerType()
{
 return this._docHostUIHandler;
}


oops. this of course should be like normal function.

[Issue 9285] dtoh utility - convert D files to C++ header files

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=9285



--- Comment #13 from Adam D. Ruppe destructiona...@gmail.com 2014-02-10 
20:55:50 PST ---
I haven't linked this in here yet so here it is:

https://github.com/D-Programming-Language/tools/pull/39

Should be good enough to start using, then we can address further issues as
they arise.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

Re: What is the common way of porting opaque types in C header files?

[Gary Willoughby]

On Friday, 13 December 2013 at 01:20:41 UTC, Mike Parker wrote:

On 12/13/2013 7:52 AM, Gary Willoughby wrote:
I have a lot of opaque types in C headers i'm porting to D. 
What is the

best way to handle these? Would you just use void pointers?

In the below example Tcl_AsyncHandler_ is not defined anywhere.

C:
typedef struct Tcl_AsyncHandler_ *Tcl_AsyncHandler;

D:
alias void* Tcl_AsyncHandler;


D:
struct Tcl_AsyncHandler_;
alias Tcl_AsyncHandler = Tcl_AsyncHandler_*;


Ah right, thanks. I didn't know D allows such types.

What is the common way of porting opaque types in C header files?

[Gary Willoughby]

I have a lot of opaque types in C headers i'm porting to D. What 
is the best way to handle these? Would you just use void pointers?


In the below example Tcl_AsyncHandler_ is not defined anywhere.

C:
typedef struct Tcl_AsyncHandler_ *Tcl_AsyncHandler;

D:
alias void* Tcl_AsyncHandler;

Re: What is the common way of porting opaque types in C header files?

[Mike Parker]

On 12/13/2013 7:52 AM, Gary Willoughby wrote:

I have a lot of opaque types in C headers i'm porting to D. What is the
best way to handle these? Would you just use void pointers?

In the below example Tcl_AsyncHandler_ is not defined anywhere.

C:
typedef struct Tcl_AsyncHandler_ *Tcl_AsyncHandler;

D:
alias void* Tcl_AsyncHandler;


D:
struct Tcl_AsyncHandler_;
alias Tcl_AsyncHandler = Tcl_AsyncHandler_*;

[Issue 9285] dtoh utility - convert D files to C++ header files

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=9285


Martin Nowak c...@dawg.eu changed:

   What|Removed |Added

 CC||c...@dawg.eu


--- Comment #12 from Martin Nowak c...@dawg.eu 2013-12-04 13:36:50 PST ---
(In reply to comment #11)
 I cleaned up and updated my old code so it works with new dmd again:
 
 http://arsdnet.net/dcode/dtoh.zip
 
 There's still some fixmes in there i can fix, but if anyone gets a chance to
 take a look and let me know if it is ok so far, i'd appreciate it.

How about reopening a pull request to tools.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 9285] dtoh utility - convert D files to C++ header files

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=9285



--- Comment #10 from Adam D. Ruppe destructiona...@gmail.com 2013-11-27 
11:43:21 PST ---
Are there any more substantial comments on the code I wrote in January? I just
tried to use it again, and there's some small updates needed. dmd's json output
has changed, it outputs mangles instead of types now! Weird. Of course, the
substantial stuff still isn't there - I'm wondering if it wouldn't be better to
do this with traits instead of the json.

Right now, the way it works is you do

dmd -X file.d
dtocpp file.json

But, maybe we should do:

translator.d:
import dtocpp;
import module_you_want_to_convert;

mixin Convert!(module_you_want_to_convert);


dmd translator.d dtocpp.d needed_modules.d
./translator

idk, the json is close to good enough, there's just still things I wish it did
better. It really needs to list enum values, if nothing else, to make them
useful. What I did before with that was to just skip enums.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 9285] dtoh utility - convert D files to C++ header files

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=9285



--- Comment #11 from Adam D. Ruppe destructiona...@gmail.com 2013-11-27 
19:51:37 PST ---
I cleaned up and updated my old code so it works with new dmd again:

http://arsdnet.net/dcode/dtoh.zip

There's still some fixmes in there i can fix, but if anyone gets a chance to
take a look and let me know if it is ok so far, i'd appreciate it.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=6019



--- Comment #9 from Martin Nowak c...@dawg.eu 2013-10-02 04:42:10 PDT ---
*** Issue 9220 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 6019] Phobos imports in autogenerated .di header files break implicit linking with DLLs

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=6019



--- Comment #11 from Martin Nowak c...@dawg.eu 2013-10-02 08:51:30 PDT ---
(In reply to comment #10)
 How about this? Using weak undefined symbols to link against the imported
 ModuleInfo would result in null pointers during runtime.

That could cause some issues with the weird multilibs. Not sure currently if
every archive member references it's module's ModuleInfo.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-18 00:59, H. S. Teoh wrote:


IOW either you don't do it at all, or you have to go all the way and
implement a fully-functional C frontend?

If so, libclang is starting to sound rather attractive...


That's what I'm telling


Hmm. We *could* pre-preprocess the code to do this conversion first to
pick out these #define's, then suppress the #define's we understand from
the input to the C preprocessor. Something like this:

bool isSimpleValue(string s) {
// basically, return true if s is something compilable
// when put on the right side of enum x = 
}

auto pipe = spawnCPreprocessor();
string[string] manifestConstants;
foreach (line; inputFile.byLine()) {
if (auto m=match(line, `^\s*#define\s+(\w+)\s+(.*?)\s+`))
{
if (isSimpleValue(m.captures[2])) {
manifestConstants[m.captures[1]] =
m.captures[2];

// Suppress enums that we picked out
continue;
}
// whatever we don't understand, hand over to
// the C preprocessor
}
pipe.writeln(line);
}

Basically, whatever #define's we can understand, we handle, and anything
else we let the C preprocessor deal with. By suppressing the #define's
we've picked out, we force the C preprocessor to leave any reference to
them as unexpanded identifiers, so that later on we can just generate
the enums and the resulting code will match up correctly.


You will just end up needing to build a full C preprocessor. Just use an 
existing one, that is libclang.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-18 00:36, Walter Bright wrote:


You could, but then you are left with failing to recognize:

 #define FOO 3

and converting it to:

 enum FOO = 3;


And things like:

#if linux
short a;
#elif _WIN32 || _WIN64
int a;
#endif

Should preferably be converted to:

version (linux)
short a;

else version (Windows)
int a;

Other example:

#define foo(a, b) a + b

Should be converted to:

auto foo (A, B) (A a, B b) { return a + b; }

--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-17 21:40, Walter Bright wrote:


Yes, but the front end itself must be complete. Otherwise,
it's not really practical when you're dealing with things like the
preprocessor - because a non-compliant front end won't even know it has
gone off the rails.

There are other issues when dealing with C .h files:

1. there may be various #define's necessary to compile it that would
normally be supplied on the command line to the C compiler

2. there are various behavior switches (see the PR for DMD that wants to
set the signed'ness of char types)

3. rather few .h files seem to be standard compliant C. They always rely
on various compiler extensions

These problems are not insurmountable, they just are non-trivial and
need to be handled for a successful .h file importer.


Yes, I agree with all the above. That's why I'm using libclang. What I'm 
saying is that when I use a library like libclang I can choose quite 
freely what to convert and not convert. Example, DStep doesn't handle 
the preprocessor at all. But since libclang does, it can parse any 
header file anyway. What happens is just that the preprocessor 
declarations won't be translated and not end up in the translated file.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Paulo Pinto]

On Wednesday, 17 July 2013 at 15:34:54 UTC, Jacob Carlborg wrote:

On 2013-07-17 13:24, Paulo Pinto wrote:


Thus we are back to the compiler as library discussion.


Yes, but for the C family of languages we already have a 
compiler as library, that is Clang.


Agreed.

I also confess that my anti-C bias got a bit softened with clang.

It does not sort out all C and C++ issues in regard with safety, 
but it helps bringing to C a Pascal like safety when integrated 
with proper tooling.


Unfortunately when using C and C++, not all compilers are like 
clang and it is not always easy to convince people to add extra 
tooling (lint and friends).


--
Paulo

Re: Proof of concept: automatically import C header files

[deadalnix]

On Wednesday, 17 July 2013 at 05:12:00 UTC, Timothee Cour wrote:
what's a non-full C front end? If it's not a real C front end 
it's gonna

break with certain macros etc. Not good.



Macro are processed before parsing? No need for a full C frontend 
to handle macros.


I see no point in re-implementing a C front end when we can 
simply use an
existing one to do the job (eg clang). This would also allow to 
parse C++

just as well.


When you only need a very limited part of the fronted, it make 
sense. Here we don't need to parse function body for instance, 
and we can skip most of semantic analysis (if not all ?).

Re: Proof of concept: automatically import C header files

[angel]

Possibly instead of 'include' would be better use 'include_C' as 
opposed to C++ or any other language.

Re: Proof of concept: automatically import C header files

[Timothee Cour]

On Tue, Jul 16, 2013 at 11:01 PM, deadalnix deadal...@gmail.com wrote:

 On Wednesday, 17 July 2013 at 05:12:00 UTC, Timothee Cour wrote:

 what's a non-full C front end? If it's not a real C front end it's gonna
 break with certain macros etc. Not good.


 Macro are processed before parsing? No need for a full C frontend to
 handle macros.


  I see no point in re-implementing a C front end when we can simply use an
 existing one to do the job (eg clang). This would also allow to parse C++
 just as well.


 When you only need a very limited part of the fronted, it make sense. Here
 we don't need to parse function body for instance, and we can skip most of
 semantic analysis (if not all ?).


you'd still need to parse C files recursively (textual inclusion...),
handle different C function calling conventions, different C standards,
you'd need a way to forward to dmd different C compiler options (include
paths to standard / custom libraries), and eventually people will want to
parse C++ as well anyways. That can be a lot of work. Whereas using
existing tools takes much less effort and is less error prone.

Re: Proof of concept: automatically import C header files

[Chad Joan]

On Tuesday, 16 July 2013 at 14:15:55 UTC, Jacob Carlborg wrote:
Made a proof of concept to automatically parse, translate and 
import C header files in D using DStep. DMD is linked against 
DStep and does not start new process to make the translation.


I added a new pragma, include, that handles everything. Use 
like this:


// foo.h
void foo ();

// main.d

module main;

pragma(include, foo.h);

void main ()
{
foo();
}

DMD: https://github.com/jacob-carlborg/dmd/tree/dstep
DStep: https://github.com/jacob-carlborg/dstep/tree/c_api


This sounds pretty cool, and the suggestion from Timothee also 
makes a lot of sense.


Is there any way we can rig this to behave as if it were a CTFE 
invocation?  It could be treated like an intrinsic up to the 
point where we have powerful-enough CTFE to replace it.  I'm 
still not sure if Walter would be OK with this, but I figure I'd 
mention it, since it could give us something really nice without 
having to wait for CTFE to get good.

Re: Proof of concept: automatically import C header files

[Walter Bright]

On 7/16/2013 10:04 PM, deadalnix wrote:

On Wednesday, 17 July 2013 at 04:14:56 UTC, Walter Bright wrote:

On 7/16/2013 8:49 PM, Timothee Cour wrote:

So how about a library solution instead, which doesn't require compiler change:


While semantically a great idea, technically I don't think CTFE is up to
implementing a C front end yet.


This is the right path. We don't need the full front end, do we ?


Yeah, you do need the full front end. It's pretty amazing how the simplest of .h 
files seem determined to exercise every last, dark corner of the language.


If the converter doesn't accept the full language, you're just going to get a 
dump truck unloading on it.

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-17 08:29, angel wrote:

Possibly instead of 'include' would be better use 'include_C' as opposed
to C++ or any other language.


Or there could be an optional argument indicating the language.

pragma(include, foo.h, C);

--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-17 07:04, deadalnix wrote:


This is the right path. We don't need the full front end, do we ?


Oh, yes we do. You will always run into corner cases your tool cannot 
handle until you have a complete front end. I tried that first, before I 
used libclang.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-16 17:05, Dicebot wrote:


While this a relatively common request, I don't think such stuff belongs
to compiler. It creates extra mandatory dependencies while providing
little advantage over doing this as part of a build system.


I started to think a bit about this. One might need to specify various 
options to translate the header file. Options like include paths and 
similar. That might be quite problematic to do in a pragam, or via DMD 
command line options.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-17 10:14, Walter Bright wrote:


Yeah, you do need the full front end. It's pretty amazing how the
simplest of .h files seem determined to exercise every last, dark corner
of the language.

If the converter doesn't accept the full language, you're just going to
get a dump truck unloading on it.


When you do have a complete front end you can choose how to handle the 
language constructs the tool cannot (yet) translate. I.e. just skip it, 
insert a comment or similar.


If you don't have a full front end and encounters something that you 
cannot translate, you will most likely have weird behaviors.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Paulo Pinto]

On Wednesday, 17 July 2013 at 09:27:53 UTC, Jacob Carlborg wrote:

On 2013-07-17 10:14, Walter Bright wrote:

Yeah, you do need the full front end. It's pretty amazing how 
the
simplest of .h files seem determined to exercise every last, 
dark corner

of the language.

If the converter doesn't accept the full language, you're just 
going to

get a dump truck unloading on it.


When you do have a complete front end you can choose how to 
handle the language constructs the tool cannot (yet) translate. 
I.e. just skip it, insert a comment or similar.


If you don't have a full front end and encounters something 
that you cannot translate, you will most likely have weird 
behaviors.


Thus we are back to the compiler as library discussion.

--
Paulo

Re: Proof of concept: automatically import C header files

[Jacob Carlborg]

On 2013-07-17 13:24, Paulo Pinto wrote:


Thus we are back to the compiler as library discussion.


Yes, but for the C family of languages we already have a compiler as 
library, that is Clang.


--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[deadalnix]

On Wednesday, 17 July 2013 at 07:17:07 UTC, Timothee Cour wrote:
you'd still need to parse C files recursively (textual 
inclusion...),
handle different C function calling conventions, different C 
standards,
you'd need a way to forward to dmd different C compiler options 
(include
paths to standard / custom libraries), and eventually people 
will want to
parse C++ as well anyways. That can be a lot of work. Whereas 
using

existing tools takes much less effort and is less error prone.


I'm talking about semantic analysis, you answer with parsing, I'm 
not sure this is going to lead anywhere. Do you understand that a 
parser is actually quite a small part of a frontend ?

Re: Proof of concept: automatically import C header files

[deadalnix]

On Wednesday, 17 July 2013 at 09:27:53 UTC, Jacob Carlborg wrote:

On 2013-07-17 10:14, Walter Bright wrote:

Yeah, you do need the full front end. It's pretty amazing how 
the
simplest of .h files seem determined to exercise every last, 
dark corner

of the language.

If the converter doesn't accept the full language, you're just 
going to

get a dump truck unloading on it.


When you do have a complete front end you can choose how to 
handle the language constructs the tool cannot (yet) translate. 
I.e. just skip it, insert a comment or similar.


If you don't have a full front end and encounters something 
that you cannot translate, you will most likely have weird 
behaviors.


My understanding is that we only want to convert declaration to 
D. Can you give me an example of such corner case that would 
require the full frontend ?

Re: Proof of concept: automatically import C header files

[Walter Bright]

On 7/17/2013 2:27 AM, Jacob Carlborg wrote:

On 2013-07-17 10:14, Walter Bright wrote:


Yeah, you do need the full front end. It's pretty amazing how the
simplest of .h files seem determined to exercise every last, dark corner
of the language.

If the converter doesn't accept the full language, you're just going to
get a dump truck unloading on it.


When you do have a complete front end you can choose how to handle the language
constructs the tool cannot (yet) translate. I.e. just skip it, insert a comment
or similar.


Yes, but the front end itself must be complete. Otherwise,
it's not really practical when you're dealing with things like the preprocessor 
- because a non-compliant front end won't even know it has gone off the rails.


There are other issues when dealing with C .h files:

1. there may be various #define's necessary to compile it that would normally be 
supplied on the command line to the C compiler


2. there are various behavior switches (see the PR for DMD that wants to set the 
signed'ness of char types)


3. rather few .h files seem to be standard compliant C. They always rely on 
various compiler extensions


These problems are not insurmountable, they just are non-trivial and need to be 
handled for a successful .h file importer.

Re: Proof of concept: automatically import C header files

[Walter Bright]

On 7/17/2013 9:48 AM, deadalnix wrote:

My understanding is that we only want to convert declaration to D. Can you give
me an example of such corner case that would require the full frontend ?


One example:


//**Header**\\

int x;


Yes, this POS is real C code I got a bug report on. Note the trailing \\. Is 
that one line splice or two? You have to get the hairy details right. I've seen 
similar nonsense with trigraphs. I've seen metaprogramming tricks with token 
pasting. You can't dismiss this stuff.

Re: Proof of concept: automatically import C header files

[H. S. Teoh]

On Wed, Jul 17, 2013 at 12:46:54PM -0700, Walter Bright wrote:
 On 7/17/2013 9:48 AM, deadalnix wrote:
 My understanding is that we only want to convert declaration to D.
 Can you give me an example of such corner case that would require the
 full frontend ?
 
 One example:
 
 
 //**Header**\\
 
 int x;
 
 
 Yes, this POS is real C code I got a bug report on. Note the
 trailing \\. Is that one line splice or two? You have to get the
 hairy details right. I've seen similar nonsense with trigraphs. I've
 seen metaprogramming tricks with token pasting. You can't dismiss
 this stuff.

I've seen C code where the header file has function bodies in them.

Though about trigraphs... I've to admit I've never actually seen *real*
C code that uses trigraphs, but yeah, needing to account for them can
significantly complicate your code.

But as for preprocessor-specific stuff, couldn't we just pipe it through
a standalone C preprocessor and be done with it? It can't be *that*
hard, right?


T

-- 
Bare foot: (n.) A device for locating thumb tacks on the floor.

Re: Proof of concept: automatically import C header files

[Walter Bright]

On 7/17/2013 3:20 PM, H. S. Teoh wrote:

Though about trigraphs... I've to admit I've never actually seen *real*
C code that uses trigraphs, but yeah, needing to account for them can
significantly complicate your code.


Building a correct C front end is a known technology, doing a half-baked job 
isn't going to impress people.



But as for preprocessor-specific stuff, couldn't we just pipe it through
a standalone C preprocessor and be done with it? It can't be *that*
hard, right?


You could, but then you are left with failing to recognize:

#define FOO 3

and converting it to:

enum FOO = 3;

Re: Proof of concept: automatically import C header files

[H. S. Teoh]

On Wed, Jul 17, 2013 at 03:36:15PM -0700, Walter Bright wrote:
 On 7/17/2013 3:20 PM, H. S. Teoh wrote:
 Though about trigraphs... I've to admit I've never actually seen
 *real* C code that uses trigraphs, but yeah, needing to account for
 them can significantly complicate your code.
 
 Building a correct C front end is a known technology, doing a
 half-baked job isn't going to impress people.

IOW either you don't do it at all, or you have to go all the way and
implement a fully-functional C frontend?

If so, libclang is starting to sound rather attractive...


 But as for preprocessor-specific stuff, couldn't we just pipe it
 through a standalone C preprocessor and be done with it? It can't be
 *that* hard, right?
 
 You could, but then you are left with failing to recognize:
 
 #define FOO 3
 
 and converting it to:
 
 enum FOO = 3;

Hmm. We *could* pre-preprocess the code to do this conversion first to
pick out these #define's, then suppress the #define's we understand from
the input to the C preprocessor. Something like this:

bool isSimpleValue(string s) {
// basically, return true if s is something compilable
// when put on the right side of enum x = 
}

auto pipe = spawnCPreprocessor();
string[string] manifestConstants;
foreach (line; inputFile.byLine()) {
if (auto m=match(line, `^\s*#define\s+(\w+)\s+(.*?)\s+`))
{
if (isSimpleValue(m.captures[2])) {
manifestConstants[m.captures[1]] =
m.captures[2];

// Suppress enums that we picked out
continue;
}
// whatever we don't understand, hand over to
// the C preprocessor
}
pipe.writeln(line);
}

Basically, whatever #define's we can understand, we handle, and anything
else we let the C preprocessor deal with. By suppressing the #define's
we've picked out, we force the C preprocessor to leave any reference to
them as unexpanded identifiers, so that later on we can just generate
the enums and the resulting code will match up correctly.


T

-- 
Prosperity breeds contempt, and poverty breeds consent. -- Suck.com

Re: Proof of concept: automatically import C header files

[deadalnix]

On Wednesday, 17 July 2013 at 19:46:54 UTC, Walter Bright wrote:

On 7/17/2013 9:48 AM, deadalnix wrote:
My understanding is that we only want to convert declaration 
to D. Can you give
me an example of such corner case that would require the full 
frontend ?


One example:


//**Header**\\

int x;


Yes, this POS is real C code I got a bug report on. Note the 
trailing \\. Is that one line splice or two? You have to get 
the hairy details right. I've seen similar nonsense with 
trigraphs. I've seen metaprogramming tricks with token pasting. 
You can't dismiss this stuff.


This do not require semantic analysis.

Re: Proof of concept: automatically import C header files

[Walter Bright]

On 7/17/2013 5:31 PM, deadalnix wrote:

On Wednesday, 17 July 2013 at 19:46:54 UTC, Walter Bright wrote:

On 7/17/2013 9:48 AM, deadalnix wrote:

My understanding is that we only want to convert declaration to D. Can you give
me an example of such corner case that would require the full frontend ?


One example:


//**Header**\\

int x;


Yes, this POS is real C code I got a bug report on. Note the trailing \\. Is
that one line splice or two? You have to get the hairy details right. I've
seen similar nonsense with trigraphs. I've seen metaprogramming tricks with
token pasting. You can't dismiss this stuff.


This do not require semantic analysis.


Semantic analysis for C is trivial. The real problems are the phases of 
translation and the preprocessor.

Proof of concept: automatically import C header files

[Jacob Carlborg]

Made a proof of concept to automatically parse, translate and import C 
header files in D using DStep. DMD is linked against DStep and does not 
start new process to make the translation.


I added a new pragma, include, that handles everything. Use like this:

// foo.h
void foo ();

// main.d

module main;

pragma(include, foo.h);

void main ()
{
foo();
}

DMD: https://github.com/jacob-carlborg/dmd/tree/dstep
DStep: https://github.com/jacob-carlborg/dstep/tree/c_api

--
/Jacob Carlborg

Re: Proof of concept: automatically import C header files

[Robert]

On Tuesday, 16 July 2013 at 14:15:55 UTC, Jacob Carlborg wrote:
Made a proof of concept to automatically parse, translate and 
import C header files in D using DStep. DMD is linked against 
DStep and does not start new process to make the translation.




awesome! :-)

Re: Proof of concept: automatically import C header files

[Dicebot]

On Tuesday, 16 July 2013 at 14:15:55 UTC, Jacob Carlborg wrote:
Made a proof of concept to automatically parse, translate and 
import C header files in D using DStep. DMD is linked against 
DStep and does not start new process to make the translation.


While this a relatively common request, I don't think such stuff 
belongs to compiler. It creates extra mandatory dependencies 
while providing little advantage over doing this as part of a 
build system.


So far I am perfectly satisfied with invoking dstep from a 
Makefile.