Re: [Tinycc-devel] Error compiling musl (_Complex not supported): Waiting for a release

[Eric Raible]

For what it's worth I would _love_ to see _Thread_local

On Wed, Nov 6, 2024, 12:02 PM Detlef Riekenberg via Tinycc-devel <
tinycc-devel@nongnu.org> wrote:

> Hi Brian
>
>
> > Hi, I'm trying to compile musl libc with tinycc, I'm getting this error:
> > ./include/complex.h:16: error: _Complex is not yet supported
>
> After the release, i plan to work on the following C99/C11 items and see,
> what is needed to implement them:
>
> * Fix atomic support (C11)
>(Every access to an _Atomic variable has to be done atomic)
>
> * _Complex support (C99)
>(optional in C11, but required for C99)
>
> * _Thread_local (C11)
>(Required for C11)
>
> * Other items, which are missing for C99
>(When you know something, please let me know)
>
>
> > I guess the compiler does not support complex types,
>
> That's correct
>
>
> > I saw some messages on this mailing list about requests for it,
> > including a patch.
>
> Can you remember,  when the message was send?
>
>
> Regards ... Detlef
>
> --
> Gesendet mit der WEB.DE Mail App
>
> ___
> Tinycc-devel mailing list
> Tinycc-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Is tcc_add_library supposed to persist?

[Eric Raible]

Sorry for the self-reply, but I'm still trying to figure this out.
One more piece of information: using -vv I can see that
libm.so is only listed as being loaded one time.

After that, it is available, but is not listed.  Which is weird, b.c.
libc.so (for instance) is listed as being loaded every time.

Can anyone help me understand what's going on with this?
It really makes no sense to me.

Thanks!
- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Is tcc_add_library supposed to persist?

[Eric Raible]

Given that tcc_add_library() takes a TCCState it seems to me
that a call shouldn't persist to subsequent TCCStates.
This example seems to show that it does.

$ tcc -version
tcc version 0.9.28rc 2024-06-11 mob@08a4c52d (AArch64 Linux)
$ tcc tcc-add-library-bug.c -ltcc && ./a.out
tcc_add_library=0
tcc: error: undefined symbol 'atan'

tcc_add_library=1
3.14159

tcc_add_library=0
3.14159

Thanks - Eric

#include #include "libtcc.h"char *C ="#include
\n""#include \n""void pi() {
printf(\"%g\\n\", 4 * atan(1)); }\n";void test(int add_lib){
TCCState *s = tcc_new();tcc_set_output_type(s, TCC_OUTPUT_MEMORY);
   printf("tcc_add_library=%d\n", add_lib);if (add_lib)
tcc_add_library(s, "m");if (!tcc_compile_string(s, C) &&
!tcc_relocate(s)) {void (*pi)() = tcc_get_symbol(s, "pi");
   pi();}tcc_delete(s);printf("\n");}int main(){
test(0);test(1);test(0);return 0;}
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] __VA_OPT__

[Eric Raible]

How do we feel about the lack of __VA_OPT__?
I know that there are clumsy workarounds, but it is a useful
construct and probably not that difficult to add.

I could take a go at it given some pointers and a little encouragement.

Thanks - Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Minimizing libtcc memory use

[Eric Raible]

First off: If you are tired of this conversation, just tell me, I get it.


> > # text 32, data.rw  4, data.ro  4, bss
> 4 bytes
> > # memory usage: 8192 to run, 649 symbols, 2901 other, 1639290 max (bytes)
> > mem_cur_size=11742 (bytes)
>
> > So tcc_print_stats() says 11742, but then displays values totaling 11786.
>
> What 11786 ?
>
> 8192 + 649 + 2901 = 11742
>
I misinterpreted the output, and didn't realize that the 48 bytes
from text, data.rw, data.ro, and bss were already included.


> There is nothing wrong here:  11742 + 32 blocks * 96 = 14814
> Where 96 is the size of tcc's mem_debug extra header.
>
> If you want to see the 11742 from valgrind then you just need to
> run the same example with a normal tcc compiled without MEM_DEBUG.
>
> Which makes sense I would think.
>
100%


> But when showing the example with MEM_DEBUG and -bench -vv  I
> did not expect you to doubt the numbers in the first place.
>
I shouldn't have.  But once my allocator and valgrind agreed
I went down the wrong path, esp considering I was parsing the
output incorrectly.


> Rather I just was trying to show how you could get some numbers
> for your own real case instead.  Which as you suspect could be
> minimized from 29kB down to 1-2 kB.  Most likely impossible but
> if we had some numbers we could tell also why.
>

And showing me was helpful, I use it below to get some numbers.

I don't know enough about the internals of tcc to really even be having
this conversation.  But I do know that my interactive application can
have many TCCStates, and with:

./configure --extra-cflags="-DMEM_DEBUG"
with a: tcc_set_options(state, "-Werror -vv -bench");
a simple hello world (single-TCCState) example reports:
-
0: .text0x14b57000  len 001bc  align 1000
1: .data.ro 0x14b571c0  len 00030  align 0008
2: .data0x14b571f0  len 00078  align 0008
2: .bss 0x14b57268  len 00050  align 0008
2: .got 0x14b572b8  len 00030  align 0008
-
protect rwx 0x14b57000  len 01000
-

That looks great!

But tcc has actually allocated 21248, according to both my
(more sophisticated custom allocator) and valgrind
(for instance if I _don't_ delete the state).

So ~80% of the bytes are unaccounted for.
Scales exactly with the number of states, so it's not ideal.

My loaded C has a callback to register all of its functions
with the main application.  After that, I have no need for
tcc_get_symbol() support, or in fact anything from libtcc
except for tcc_delete().

So I'm trying to pre-delete() all of the unneeded stuff that
tcc_delete() will eventually free anyway.  I was calling that
tcc_finalize().  The goal is to reduce the minimal TCCState
size from ~21k to the 4K required for PROT_EXEC.

- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Minimizing libtcc memory use

[Eric Raible]

I'm sorry for this delayed response.  I've been camping, no wifi for a
minute...

On Mon, Mar 4, 2024 at 3:47 AM grischka via Tinycc-devel <
tinycc-devel@nongnu.org> wrote:

> On 03.03.2024 21:26, Eric Raible wrote:
> >  > isn't there a garbage collecting done at the end to remove all the
> unused stuff
> >  > to produce a binary that contains only the necessary parts ?
> >
> > That very well might be the case, but given that tcc_get_symbol()
> > can be used at any time between tcc_relocate() and tcc_delete(),
> > it follows that _at least_ symbols are resident in the TCCState.
> > What I'm wondering about is the feasibility of keeping just code and
> > data, and flushing everything else.  This would require a new API -
> > something like tcc_finalize(TCCState *) or perhaps
> > tcc_finalize(TCCState *, flags), where flags specify what to flush.
>
> Me thinks this discussion is going around in circles.
>

I'm sorry for that.  tcc as it has been has been tremendously useful to me,
and all I'm trying to do is help.  Btw, I was not advocating for any changes
to tcc_relocate() except that once it became a 1-arg function I suggested
that it could be eliminated and tcc_add_symbol() could call it on demand.

Your tcc_finalize() subject of obsession was what we already had
> all the time since 0.9.25, that is the option to allocate the
> executable code separately from the state and therefor the option
> to delete (finalize) the state and still keep the code.
>

I understand that now.  I had previously used only automatic allocation
in tcc_relocate(), and since it worked I never explored further.  My main
interest was for tcc_set_realloc(), and I am happy that it was "approved".

...
> At which point I found that maintaining two options for running
> code might not be to the best for both sides, neither for tcc wrt.
> maintainability nor for users wrt. avoidance of inconvenience.
> ...
> Where simple plus complicated is more complicated than just
> complicated btw.
>
> I understand the frustration and agree that maintainability is a
crucial consideration.


> > I don't know enough about the internals, but if I'm willing to run with
> > CONFIG_RUNMEM_RO, it seems like the per TCCState memory use in my case
> > could be decreased from something like 29K to 1K or 2K.
> >
> > I should mention that the memory usage in my case is 29K regardless
> > of whether CONFIG_RUNMEM_RO is 0 or 1.
>
> How do you know this?
>

B.c I installed a custom allocator with tcc_set_realloc().
My exact case is hard to reproduce except for me, but the
following standalone example illustrates that mem_cur_size
might be incorrect:

#if 0Everything here assumes the following patch:diff --git a/libtcc.c
b/libtcc.cindex 6d720e74..332ec13d 100644--- a/libtcc.c+++
b/libtcc.c@@ -844,6 +844,12 @@ LIBTCCAPI TCCState *tcc_new(void)
LIBTCCAPI void tcc_delete(TCCState *s1) {+if (s1->do_bench) {+
   fprintf(stderr, "Top of tcc_delete()\n");+
tcc_print_stats(s1, 0);+fprintf(stderr, "Early return from
tcc_delete() - expect leaks!\n");+return;+} /* free
sections */ tccelf_delete(s1); @@ -2209,5 +2215,6 @@ PUB_FUNC void
tcc_print_stats(TCCState *s1, unsigned total_time) } #endif
fprintf(stderr, " %d max (bytes)\n", mem_max_size);+
fprintf(stderr, "mem_cur_size=%d (bytes)\n", mem_cur_size); #endif }
Running this program under valgrind on my arm debian with either
theseconfigs seems to show that mem_max_size is
undercounting../configure --extra-cflags="-DMEM_DEBUG"./configure
--extra-cflags="-DMEM_DEBUG -DCONFIG_RUNMEM_RO=0"#endif#include
#include #include #include
"libtcc.h"typedef struct block {unsigned long size;char
data[];} block;#define _B2S(block)((block) ? (block)->size :
0)#define _B2D(block)((block) ? (block)->data : 0)#define
_D2B(data)((data)  ? (block *)(data) - 1 : 0)#define
_D2S(data)_B2S(_D2B(data))static unsigned int blocks, mcalls,
rcalls, fcalls, bytes, max;static void *reallocator(void *data,
unsigned long want){block *b = _D2B(data);int had =
_D2S(data);if (want) {// some accountingif (had) {
   ++rcalls;bytes -= had + sizeof b;} else
{++blocks;++mcalls;}bytes +=
want + sizeof b;if (max < bytes) max = bytes;b =
realloc(b, want + sizeof b);b->size = want;} else if (had)
{--blocks;++fcalls;bytes -= had + sizeof b;
free(b);b = 0;}return _B2D(b);}static void
fatal(char *msg){fprintf(stderr, "%s\n", msg);exit(1);}int
main(){tcc_set_realloc(reallocator);TCCState *state =
tcc_new(); 

Re: [Tinycc-devel] Minimizing libtcc memory use

[Eric Raible]

> isn't there a garbage collecting done at the end to remove all the unused
stuff
> to produce a binary that contains only the necessary parts ?

That very well might be the case, but given that tcc_get_symbol()
can be used at any time between tcc_relocate() and tcc_delete(),
it follows that _at least_ symbols are resident in the TCCState.
What I'm wondering about is the feasibility of keeping just code and
data, and flushing everything else.  This would require a new API -
something like tcc_finalize(TCCState *) or perhaps
tcc_finalize(TCCState *, flags), where flags specify what to flush.

I don't know enough about the internals, but if I'm willing to run with
CONFIG_RUNMEM_RO, it seems like the per TCCState memory use in my case
could be decreased from something like 29K to 1K or 2K.

I should mention that the memory usage in my case is 29K regardless
of whether CONFIG_RUNMEM_RO is 0 or 1.

Thanks - Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] problem on win64 with latest commit

[Eric Raible]

I cannot speak for Griska or the reasons why that change was committed.
But with respect to my mail from 2/25, once again, all I was doing was
hinting
that tcc could be configured to not call (the since reverted) unportable
APIs.

Anyway, from my point of view, a default of RO=1 is fine.  But it should be
#ifdef-protected so that tinycc/configure or win32/build-tcc.bat can change
it.

For instance:

#ifndef CONFIG_RUNMEM_RO
#   define CONFIG_RUNMEM_RO 1
#endif

But since I'm new here I'm not going to push such a commit without
encouragement.

Thanks - Eric

On Wed, Feb 28, 2024 at 10:26 PM Herman ten Brugge 
wrote:

> On 2/29/24 04:57, Eric Raible wrote:
>
> Huh?
>
> My email on 2/24 was definitely not intended to imply anything
> about "old windows versions".  I was simply trying to make a suggestion
> about how to locally configure tcc to avoid a reported compiler error.
>
> In any case grischka responded 9 or so hours later to say
> that the commit that caused that compiler error on windows
> had been reverted.
>
> Griska reverted the memalign code but also made a change to
> CONFIG_RUNMEM_RO=0.
> I asume this was done because of your mail???
>
> Setting CONFIG_RUNMEM_RO=0 looks incorrect to me because it sets write in
> executables.
> Apple has implemented W^X (Writes can not occur in executables) for
> security reasons.
> This may also be implemented in in future linux/bsd releases.
>
> Can I revert the change and set CONFIG_RUNMEM_RO=1 for all targets as
> before?
>
> Herman
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Minimizing libtcc memory use

[Eric Raible]

In my use case I typically give libtcc code a few hundred lines of code:

1) some standard includes:
#include 
#include 
#include 

2) ~100 lines of #defines
3) ~50 lines of structs and typedefs
4) ~70 lines of function and data declarations (externs)
5) ~100 lines of actual code and data
6) The TCCState has ~50 symbols added with tcc_add_symbol().

All that is well and good, and it works perfectly.  The problem is that the
above uses roughly 30K bytes of memory.  But the actual code and data
looks to be between a few hundred and a few thousand bytes.  So a
50x reduction looks feasible.

I can envision a new API that would flush all unnecessary info.
For instance:  int tcc_finalize(TCCState *);
After such a call the only valid operation on the TCCState would be
tcc_delete().

Two questions, if I may:
1) Is something like this of interest to anyone else?
2) Can anyone estimate how difficult this is?  I might be able to sponsor
development of such a feature if that would help.

Thanks - Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] problem on win64 with latest commit

[Eric Raible]

Huh?

My email on 2/24 was definitely not intended to imply anything
about "old windows versions".  I was simply trying to make a suggestion
about how to locally configure tcc to avoid a reported compiler error.

In any case grischka responded 9 or so hours later to say
that the commit that caused that compiler error on windows
had been reverted.

- Eric

On Tue, Feb 27, 2024 at 10:51 PM Herman ten Brugge via Tinycc-devel <
tinycc-devel@nongnu.org> wrote:

> On 2/27/24 19:51, grischka wrote:
> > On 26.02.2024 19:55, Herman ten Brugge via Tinycc-devel wrote:
> >> I commited a change to lib/runmain.c to silence a warning from gcc.
> >> After that testcase 112 did not work any more on 64 bits windows
> >> (tested with wine).
> >>
> >> I can fix this with fix1 (see below) or fix2 (see below).
> >> I do not know enough about windows to make the correct fix.
> >> Both fix1/fix2 look not correct to me.
> >
> > Hi Herman,
> >
> > well, first we'd need to know what problem we want to fix.  Maybe
> > the problem is not in tinycc at all.
> >
> > Also, the obvious to avoid that gcc warning IMO would be using:
> >
> > __attribute__((noreturn)) void __rt_exit(rt_frame *, int);
> >
> > Also, while at it:
> >
> > # ifdef _WIN32
> > #   define CONFIG_RUNMEM_RO 0
> > # else
> > #   define CONFIG_RUNMEM_RO 1
> > # endif
> >
> > would suggest a special case needed for windows.  However that is not
> > the case.  Whereas according to your comment, the exception is APPLE
> > actually.  So we'd better say #ifdef __APPLE__ or just use '1' for all
> > and leave a comment that it's needed for APPLE.
> >
> > Although tests on github actions did not show such problem, at least
> > not on macos-11.  Note that CONFIG_RUNMEM_RO was first introduced with
> > .rodata sections in commit 72f1dea5372ed551d203311e4f2ab769a54de9bd
> > from 2021-02,  and it seems that tcc already did work on MacOS even
> > before that commit.  If those previous commits still work then the
> > problem now might be elsewhere.
> >
> > Anyway, we cannot disable the runtime-function-unwind table because
> > without it longjmp() would crash on (a real) _WIN64.
> >
> > Also obviously, if __rt_exit doesn't return, then any code after it
> > cannot have any effect:
> >
> >>   __rt_exit(&f, code);
> >> +abort (); // avoid noreturn warning
> >
> > I'd assume something weird going on with set/longjmp of WINE and/or
> > the mingw-gcc that you use.  In any case it is important that
> > setjmp/longjmp/jmp_buf implementations do match each other.
> >
> > Which is why I made it so that tcc_setjmp records the longjmp
> > function pointer too because otherwise it could end up using the
> > longjmp from libtcc.dll/so and the setjmp from the application
> > that links to it, with possibly different concepts each. (Not
> > sure if/where it really can happen though).
> >
> There is probably a bug in the wine stack unwinding code.
> The __attribute__((noreturn)) looks like the correct fix. I will apply it.
>
> I made the change to CONFIG_RUNMEM_RO because it was set to 1
> before and that worked on all targets. I saw a mail from Eric Raible
> on 25 Feb 2024 that it should be set to 0 for old window versions.
> This is why I made the change to only effect windows.
> On most targets setting it to 0 or 1 did work. (Not tested on all)
>
> I use an apple machine from the gcc compile farm.
> Darwin cfarm104.cfarm.net 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug
> 22 20:20:05 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T8101 arm64
> I did not check why it failed but with CONFIG_RUNMEM_RO=0 it reported:
> mprotect failed (did you mean to configure --with-selinux?)
> Using --with-selinux on apple works but looked strange.
>
> I also noticed that maybe --with-selinux is not needed any more
> with the recent changes. I removed it on all my test targets.
>
>  Herman
>
>
> ___
> Tinycc-devel mailing list
> Tinycc-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Newest TCC builds broken on legacy Windows versions on GCC 4.5.2

[Eric Raible]

FWIW I'm happy about that revert.  But that being the case,
wouldn't we also want:

diff --git a/tccrun.c b/tccrun.cindex 94c61ceb..531f5cc7 100644---
a/tccrun.c+++ b/tccrun.c@@ -91,17 +91,6 @@ static void
*win64_add_function_table(TCCState *s1); static void
win64_del_function_table(void *); #endif -#ifdef __APPLE__-# define
CONFIG_RUNMEM_ALIGNED 0-#else-# ifndef CONFIG_RUNMEM_ALIGNED-#  define
CONFIG_RUNMEM_ALIGNED 1-# endif-# if CONFIG_RUNMEM_ALIGNED-#  include
 /* memalign() */-# endif-#endif- #if !_WIN32 && !__APPLE__
//#define HAVE_SELINUX 1 #endif



On Sun, Feb 25, 2024 at 11:07 AM grischka via Tinycc-devel <
tinycc-devel@nongnu.org> wrote:

> On 25.02.2024 11:02, Eric Raible wrote:
> > If build with:
> > -DCONFIG_RUNMEM_ALIGNED=0
> > I'm hoping you'd be good to go.
>
> Don't mind, I reverted the usage of those "more recent" system
> features (as in memalign() & gettid()).  Simpler code, less
> problems ...
>
> -- grischka
>
> >
> > On linux one would:
> > ./configure --extra-cflags=-DCONFIG_RUNMEM_ALIGNED=0
> >
> > On windows I'm not sure, but perhaps augmenting the lines in
> > win32/build-tcc.bat to look like this would do the same:
> >
> > set D32=-DTCC_TARGET_PE -DTCC_TARGET_I386 -DCONFIG_RUNMEM_ALIGNED=0
> > set D64=-DTCC_TARGET_PE -DTCC_TARGET_X86_64 -DCONFIG_RUNMEM_ALIGNED=0
> >
> > - Eric
> >
> >
> > On Sat, Feb 24, 2024 at 7:29 AM Dog Dog  <mailto:andreywilliamsjeff...@gmail.com>> wrote:
> >
> > I get an error about missing entry point _aligned_free in
> msvcrt.dll. Since the newest builds are labeled as TCC version 0.9.28 RC
> one could wait until after an official TCC version 0.9.28 to break
> compatibility with legacy Windows versions.
> >
> > The newest builds of TCC are broken on Windows 2000, Windows NT 4.0,
> Windows NT 3.51, Windows Me, Windows 98 and Windows 95. I realize that old
> versions of Windows can't be supported forever and that is an amazing level
> of compatibility for builds prior to recent builds.
> >
> > The problem started with this build:
> >
> https://repo.or.cz/tinycc.git/commit/7b9f19eaab7e568a7c7a42725da812377a588f50
> > ___
> > Tinycc-devel mailing list
> > Tinycc-devel@nongnu.org <mailto:Tinycc-devel@nongnu.org>
> > https://lists.nongnu.org/mailman/listinfo/tinycc-devel
> >
> >
> >
> > ___
> > Tinycc-devel mailing list
> > Tinycc-devel@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/tinycc-devel
> >
>
>
> ___
> Tinycc-devel mailing list
> Tinycc-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Newest TCC builds broken on legacy Windows versions on GCC 4.5.2

[Eric Raible]

If build with:
-DCONFIG_RUNMEM_ALIGNED=0
I'm hoping you'd be good to go.

On linux one would:
./configure --extra-cflags=-DCONFIG_RUNMEM_ALIGNED=0

On windows I'm not sure, but perhaps augmenting the lines in
win32/build-tcc.bat to look like this would do the same:

set D32=-DTCC_TARGET_PE -DTCC_TARGET_I386 -DCONFIG_RUNMEM_ALIGNED=0
set D64=-DTCC_TARGET_PE -DTCC_TARGET_X86_64 -DCONFIG_RUNMEM_ALIGNED=0

- Eric


On Sat, Feb 24, 2024 at 7:29 AM Dog Dog 
wrote:

> I get an error about missing entry point _aligned_free in msvcrt.dll.
> Since the newest builds are labeled as TCC version 0.9.28 RC one could wait
> until after an official TCC version 0.9.28 to break compatibility with
> legacy Windows versions.
>
> The newest builds of TCC are broken on Windows 2000, Windows NT 4.0,
> Windows NT 3.51, Windows Me, Windows 98 and Windows 95. I realize that old
> versions of Windows can't be supported forever and that is an amazing level
> of compatibility for builds prior to recent builds.
>
> The problem started with this build:
>
> https://repo.or.cz/tinycc.git/commit/7b9f19eaab7e568a7c7a42725da812377a588f50
> ___
> Tinycc-devel mailing list
> Tinycc-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/tinycc-devel
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] tcc_set_realloc() no longer works as advertised

[Eric Raible]

Was it intended that d2f8ceac7 (tccrun: review last changes) would go
against the comment in libtcc.h?

/* set custom allocator for all allocations (optional) */
LIBTCCAPI void tcc_set_realloc(TCCReallocFunc *my_realloc);

Because default (unless -DCONFIG_RUNMEM_ALIGNED=0) since d2f8ceac7
is to use memalign() / libc_free() for some memory (mprotect()-ed ones?)
and the user-specified allocator for others.

At the very least, this diminishes the utility of a user-specified
allocator.

- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Linkage error using -bt

[Eric Raible]

Is it expected that -bt can't be used in a program that uses tcc?

#include int main(){TCCState *s = tcc_new();return 0;}

$ tcc -bt a.c -ltcc -ldl -lpthread
/usr/local/lib/tcc/bt-exe.o: error: 'rt_wait_sem' defined twice
/usr/local/lib/tcc/bt-exe.o: error: 'rt_post_sem' defined twice
/usr/local/lib/tcc/bt-exe.o: error: '_rt_error' defined twice
$

- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation, target

[Eric Raible]

That is nice, I wish I had thought of it.

Any thoughts about my suggestion from a few days back that tcc_relocate()
could be dropped from libtcc.h, and instead be automatically called on the
first
call to tcc_get_symbol()?

On Tue, Feb 13, 2024 at 4:15 PM grischka  wrote:

> On 11.02.2024 11:08, draco via Tinycc-devel wrote:
> > You're right, I didn't verify closely, but the ABI is still intact.
> Sorry...
>
> To be nice I've made tcc_relocate() abort with a notice
> when it's called with the former two-step method ;)
>
> -- gr
>
> >
> > Am 10.02.24 um 22:12 schrieb Eric Raible:
> >> > This means, that not only the public API changes, but also the
> >> > libtcc.dll/so ABI, making all programs using libtcc crash without
> warning.
> >> >
> >> > Is this intended?
> >> >
> >> > Michael
> >> I just tried it.  It looks like only programs that _don't_ use
> >> TCC_RELOCATE_AUTO would be affected (at least on my x86_64 debian box).
> >> Looks like the breakage would only be in a program uses manual memory
> management,
> >> that uses the system tcc (instead of a private  version), and where the
> system tcc is
> >> updated w/out rebuilding the application.  Anyone out there who would
> be in that situation?
> >> - Eric
> >
> >
> > ___
> > Tinycc-devel mailing list
> > Tinycc-devel@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/tinycc-devel
> >
>
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation, target

[Eric Raible]

I've just taken a closer look at libtcc.h.  Now, I realize that this is
radical,
but for the sake of discussion... if we're willing to break compatibility
it seems
to me that just one new API:

/* add option as on a comment line (multiple supported) */
LIBTCCAPI int tcc_argv_add(TCCState *s, const char *str);

could replace all of:

LIBTCCAPI int tcc_set_options(TCCState *s, const char *str);
LIBTCCAPI void tcc_set_lib_path(TCCState *s, const char *path);
LIBTCCAPI int tcc_add_include_path(TCCState *s, const char *pathname);
LIBTCCAPI int tcc_add_sysinclude_path(TCCState *s, const char *pathname);
LIBTCCAPI void tcc_define_symbol(TCCState *s, const char *sym, const char
*value);
LIBTCCAPI void tcc_undefine_symbol(TCCState *s, const char *sym);
LIBTCCAPI int tcc_add_file(TCCState *s, const char *filename);

Also, now that tcc_relocate() has no options, tcc_get_symbol() could
just remember if it's been called and call tcc_relocate(s) on its
first invocation.

So this would also disappear:

/* do all relocations (needed before using tcc_get_symbol()) */
LIBTCCAPI int tcc_relocate(TCCState *s1);

Food for thought, perhaps.

- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation, target

[Eric Raible]

> This means, that not only the public API changes, but also the
> libtcc.dll/so ABI, making all programs using libtcc crash without warning.
>
> Is this intended?
>
> Michael


I just tried it.  It looks like only programs that _don't_ use

TCC_RELOCATE_AUTO would be affected (at least on my x86_64 debian box).


Looks like the breakage would only be in a program uses manual memory
management,

that uses the system tcc (instead of a private  version), and where
the system tcc is

updated w/out rebuilding the application.  Anyone out there who would
be in that situation?


- Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Recent patches for the TinyCC 0.9.28 Release Candidate

[Eric Raible]

Detlef,

Thanks for taking the time to respond!  I'm new here, and still finding my
way around.

Let me respond to everything:
> A custom allocator is not a new idea.  TinyCC has already a custom
allocator and uses it by default.
I
'm using libtcc in my application, and I didn't see a way to tell libtcc to
use my _application_'s allocator.
If by "custom allocator" you mean tcc_realloc()/tcc_realloc_debug() etc,
that's fine, but unless
I can replace them the fact that it's "custom" does me no good.

> How often will it be useful for a libtcc user to force TinyCC to use an
application provided allocator?

I have no idea, but if the application is a language that uses libtcc as a
back-end then perhaps often.
libtcc is reliable enough to be part of long-lived
sophisticated applications.  Such applications
often care a great deal about memory management.

> You add an overhead for every user of TinyCC for every memory
allocation/free.

Yes.  The preexisting tcc_realloc() called realloc() directly, mine calls a
function which calls realloc().
I measured it on some large input (40K lines of C) and saw no difference.

> * it is a far to big change for a release candidate

That is not up to me to evaluate.  But as the commit message says, the
resulting code is quite
a bit more clear.  The diffs don't show that!  But if you look I"ll bet
your eyeballs would agree.

> * it needs a lot of testing

I wasn't aware of the release candidate status, but I have tested it
extensively.
With tcc's tests, with my application, with valgrind.  Personally, I'm hard
to satisfy...
but I'm satisfied.

> * it adds more complexity

I would seriously claim that it is _simpler_.  Again, the diffs don't show
that but
the resulting code is more modular and easier to read.  The #defines are
much
better organized, and my addition of "void libc_free(void *ptr)" makes what
was
slightly obscure code more intelligible.

Thanks again.  If it turns out that it needs to be reverted for the
release, that's fine.
In that case, I'd certainly like it to become official after the release is
tagged.

- Eric


On Wed, Feb 7, 2024 at 5:49 PM Detlef Riekenberg  wrote:

> Hi TinyCC follower.
>
>
> The current branch is a release candidate for TinyCC 0.9.28 since some
> month.
>
> From the active people here, only grischka has the rights to make a
> release,
> and I suggest, that we should help him to be comfortable to make a release
> by:
> * more testing
> * add only minimal fixes to the current mob branch to increase stability
> * avoid to push any new features for now
>
> I further suggest, that for any patch, which is longer than one or two
> lines,
> the change should be discussed here before pushing.
>
>
> @grischka
> I tested your recent fixes:
> The i386-tcc search path works now. Thanks
> (I tested with a simple Hello_World for X11 and a simple ppm viewer for
> X11)
>
>
> @Eric Raible:
> A custom allocator is not a new idea.
> TinyCC has already a custom allocator and uses it by default.
> How often will it be useful for a libtcc user to force TinyCC
> to use an application provided allocator?
> You add an overhead for every user of TinyCC for every memory
> allocation/free.
>
> My additional comments to your patch:
> * it is a far to big change for a release candidate
> * it needs a lot of testing
> * it adds more complexity
>
> #
>
> @kbkpbot
> Yes, there are many things missing for atomic support,
> but when i read your patch, i'm not sure,
> if the code works with any windows compiler, which is incompatible to gcc.
> (I changed my SSD and msvc is not installed yet, so I can't test that now)
>
> Reason: "__attribute__" and "__asm__" are gcc extensions.
> Are they supported by MSVC?
> (Yes, i saw, that you mostly just moved the code around...)
>
> +#ifndef __TINYC__
> +void __atomic_signal_fence(int memorder)
> __attribute__((alias("__tcc_atomic_signal_fence")));
>
>
> In the meantime, a macro for a fence can help
>
>
> #
>
> I suggest to revert both changes (with the cleanup from @Herman)
> and wait until @grischka pushes a release.
>
> @grischka, what do you think about reverting everything after your cleanup
> patch
> and then push a release?
>
>
> --
> Bye bye ... Detlef
>
>
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Allow use of a custom allocator in libtcc

[Eric Raible]

I just pushed a change with the following commit message.
I'm sending this because the instructions say to chat it up after pushing.

Allow use of a custom allocator in libtcc

When using libtcc it's reasonable to be able to use the application's
memory allocator for all allocations, including tcc_new(), and including
#define MEM_DEBUG

Ideally the allocator would be stored in the TCCState, like
TCCErrorFunc.
That would imply a new API tcc_new_with_allocator(), but more
importantly
would require all uses of tcc_malloc(x) to be changed to
s->tcc_malloc(x).
That's a non-starter in my book.

Instead I refactored the memory management code so that all allocations
flow through tcc_realloc().  Which simply calls a function pointer, the
default value of which is the previous tcc_realloc().

It then becomess trivial to install a new allocator with the new
function:
LIBTCCAPI void tcc_set_realloc(TCCReallocFunc realloc);

The resulting code adds the trivial cost of an additional function call
per allocation/free.  It also doesn't distinguish between malloc failure
and realloc failure, but since both just fprintf then exit() that seems
unimportant to me.

On the plus side the preprocessor magic is much more clear.  The diffs
don't hightlight that, but take a look at the result to see if you
agree.

All tests passed on my x86 linux box.
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] "tcc -run -" closes stdin so that program itself can't read from stdin

[Eric Raible]

tcc version 0.9.28rc 2023-11-27 mob@0655fd9 (x86_64 Linux)

Given this straightforward file named foo.c:

#include #include int main(int argc, char *argv[]){
   for (int i = 0; i < argc; ++i) printf("argv[%d]=%s", i, argv[i]);
 printf("\n");if (argc == 2 && !strcmp(argv[1], "-")) {
char buf[1024] = {};printf("enter a line: ");if
(fgets(buf, sizeof buf, stdin))printf("[%s]\n", buf);
  elseprintf("fgets failed");}return 0;}

The following command does what I would expect (it allows me to type
in a line of input):

$ tcc -run foo.c -

But if I try to read foo.c from stdin then foo.c itself is unable to
read, b.c. stdin is already closed:

$ cat foo.c | tcc -run - -

I tried duplicating fd 0 in _tcc_open() but that didn't work.
Suggestions welcome.

if (strcmp(filename, "-") == 0) {fd = dup(0), filename =
"";...

Thanks - Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Misleading warning message - both gcc and clang refuse to compile and error out

[Eric Raible]

The following program has an obvious bug, and both gcc and clang refuse
to compile it with an on-point error.  tcc produces an incorrect warning and
creates code that has no chance of working:

#include 
>
> typedef struct symbol symbol;
> extern void foo(symbol *);
>
> int main()
> {
>   foo(&(symbol){});
>   return 0;
> }
>
> struct symbol {
>   char *name;
>   int size;
> };
>
> void foo(symbol *s)
> {
>   printf("name=[%s]", s->name);
> }
>

tcc version 0.9.28rc 2023-11-08 mob@be8f894 (x86_64 Linux)

This is obviously not a show-stopper, but using an incomplete type _is_
something
that can happen in real life, and tcc does a pretty poor job here.  I'd be
willing to take
a look if someone can give me some hints.

Thanks - Eric
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Minor error-reporting bug

[Eric Raible]

Here's a minimal test case:

void f()
{
do { };
}

which gives:

tccbug.c:3: error: '[bletch]' expected (got ";")
where bletch is the code for TOK_WHILE (value==261 in my build) rendered as %c.

In emacs/eshell this displays as   error: '^E' expected (got ";")
In bash this displays as:   error: '' expected (got ";")

It seems to me that "ST_FUNC void skip(int c)" needs to be smarter
about handling 'c',
perhaps formatting it into a string before calling tcc_error().  I
could (in theory) fix this,
but it looks like it would require refactoring get_tok_str() to not
use the global cstr_buf).
A bit above my pay grade.

Anyway, this is not a show stopper by any means, but it is an ugly and
initially confusing message.

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] Replacing memory default allocator with an application-specific one

[Eric Raible]

I've got a project that makes heavy use of libtcc, and it would be
useful if I could
inform libtcc to use the application's memory allocator.  Since tcc_new() itself
of course allocates perhaps a reasonable approach would be something like:

TCCState *tcc_new_with_realloc(void *(*func)(void *ptr, unsigned long size));

Is there any interest in this?
Does something like this already exist? (I looked)

Thanks - Eric (rai...@gmail.com)

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

[Tinycc-devel] libtcc crash - full test case enclosed

[Eric Raible]

I am a very satisfied user of libtcc but just found this crash.
I am not on the mailing list so feel free to CC me or email directly.

tcc version 0.9.27 mob:b006b98 2023-04-09T06:06:10+08:00 (AArch64 Linux)
no modifications

This test case crashes with a segmentation fault whether compiled with
tcc or gcc.
I realize that the code I'm trying to compile has an error.
But obviously that error is not being properly handled.
The fact that the _second_ (correct) call to use_libtcc() crash is the
interesting part.

#include 
#include "libtcc.h"

#define ERROR(what) { printf("couldn't %s()", what); goto end; }
static void use_libtcc(char *code) {
  TCCState *s = tcc_new();
  if (!s) ERROR("tcc_new")
  if (tcc_set_output_type(s, TCC_OUTPUT_MEMORY) == -1)
ERROR("tcc_set_output_type")
  if (tcc_compile_string(s, code) == -1) ERROR("tcc_compile_string")
  if (tcc_relocate(s, TCC_RELOCATE_AUTO) == -1) ERROR("tcc_relocate")
end:
  tcc_delete(s);
}

int main() {
  use_libtcc("void foo() { a: int x;}"); // removing the label fixes the problem
  use_libtcc("void foo() { }"); // the crash occurs when compiling THIS line!
}

Valgrind reports the following:

==1341== Memcheck, a memory error detector
==1341== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1341== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==1341== Command: ./a.out
==1341==
:1: error: identifier expected
==1341== Invalid read of size 8
==1341==at 0x11A098: label_pop (tccgen.c:786)
==1341==by 0x12C247: gen_function (tccgen.c:8290)
==1341==by 0x12CA93: decl (tccgen.c:8543)
==1341==by 0x1192FF: tccgen_compile (tccgen.c:393)
==1341==by 0x10BC2F: tcc_compile (libtcc.c:746)
==1341==by 0x10BCA7: tcc_compile_string (libtcc.c:760)
==1341==by 0x10A8E3: use_libtcc (in /home/raible/qroq/bug/a.out)
==1341==by 0x10A967: main (in /home/raible/qroq/bug/a.out)
==1341==  Address 0x4bc1ab8 is 1,160 bytes inside a block of size 8,176 free'd
==1341==at 0x484AFE0: free (vg_replace_malloc.c:538)
==1341==by 0x10AD37: tcc_free (libtcc.c:257)
==1341==by 0x10AF57: dynarray_reset (libtcc.c:481)
==1341==by 0x11938B: tccgen_finish (tccgen.c:411)
==1341==by 0x10BC3F: tcc_compile (libtcc.c:751)
==1341==by 0x10BCA7: tcc_compile_string (libtcc.c:760)
==1341==by 0x10A8E3: use_libtcc (in /home/raible/qroq/bug/a.out)
==1341==by 0x10A95B: main (in /home/raible/qroq/bug/a.out)
==1341==  Block was alloc'd at
==1341==at 0x4849E4C: malloc (vg_replace_malloc.c:307)
==1341==by 0x10AD57: tcc_malloc (libtcc.c:263)
==1341==by 0x1198C3: __sym_malloc (tccgen.c:578)
==1341==by 0x11997B: sym_malloc (tccgen.c:598)
==1341==by 0x1199F3: sym_push2 (tccgen.c:622)
==1341==by 0x117147: macro_subst_tok (tccpp.c:3366)
==1341==by 0x1172FF: macro_subst (tccpp.c:3421)
==1341==by 0x116203: macro_arg_subst (tccpp.c:3014)
==1341==by 0x1170AB: macro_subst_tok (tccpp.c:3350)
==1341==by 0x117683: next (tccpp.c:3508)
==1341==by 0x10E76F: skip (tccpp.c:105)
==1341==by 0x12CF2F: decl (tccgen.c:8637)
==1341==
==1341== Invalid read of size 2
==1341==at 0x11A0A4: label_pop (tccgen.c:787)
==1341==by 0x12C247: gen_function (tccgen.c:8290)
==1341==by 0x12CA93: decl (tccgen.c:8543)
==1341==by 0x1192FF: tccgen_compile (tccgen.c:393)
==1341==by 0x10BC2F: tcc_compile (libtcc.c:746)
==1341==by 0x10BCA7: tcc_compile_string (libtcc.c:760)
==1341==by 0x10A8E3: use_libtcc (in /home/raible/qroq/bug/a.out)
==1341==by 0x10A967: main (in /home/raible/qroq/bug/a.out)
==1341==  Address 0x4bc1a94 is 1,124 bytes inside a block of size 8,176 free'd
==1341==at 0x484AFE0: free (vg_replace_malloc.c:538)
==1341==by 0x10AD37: tcc_free (libtcc.c:257)
==1341==by 0x10AF57: dynarray_reset (libtcc.c:481)
==1341==by 0x11938B: tccgen_finish (tccgen.c:411)
==1341==by 0x10BC3F: tcc_compile (libtcc.c:751)
==1341==by 0x10BCA7: tcc_compile_string (libtcc.c:760)
==1341==by 0x10A8E3: use_libtcc (in /home/raible/qroq/bug/a.out)
==1341==by 0x10A95B: main (in /home/raible/qroq/bug/a.out)
==1341==  Block was alloc'd at
==1341==at 0x4849E4C: malloc (vg_replace_malloc.c:307)
==1341==by 0x10AD57: tcc_malloc (libtcc.c:263)
==1341==by 0x1198C3: __sym_malloc (tccgen.c:578)
==1341==by 0x11997B: sym_malloc (tccgen.c:598)
==1341==by 0x1199F3: sym_push2 (tccgen.c:622)
==1341==by 0x117147: macro_subst_tok (tccpp.c:3366)
==1341==by 0x1172FF: macro_subst (tccpp.c:3421)
==1341==by 0x116203: macro_arg_subst (tccpp.c:3014)
==1341==by 0x1170AB: macro_subst_tok (tccpp.c:3350)
==1341==by 0x117683: next (tccpp.c:3508)
==1341==by 0x10E76F: skip (tccpp.c:105)
==1341==by 0x12CF2F: decl (tccgen.c:8637)
==1341==
==1341== Invalid read of size 2
==1341==at 0x11A0EC: label_pop (tccgen.c:789)
==1341==by 0x12C247: gen_function (tccgen.c:8290)
==1341==by 0x12CA93: decl (tccgen.c:8543)
==1341==by 0x1192FF: tccgen_compile (tccgen.c

[Tinycc-devel] e41730f11 (tcc -vv: show cross-libtcc1.a correctly (and more)) broke my libtcc code

[Eric Raible]

Perhaps I was using libtcc incorrectly, but libtcc.h is a bit unclear about it.

The comment for tcc_set_output_type() says:
/* set output type. MUST BE CALLED before any compilation */

In reality, as of e41730f11 that is no longer good enough.
It used to be that tcc_add_symbol() would work if called before
tcc_set_output_type().
After e41730f11 tcc_set_output_type() must be called first in order to
avoid symtab_section==0.

Sure, I can fix this by calling tcc_set_output_type() first, but it
was a regression for my use case.

A more general comment: tcc and the ongoing work is awesome awesome
awesome, THANK YOU!

Thanks - Eric

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel