Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
1) I prefer the two-function interface, especially with the nicer names Sean suggested, but who cares, beggars can't be choosers. 2) This whole exchange took more effort than it should have taken. 3) Thanks! Now it works! (I presume; I've not yet tested it.) David On Mon, Sep 3, 2012 at 6:07 AM, grischka gris...@gmx.de wrote: Michael Matz wrote: You earnestly added this interface abomination to avoid exporting a function?Well, as you asked: yes it's ugly, and yes it's confusing. (and it's also non-C, ((void*)-1) might not be representable; the only integer literal that definitely is convertible into a pointer is 0; but I'll admit that this would just be a nitpick). But why you think this interface would be better than an explicit one with two functions is a complete mystery to me. I mean, really, can you give a reason? I'm seriously interested. So you seriously wanted to export the wrapper AND its subroutine? I mean, after all, an API is an abstraction layer and not a self-service store. Anyhow, I think it is not more complex/ugly/confusing than the alternative. We would need to show users: - the abomination just minus the TCC_RELOCATE_AUTO part - plus the declaration of tcc_relocate + usage comment - some hint at least how the two functions relate to each other So, my reasons, short version: Smaller libtcc.h --- grischka Ciao, Michael. ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel -- Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian Kernighan ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated:
Michael Matz wrote:
You earnestly added this interface abomination to avoid exporting a
function?Well, as you asked: yes it's ugly, and yes it's
confusing. (and it's also non-C, ((void*)-1) might not be
representable; the only integer literal that definitely is convertible
into a pointer is 0; but I'll admit that this would just be a nitpick).
But why you think this interface would be better than an explicit one
with two functions is a complete mystery to me. I mean, really, can you
give a reason? I'm seriously interested.
So you seriously wanted to export the wrapper AND its subroutine?
I mean, after all, an API is an abstraction layer and not a
self-service store.
It depends on how you view APIs. Under Unix, there are four system calls
to read from a socket:
ssize_t read(int fh,void *buf,size_t count);
ssize_t recv(int s,void *buf,size_t count,int flags);
ssize_t recvfrom(int s,void *buf,size_t count,int flags,struct sockaddr
*from,socklen_t *fromlen);
ssize_t recvmsg(int s,struct msghdr *msg,int flags);
Each give a different level of control over how data from the socket is
read, from I don't really care (read()) to I want total control
(recvmsg()). Sure, read recv() probably calls recvfrom(), which in turn
calls recvmsg() [1] and I, for one, like the ability to pick and choose the
level of abstraction I'm comfortable with. For TCP connections, read() is
good enough, while for UDP, recvfrom() might fit what I'm doing better, and
if I'm implementing OSPF [2] I will most likely use recvmsg().
Yes, it appears to be a self-service store, but as Joel Spolsky has
stated, all abstractions leak [3] and if that's the case, I want to pick
where the leaks are (so to speak).
Anyhow, I think it is not more complex/ugly/confusing than the
alternative. We would need to show users:
- the abomination just minus the TCC_RELOCATE_AUTO part
- plus the declaration of tcc_relocate + usage comment
- some hint at least how the two functions relate to each other
So, my reasons, short version: Smaller libtcc.h
Seriously? *That's* your argument? You want a smaller header file? I
don't see that as being an argument. Heck, you can make a very small
libtcc.h file:
#ifndef LIBTCC_H
#define LIBTCC_H
/**
*
* Please see tcc-doc.txt or tcc-doc.html for information on how to
* use the routines defined. Or visit http://...
*
**/
#ifndef LIBTCCAPI
# define LIBTCCAPI
#endif
#ifdef __cplusplus
extern C {
#endif
#define TCC_OUTPUT_MEMORY0
#define TCC_OUTPUT_EXE 1
#define TCC_OUTPUT_DLL 2
#define TCC_OUTPUT_OBJ 3
#define TCC_OUTPUT_PREPROCESS4
#define TCC_OUTPUT_FORMAT_ELF0
#define TCC_OUTPUT_FORMAT_BINARY 1
#define TCC_OUTPUT_FORMAT_COFF 2
#define TCC_RELOCATE_AUTO (void*)1
struct TCCState;
typedef struct TCCState TCCState;
LIBTCCAPI TCCState *tcc_new(void);
LIBTCCAPI void tcc_delete(TCCState *s);
LIBTCCAPI void tcc_enable_debug(TCCState *s);
LIBTCCAPI void tcc_set_error_func(TCCState *s, void *error_opaque,void
(*error_func)(void *opaque, const char *msg));
LIBTCCAPI int tcc_set_warning(TCCState *s, const char *warning_name, int value);
LIBTCCAPI const char * tcc_set_linker(TCCState *s, char *option, int multi);
LIBTCCAPI int tcc_add_include_path(TCCState *s, const char *pathname);
LIBTCCAPI int tcc_add_sysinclude_path(TCCState *s, const char *pathname);
LIBTCCAPI void tcc_define_symbol(TCCState *s, const char *sym, const char
*value);
LIBTCCAPI void tcc_undefine_symbol(TCCState *s, const char *sym);
LIBTCCAPI int tcc_add_file(TCCState *s, const char *filename);
LIBTCCAPI int tcc_compile_string(TCCState *s, const char *buf);
LIBTCCAPI int tcc_set_output_type(TCCState *s, int output_type);
LIBTCCAPI int tcc_add_library_path(TCCState *s, const char *pathname);
LIBTCCAPI int tcc_add_library(TCCState *s, const char *libraryname);
LIBTCCAPI int tcc_add_symbol(TCCState *s, const char *name, const void *val);
LIBTCCAPI int tcc_output_file(TCCState *s, const char *filename);
LIBTCCAPI int tcc_run(TCCState *s, int argc, char **argv);
LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr);
LIBTCCAPI void *tcc_get_symbol(TCCState *s, const char *name);
LIBTCCAPI void tcc_set_lib_path(TCCState *s, const char *path);
#ifdef __cplusplus
}
#endif
#endif
There you go---small libtcc.h.
Further more, zlib, a library that does way less than compile C code, has
a 1200 line header file. jpeglib.h is 1100. png.h is 3362. Not wanting to
export a function because it makes the header file smaller is a piss-poor
reason to do it.
-spc (libtcc.h clocks in at only 113 ... )
[1] I haven't checked the code, so it may very well be that recvfrom()
does not call recvmsg() but both call the same set of
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Hi, On Wed, 29 Aug 2012, grischka wrote: What about this: Keep the 0.9.25 declaration of tcc_relocate, which was: Sure. Keep tcc_relocate with its current signature. /* copy code into memory passed in by the caller and do all relocations (needed before using tcc_get_symbol()). returns -1 on error and required size if ptr is NULL */ LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); but define a special value for 'ptr' to get behavior as of 0.9.24, as in: /* Do all relocations. Needed before using tcc_get_symbol(). Possible values for ptr: - TCC_RELOCATE_AUTO : Allocate memory internally - NULL : return required memory size for the step below - memory address: copy code into memory passed by the caller returns -1 on error. */ LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); #define TCC_RELOCATE_AUTO (void*)-1 Is this ugly? Confusing? 0.9.26 isn't out yet so we can still decide. You earnestly added this interface abomination to avoid exporting a function? I mean, huh? Well, as you asked: yes it's ugly, and yes it's confusing. (and it's also non-C, ((void*)-1) might not be representable; the only integer literal that definitely is convertible into a pointer is 0; but I'll admit that this would just be a nitpick). But why you think this interface would be better than an explicit one with two functions is a complete mystery to me. I mean, really, can you give a reason? I'm seriously interested. Ciao, Michael. ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: Still no. It returns (error ? -1 : ptr == NULL ? size : 0); Isn't that what I said? returns the number of bytes iff [1] ptr is NULL; othersise it returns 0 on success and -1 on an error. Where in there did I go wrong? I don't know. Either you don't understand my C or you don't understand your English. I see my mistake---I forgot a not between ptr and No, I was right the first time. Aah! Okay, I give up! I can't understand your code. I can't understand English. I can't understand why you won't take a simple modification. I'm done here. Look, the return value depends on two variables (ptr, error) as in: ptr | NULL | mem | NULL | mem | error | no | no | yes | yes | -+--+--+--+--+ return | size | 0 | -1 | -1 | Your statement in English as I read it however suggests the following behavior: ptr | NULL | mem | NULL | mem | error | no | no | yes | yes | -+--+--+--+--+ return | size | 0 | size | -1 | See the difference? See the consequences (crashes) if libtcc users follow your specification? Despite of your problems to describe it, your usage is correct though: https://github.com/spc476/lua-conmanorg/blob/master/src/tcc.c#L397 Anyway, like anyone working for free I rather push my own brilliant ideas than anyone else's so now we have the one-function/triple-option approach and I'm fine with it and you can live with it, at least. http://repo.or.cz/w/tinycc.git/commitdiff/ca38792df17fc5c8d2bb6757c512101610420f1e Thanks, --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Speaking as someone who has been mostly quiet on this mailing list, but maintaining my own fork of tcc for quite some time now, you are astoundingly difficult to work with. Just export the damn symbol. -Original Message- From: tinycc-devel-bounces+michael=theessentialexchange@nongnu.org [mailto:tinycc-devel-bounces+michael=theessentialexchange@nongnu.org] On Behalf Of grischka Sent: Friday, August 31, 2012 5:39 PM To: tinycc-devel@nongnu.org Subject: Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex() Sean Conner wrote: Still no. It returns (error ? -1 : ptr == NULL ? size : 0); Isn't that what I said? returns the number of bytes iff [1] ptr is NULL; othersise it returns 0 on success and -1 on an error. Where in there did I go wrong? I don't know. Either you don't understand my C or you don't understand your English. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Anyway, like anyone working for free I rather push my own brilliant ideas than anyone else's so now we have the one-function/triple-option approach and I'm fine with it and you can live with it, at least. Thank you. -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: In both cases, it returns the number of bytes required to relocate the code. No. Let me amend my previous statement on this---you are correct. And it's way worse than I expected. tcc_relocate_ex() returns the number of bytes iff ptr is NULL; otherwise it returns 0 on success and -1 on an error. Still no. It returns (error ? -1 : ptr == NULL ? size : 0); --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: In both cases, it returns the number of bytes required to relocate the code. No. Let me amend my previous statement on this---you are correct. And it's way worse than I expected. tcc_relocate_ex() returns the number of bytes iff ptr is NULL; otherwise it returns 0 on success and -1 on an error. Still no. It returns (error ? -1 : ptr == NULL ? size : 0); Isn't that what I said? returns the number of bytes iff [1] ptr is NULL; othersise it returns 0 on success and -1 on an error. Where in there did I go wrong? You know, at this point, I'm getting ready to just fork tcc and maintain my own copy, because it's taken what? Almost a month to get a simple change in? -spc (sigh) [1] If and only if. ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Jared Maddox wrote: but define a special value for 'ptr' to get behavior as of 0.9.24, as in: /* Do all relocations. Needed before using tcc_get_symbol(). Possible values for ptr: - TCC_RELOCATE_AUTO : Allocate memory internally - NULL : return required memory size for the step below - memory address: copy code into memory passed by the caller returns -1 on error. */ LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); #define TCC_RELOCATE_AUTO (void*)-1 Is this ugly? Confusing? 0.9.26 isn't out yet so we can still decide. --- grischka If you're going to use that method, then I'd suggest this definition of TCC_RELOCATE_AUTO: #define TCC_RELOCATE_AUTO (void*)tcc_relocate If I recall the C standard correctly, that will result on a value that's unique from all other valid addresses even on Harvard architectures (though porting TCC to one of those would be difficult), thus ensuring that the new behavior can only be invoked by accident if the calling code has a bug. It also makes it easy to not run into a conflict with anything else, which could be relevant, since I half recall some architectures having valid negative addresses for some reason.. Thanks, and so we conclude that even with most careful considerations unique can still be different to really unique, in practice. In this case because tcc_relocate in a libtcc.dll is resolved to the function itself, but in the client code is resolved to the function vector (on windows, with any of GCC, CL, TCC). I dropped the minus and currently plan to go with #define TCC_RELOCATE_AUTO (void*)1 Better name or other suggestions still welcome. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: In this case because tcc_relocate in a libtcc.dll is resolved to the function itself, but in the client code is resolved to the function vector (on windows, with any of GCC, CL, TCC). I dropped the minus and currently plan to go with #define TCC_RELOCATE_AUTO (void*)1 Better name or other suggestions still welcome. Um ... did you miss my previous email? The one that goes: It was thus said that the Great grischka once stated: Michael Matz wrote: So, while the complaint about the leak is correct, the conservatism about exporting one internal function (whose functionality once was available as an exported function that was removed from the API) is hypocritic at best. Sorry for that assessment. IOW: I don't understand your reservations. To make it short: just export the damn thing. It makes sense. Well I hope this damn thing makes sense because I wrote it. ;) Anyway, we're not in a hurry, are we? Define hurry. I'd like to see this happen sooner than later, as I have code I already depend on and would rather not keep a separate fork of tcc around any longer than I have to. What about this: Keep the 0.9.25 declaration of tcc_relocate, which was: /* copy code into memory passed in by the caller and do all relocations (needed before using tcc_get_symbol()). returns -1 on error and required size if ptr is NULL */ LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); but define a special value for 'ptr' to get behavior as of 0.9.24, as in: /* Do all relocations. Needed before using tcc_get_symbol(). Possible values for ptr: - TCC_RELOCATE_AUTO : Allocate memory internally - NULL : return required memory size for the step below - memory address: copy code into memory passed by the caller returns -1 on error. */ LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); #define TCC_RELOCATE_AUTO (void*)-1 Is this ugly? Confusing? Personally, I don't mind the two functions, although I might rename tcc_relocate_ex() to tcc_relocate_mem() or tcc_relocate_here() (which I personally like, as it describes what is being done much better). That way, you wouldn't have to change the example codes and they remain as simple as possible (which is the reason for the change in tcc_relocate() in the first place). On the other hand ... 0.9.26 isn't out yet so we can still decide. Um, about that ... Version 0.9.25 from http://www.tcc.org/ (which is where anyone looking for tcc compiler will find first): LIBTCCAPI int tcc_relocate(TCCState *s1, void *ptr); Version 0.9.25 from http://repo.or.cz/w/tinycc.git (which someone will find if they look a bit further): LIBTCCAPI int tcc_relocate(TCCState *s1); You made an incompatible API change and didn't bother to bump the version number. So saying that 0.9.26 isn't out yet is a moot point I would think. Now, in light of that, you could change tcc_relocate() back to the www.tcc.org version, and add: LIBTCCAPI int tcc_relocate_simple(TCCState *s1); to remain compatible with the www.tcc.org version (although I'm not happy with the name, but I have nothing better at this time). -spc (Not a fan of TCC_RELOCATE_AUTO ... ) ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: Um ... did you miss my previous email? The one that goes: ... No, however I'm assuming that you (as well as some other people) wouldn't mind to see their 0.9.25 based code just work. In any case this looks to me like an argument, and I didn't see a better one yet. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: I do not like this idea, and here's why: it produces a function that does two different things depending upon one parameter: Count. It is three different things. In both cases, it returns the number of bytes required to relocate the code. No. I can't stand functions that do more than one thing for no real good reason. So being serious, you need - tcc_relocate - tcc_relocate_size - tcc_relocate_copy which then are three functions that do almost the same thing. Actually an earlier approach was to declare it like this: void *tcc_relocate(TCCState *s1, void *(*alloc_fn)(size_t)); So you could write tcc_relocate(TCCState *s1, NULL); or mem = tcc_relocate(TCCState *s1, malloc); I don't think this is one functions doing two entirely different things. However one probably would want an user parameter like this void *tcc_relocate(TCCState *s1, void *(*alloc_fn)(size_t, void*), void *up); which looks complicated and since you can't pass just malloc, also makes things complicated. So I decided for the two step approach. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: Um ... did you miss my previous email? The one that goes: ... No, however I'm assuming that you (as well as some other people) wouldn't mind to see their 0.9.25 based code just work. Which 0.9.25 codebase? There's the one from tinycc.org, and the one from your git repository, which aren't compatible. In any case this looks to me like an argument, and I didn't see a better one yet. What don't you like about promoting tcc_relocate_ex() to public? -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: I do not like this idea, and here's why: it produces a function that does two different things depending upon one parameter: Count. It is three different things. In both cases, it returns the number of bytes required to relocate the code. No. No? It doesn't return the number of bytes required to relocate the code? Because my reading of tcc_relocate() seems to do just that to me. I can't stand functions that do more than one thing for no real good reason. So being serious, you need - tcc_relocate - tcc_relocate_size - tcc_relocate_copy which then are three functions that do almost the same thing. Technically, you are correct, but I have seen the pass NULL to get the size required to allocate approach elsewhere. It doesn't quite sit well with me, but I can accept that approach. -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: I do not like this idea, and here's why: it produces a function that does two different things depending upon one parameter: Count. It is three different things. In both cases, it returns the number of bytes required to relocate the code. No. Let me amend my previous statement on this---you are correct. And it's way worse than I expected. tcc_relocate_ex() returns the number of bytes iff ptr is NULL; otherwise it returns 0 on success and -1 on an error. Somehow, I get the feeling that changing tcc_relocate_ex() to int tcc_relocate_ex(TCCState *state,void *ptr,size_t *psize); would be an uphill battle. -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Date: Tue, 28 Aug 2012 14:04:10 -0400
From: Sean Conner s...@conman.org
To: tinycc-devel@nongnu.org
Subject: Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Message-ID: 20120828180409.gc5...@brevard.conman.org
Content-Type: text/plain; charset=us-ascii
It was thus said that the Great grischka once stated:
Sean Conner wrote:
Can you at least commit the change that makes tcc_relocate_ex()
global?
Yes, perhaps at least. You wrote:
... but fixing my code to match the current definition
of tcc_relocate() was out of the question (the surrounding code manages
the
lifetime of both the TCCState and the resulting executable code compiled
into memory).
I'd actually like to know some details why you can't just use
tcc_relocate()?
Admittedly we know little about how people actually use the libtcc API.
I have three use cases right now in using libtcc.
Case 1: Synthesis OS (http://valerieaurora.org/synthesis/SynthesisOS/)
Synthesis OS was an experiment operating system from 1992 written for a
Ph. D. dissertation. Dispite it being written in assembly language
(MC68030) it basically would recompile system calls on the fly (the whole
thing is mind blowing and its worth reading the dissertation)---think of a
just-in-time compiler but one that recompiles the kernel as needed. This
is a technique I'm interested in (on my own time).
I have a wow---it can work! type of prototype using libtcc (nothing
even
close to a kernel, just some simple code). I have a custom open()
function that will compile a custom read() routine based upon the
paramteres given it---if I open a file read-only for character-by-character
access, and if the file can be mmap()'ed into memory, then you get a
specialized version of read() that returns successive characters from an
mmap()'ed file; if the file can't be mmap()'ed, you get a more traditional
read() function. This re-compilation happens on a file-by-file basis (each
file returned from my custom open() will have a custom read() function
compiled that deals only with the given file).
The issue here with tcc_relocate() is that once the compilation phase is
done, I don't need (or want) the symbol table, section definitions,
buffers,
or anything else of the TCCState *except for the code just compiled*, and
the memory for that (the compiled code) is reclaimed on the call to a
custom
close() function.
Yes, I could skip the compilation step and load pre-written functions (as
a shared object) but the ability to compile a custom version, quickly (one
of the strengths of TCC) in memory and use it immediately, is not something
I want to give up here.
Quajects are indeed very neat. I've played around with them myself in
C writing some VMs, though I was focusing on the mutability and
never have compiled any of them.
Case 3: Closures in C (or, a mixture of case 1 and case 2)
Why this exists doesn't matter, but I have written Lua bindings to Xlib
(the lowest level of X programming without talking the protocol directly).
Xlib, however, reports some errors via a callback function you register
using XSetErrorHandler():
int (*XSetErrorHandler(int (*handler)(Display *, XErrorEvent *)))();
I would like to push the error into Lua, but the issue I have is---I need
a global variable pointing to the current Lua state, which is something I
don't particularly want (the global variable), because the following:
int foo(lua_State *L)
{
XSetErrorHandler(int (Display *display,XEerrorEvent *event)
{
lua_pushlightuserdata(L,display);
lua_gettable(L,LUA_REGISTRYINDEX);
lua_getfield(L,-1,error_handler);
mylua_pushevent(L,event);
lua_call(L,1,0);
return 0;
}
)
return 0;
}
is not legal C. However, using libtcc, I can fake a closure with C:
#include stdlib.h
#include X11/Xlib.h
#include syslog.h
#include lua.h
#include lualib.h
#include lauxlib.h
lua_State *L;
int Xcallback_function(Display *display,XErrorEvent *event)
{
lua_pushlightuserdata(...);
/* ... */
}
and compile it with (excluding error checking for clarity):
lua_State *L; /* assume it's set properly */
TCCState *s;
int size;
void *mem;
lua_State **pL;
int (*cb)(Display *,XErrorEvent *);
s = tcc_new();
tcc_set_output_type(s,TCC_OUT_MEMORY);
tcc_add_file(s,xcallback.c);
size = tcc_relocate_ex(s,NULL);
mem = lua_newuserdata(L,size);
tcc_relocate_ex(s,mem);
pL = tcc_get_symbol(s,L);
cb = tcc_get_symbol(s,Xcallback_function);
*pL = L;
XSetErrorHandler(cb);
/*
; I don't need the compiler info any more
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated:
Sean Conner wrote:
Can you at least commit the change that makes tcc_relocate_ex() global?
Yes, perhaps at least. You wrote:
... but fixing my code to match the current definition
of tcc_relocate() was out of the question (the surrounding code manages the
lifetime of both the TCCState and the resulting executable code compiled
into memory).
I'd actually like to know some details why you can't just use
tcc_relocate()?
Admittedly we know little about how people actually use the libtcc API.
I have three use cases right now in using libtcc.
Case 1: Synthesis OS (http://valerieaurora.org/synthesis/SynthesisOS/)
Synthesis OS was an experiment operating system from 1992 written for a
Ph. D. dissertation. Dispite it being written in assembly language
(MC68030) it basically would recompile system calls on the fly (the whole
thing is mind blowing and its worth reading the dissertation)---think of a
just-in-time compiler but one that recompiles the kernel as needed. This
is a technique I'm interested in (on my own time).
I have a wow---it can work! type of prototype using libtcc (nothing even
close to a kernel, just some simple code). I have a custom open()
function that will compile a custom read() routine based upon the
paramteres given it---if I open a file read-only for character-by-character
access, and if the file can be mmap()'ed into memory, then you get a
specialized version of read() that returns successive characters from an
mmap()'ed file; if the file can't be mmap()'ed, you get a more traditional
read() function. This re-compilation happens on a file-by-file basis (each
file returned from my custom open() will have a custom read() function
compiled that deals only with the given file).
The issue here with tcc_relocate() is that once the compilation phase is
done, I don't need (or want) the symbol table, section definitions, buffers,
or anything else of the TCCState *except for the code just compiled*, and
the memory for that (the compiled code) is reclaimed on the call to a custom
close() function.
Yes, I could skip the compilation step and load pre-written functions (as
a shared object) but the ability to compile a custom version, quickly (one
of the strengths of TCC) in memory and use it immediately, is not something
I want to give up here.
Case 2: Lua (http://www.lua.org/)
Just to recap Lua---it's a small scripting language intended to be
embedded in a larger application to give said application scripting
abilities. You can also load additional modules either written in Lua or
written in C (as shared objects, either .so for Unix or .dll for Windows).
Anyway, I wrote Lua bindings for libtcc [1], so from within a Lua script,
I can include C code, compile it and then call the newly compiled C code
from Lua (you can see an example of this starting with line 112 of [2]).
Just as with case 1, once the code is compiled, I don't care about the rest
of the TCCState---just the compiled code.
In fact, I use the libtcc bindings to extend Lua's require() function to
compile a Lua module upon loading it, and have Lua manage the memory for the
resulting code automatically (Lua is a garbage collected language).
[1] https://github.com/spc476/lua-conmanorg/blob/master/src/tcc.c
[2] https://github.com/spc476/lua-conmanorg/blob/master/lua/cc.lua
Case 3: Closures in C (or, a mixture of case 1 and case 2)
Why this exists doesn't matter, but I have written Lua bindings to Xlib
(the lowest level of X programming without talking the protocol directly).
Xlib, however, reports some errors via a callback function you register
using XSetErrorHandler():
int (*XSetErrorHandler(int (*handler)(Display *, XErrorEvent *)))();
I would like to push the error into Lua, but the issue I have is---I need
a global variable pointing to the current Lua state, which is something I
don't particularly want (the global variable), because the following:
int foo(lua_State *L)
{
XSetErrorHandler(int (Display *display,XEerrorEvent *event)
{
lua_pushlightuserdata(L,display);
lua_gettable(L,LUA_REGISTRYINDEX);
lua_getfield(L,-1,error_handler);
mylua_pushevent(L,event);
lua_call(L,1,0);
return 0;
}
)
return 0;
}
is not legal C. However, using libtcc, I can fake a closure with C:
#include stdlib.h
#include X11/Xlib.h
#include syslog.h
#include lua.h
#include lualib.h
#include lauxlib.h
lua_State *L;
int Xcallback_function(Display *display,XErrorEvent *event)
{
lua_pushlightuserdata(...);
/* ... */
}
and compile it with (excluding error checking for clarity):
lua_State *L; /* assume it's set properly */
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner pointed out: ... snip ... Case 2: Lua (http://www.lua.org/) Just to recap Lua---it's a small scripting language intended to be embedded in a larger application to give said application scripting abilities. You can also load additional modules either written in Lua or written in C (as shared objects, either .so for Unix or .dll for Windows). Anyway, I wrote Lua bindings for libtcc [1], so from within a Lua script, I can include C code, compile it and then call the newly compiled C code from Lua (you can see an example of this starting with line 112 of [2]). Just as with case 1, once the code is compiled, I don't care about the rest of the TCCState---just the compiled code. In fact, I use the libtcc bindings to extend Lua's require() function to compile a Lua module upon loading it, and have Lua manage the memory for the resulting code automatically (Lua is a garbage collected language). [1] https://github.com/spc476/lua-conmanorg/blob/master/src/tcc.c [2] https://github.com/spc476/lua-conmanorg/blob/master/lua/cc.lua I have been working, on-and-off, on Perl bindings for TCC. I have lamented that I have to carry around the whole compiler state after having compiled because, as Sean writes, cleaning up the compiler state also destroys the machine code. After the compile step, all I need is the machine code. If there were some way to manage the memory on my end, that'd be great. If tcc_relocate_ex can eliminate this concern, then I am *VERY* interested in knowing about it. :-) David -- Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian Kernighan ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Hi, On Mon, 27 Aug 2012, grischka wrote: Can you at least commit the change that makes tcc_relocate_ex() global? Yes, perhaps at least. You wrote: ... but fixing my code to match the current definition of tcc_relocate() was out of the question (the surrounding code manages the lifetime of both the TCCState and the resulting executable code compiled into memory). I'd actually like to know some details why you can't just use tcc_relocate()? So Sean meanwhile answered this extensively, but I'd actually ask back why the answer to this question matters? The _ex function exposes a strict superset of possibilities of the non _ex variant, that much is very obvious, so asking why the more limited API isn't enough for everyone, when it once was more capable and actually used in that extended fashion seems patronizing to me. There's clearly a use for the extended API (clearly in the sense, it once worked, now with the limited API it doesn't, and I don't see how to work around the limitation with the current API), and exporting the relevant, though renamed symbol, isn't affecting the future of libtcc stability, insofar it exists (which it doesn't), at all. So, while the complaint about the leak is correct, the conservatism about exporting one internal function (whose functionality once was available as an exported function that was removed from the API) is hypocritic at best. Sorry for that assessment. IOW: I don't understand your reservations. To make it short: just export the damn thing. It makes sense. Admittedly we know little about how people actually use the libtcc API. Well, we now know a little bit more. Ciao, Michael. ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: Okay, fine! I see it! It's a memory leak. Yes, an ANSI compliant Elbonian cross-complication IOW. Can you at least commit the change that makes tcc_relocate_ex() global? Yes, perhaps at least. You wrote: ... but fixing my code to match the current definition of tcc_relocate() was out of the question (the surrounding code manages the lifetime of both the TCCState and the resulting executable code compiled into memory). I'd actually like to know some details why you can't just use tcc_relocate()? Admittedly we know little about how people actually use the libtcc API. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: Hmmm ... I can see where that appears to be the case, but I did run valgrind and it reported no memory leaks. You see nothing. Try valgrind with ./tcc -run examples/ex1.c --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: Hmmm ... I can see where that appears to be the case, but I did run valgrind and it reported no memory leaks. You see nothing. Try valgrind with ./tcc -run examples/ex1.c Okay, fine! I see it! It's a memory leak. Can you at least commit the change that makes tcc_relocate_ex() global? -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: But again, how is what I did a memory leak? The statement s1-runtime_mem = NULL; in tcc_relocate_ex would discard the previous value. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: But again, how is what I did a memory leak? The statement s1-runtime_mem = NULL; in tcc_relocate_ex would discard the previous value. Hmmm ... I can see where that appears to be the case, but I did run valgrind and it reported no memory leaks. Anyway, at the very least, I would still like the promotion of tcc_relocate_ex() to public. I can live without the NULL assignment. -spc ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: It was thus said that the Great grischka once stated: Sean Conner wrote: As to the patch I think that +s1-runtime_mem = NULL; would break tcc_relocate and I don't see why you need it. I did that so that the call to tcc_delete() wouldn't crash when it tried to free s-runtime_mem that might contain a garbage value (since tcc_relocate_ex() would never set that field). Good thing too, because tcc_new() *might not* be initializing NULL pointers correctly [1]. Well, from my point of view that line would not fix tcc_new but cause a memory leak for tcc_relocate. I did a bit more work. The assignment of s1-runtime_mem to NULL does not lead to any memory leaks. On systems where the NULL pointer *is* all zeros (most, if not all, modern systems I am aware of) removing the assignment does not cause a crash (since the memory used to store the state is zeroed out). On systems where the NULL pointer *IS NOT* all zeros [1], removing the assignment *WILL CRASH*. Not might crash, *WILL CRASH*. [2] Well, behavior of free for invalid values is undefined. It might crash but might as well restart the computer or do nothing or ... Anyway, features and bugs are not different in that both are generated as side-effects from the written lines of code. I can't help and sorry and all but the feature that you wrote here is ... a 100% state of the art memory leak. However the most grave argument against this line is that it breaks assumptions, here the assumption that tcc_new *DOES* the right thing with initialization. You never ever put code to implement safety precaution against the failure of assumptions made elsewhere in the same program. See, on a system where memset doesn't clear pointers tcc or libtcc would fail already in tcc_new. It would never reach your safety line. --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Sean Conner wrote: As to the patch I think that +s1-runtime_mem = NULL; would break tcc_relocate and I don't see why you need it. I did that so that the call to tcc_delete() wouldn't crash when it tried to free s-runtime_mem that might contain a garbage value (since tcc_relocate_ex() would never set that field). Good thing too, because tcc_new() *might not* be initializing NULL pointers correctly [1]. Well, from my point of view that line would not fix tcc_new but cause a memory leak for tcc_relocate. -spc (But I would feel safer with the NULL pointer assignment ... ) (That's what I suspected ...) --- grischka ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
It was thus said that the Great grischka once stated: Sean Conner wrote: It was thus said that the Great Thomas Preud'homme once stated: Indeed, I agree the commit message is not very verbose as to the reason of the change. For people interested, the said commit is 8bbde91f62291cb0383c2406ae6123be903c944b That said, the change of API seems clearly intentional, as if 0.9.25's behavior had a fundamental problem. Thus, I'm reluctant to apply your patch right now. Grishka, could you tell us the context behind this revert? I don't think 0.9.25 had problems, since currently, tcc_relocate() calls tcc_relocate_ex() (if HAVE_SELINUX is not defined). The only thing I can think of is that a significant amount of code expected the 0.9.24 version of tcc_relocate() and it was easier to change 0.9.25 than to change the existing code, but that's just conjecture on my part. -spc (I did try to keep the patch as simple and isloated as possible) The reason to revert tcc_relocate was to make the libtcc_test example look as simple as it was before. The reason for not yet to export tcc_relocate_ex was that I wanted to know if anyone would actually care for the 'extended' version, i.e. to wait for someone to complain. :-) Well, it looks like it took a few years, but yes, I build code around the 0.9.25 version of tcc_relocate(). As to the patch I think that +s1-runtime_mem = NULL; would break tcc_relocate and I don't see why you need it. I did that so that the call to tcc_delete() wouldn't crash when it tried to free s-runtime_mem that might contain a garbage value (since tcc_relocate_ex() would never set that field). Good thing too, because tcc_new() *might not* be initializing NULL pointers correctly [1]. If you agree with I will push a patch to declare tcc_relocate_ex as LIBTCCAPI and nothing else. Well, I would like to see tcc_relocate_ex() promoted, so I agree to that. -spc (But I would feel safer with the NULL pointer assignment ... ) [1] Depending upon how pedantic you want to be, doing: char *p; memset(p,0,sizeof(p)); does not guarantee that p will be a valid NULL pointer. In ANSI C, a zero value in a pointer context, like: char *p = 0; will convert the '0' to a valid NULL pointer for the platform, which need not be an actual 0 value (it could very well be a bit pattern of alternating ones and zeros for some obscure Elbonia [4] CPU). A lot of popular architectures in use today *do* use all zero bits in a pointer as a valid NULL pointer, so mallocz() [2] will initialize pointers to a valid NULL value, but not ANSI C doesn't mandate that. Now, given the systems that TCC builds code for, this might not be an issue. [3] [2] The standard ANSI C function calloc() does this---why not use it? [3] Except for the odd Elbonian [4] trying to build a cross-compiler for x86 platforms. [4] http://en.wikipedia.org/wiki/Dilbert#Elbonia ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()
Le jeudi 9 août 2012 23:52:35, Sean Conner a écrit : I have some existing code that used the stock TCC 0.9.25 distribution. I was happy to find http://repo.or.cz/w/tinycc.git as it fixed some problems I had with stock TCC 0.9.25. But I was surprised that the API for tcc_relocate() was modified (and the log didn't reveal why the change happened) and that the change not only broke my existing code, but fixing my code to match the current definition of tcc_relocate() was out of the question (the surrounding code manages the lifetime of both the TCCState and the resulting executable code compiled into memory). To fix the problem, I patched the current version of TCC to promote tcc_relocate_ex() (which matches the original 0.9.25 behavior) to a public function. Yes, I won't be able to use SELINUX, but that wasn't an issue with me. Indeed, I agree the commit message is not very verbose as to the reason of the change. For people interested, the said commit is 8bbde91f62291cb0383c2406ae6123be903c944b That said, the change of API seems clearly intentional, as if 0.9.25's behavior had a fundamental problem. Thus, I'm reluctant to apply your patch right now. Grishka, could you tell us the context behind this revert? I tried following the directions to push my changes back into the mob branch, but the directions are a bit vague and they didn't work for me. Attached is an email patch. -spc (For the curious, I made a Lua module for TCC, and it's Lua that manages the memory for my application) Best regards, Thomas Preud'homme ___ Tinycc-devel mailing list Tinycc-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/tinycc-devel
