Re: [patch] changing lua_Number to int64_t
Am 17.11.13 22:03, schrieb Terry Moore: >> From: Marc Balmer [mailto:m...@msys.ch] >> >>> It's not *much* less safe than compiling and executing a string in the >>> kernel. The only additional attack surfaces are that you can write > things >>> that the compiler wouldn't write. This can (1) cause a crash at load > time, >>> or it can (2) cause surprising behavior later. >> >> The problem is that malicious bytecode in 5.1 is possible. That is why >> there is a guad against loading bytecode. > > Malicious in what sense? Is this a type-1 problem or a type-2 problem? Or > something else that I've not considered? See Peter Cawleys talk he gave during LWS 2011 in Frick. "Mitigating the danger of malicious bytecode Peter Cawley (University of Oxford) In Lua 5.1 and 5.2, the family of load functions permit the loading of precompiled bytecode in addition to plain source code. We shall look at some relevant details of the bytecode, and see how these details can be maliciously abused to escape from a Lua sandbox (in both 5.1 and 5.2). With the dangers presented, we shall then move to look at two ways of mitigating them. Firstly, one can wrap the load functions as to reject bytecode before it is loaded. Secondly, and more interestingly, one can attempt to algorithmically decide whether a given piece of bytecode is malicious or not. We shall look at two such algorithms for validating bytecode: LBCV, which tries to reject as little as possible, and my current experimental verifier, which tries to accept as little as possible. " http://www.lua.org/wshop11/Cawley.pdf
Re: [patch] changing lua_Number to int64_t
On Nov 17, 7:14pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | Humm.. I think that =A72.1 brings a good argument: "Standard Lua uses | 64-bit integers and double-precision floats, (...)". I think that | would not hurt to stick to the future standard; once 64 bit is good | enough for kernel purposes. Ok, christos
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 6:45 PM, Justin Cormack wrote: > On Sun, Nov 17, 2013 at 8:39 PM, Lourival Vieira Neto > wrote: >> Well, I don't think I fully understood that; mainly because I'm not >> aware about Lua 5.3. It will provide two number types for the scripts? >> Or you are just talking about lua_Integer type on the C-side. Lua 5.1 >> already has a lua_Integer type that is defined as ptrdiff_t. > > Yes that is correct, so 2 will be integer and 5.3 float, operations > will be defined in terms of how they convert, so there will be int and > float division. The draft manual is here > http://www.lua.org/work/doc/manual.html (see 3.4.1). This will not > happen for a while, but it will make it much easier in future for > interfaces like the kernel that need 64 bit int support, which is why > it is being implemented. So not being compatible with this seems a > mistake. Humm.. I think that ยง2.1 brings a good argument: "Standard Lua uses 64-bit integers and double-precision floats, (...)". I think that would not hurt to stick to the future standard; once 64 bit is good enough for kernel purposes. Regards, -- Lourival Vieira Neto
RE: [patch] changing lua_Number to int64_t
> From: Marc Balmer [mailto:m...@msys.ch] > > > It's not *much* less safe than compiling and executing a string in the > > kernel. The only additional attack surfaces are that you can write things > > that the compiler wouldn't write. This can (1) cause a crash at load time, > > or it can (2) cause surprising behavior later. > > The problem is that malicious bytecode in 5.1 is possible. That is why > there is a guad against loading bytecode. Malicious in what sense? Is this a type-1 problem or a type-2 problem? Or something else that I've not considered? --Terry
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 8:39 PM, Lourival Vieira Neto wrote: > Well, I don't think I fully understood that; mainly because I'm not > aware about Lua 5.3. It will provide two number types for the scripts? > Or you are just talking about lua_Integer type on the C-side. Lua 5.1 > already has a lua_Integer type that is defined as ptrdiff_t. Yes that is correct, so 2 will be integer and 5.3 float, operations will be defined in terms of how they convert, so there will be int and float division. The draft manual is here http://www.lua.org/work/doc/manual.html (see 3.4.1). This will not happen for a while, but it will make it much easier in future for interfaces like the kernel that need 64 bit int support, which is why it is being implemented. So not being compatible with this seems a mistake. Justin
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 6:13 PM, Justin Cormack wrote: > On Sun, Nov 17, 2013 at 7:56 PM, Christos Zoulas wrote: >> On Nov 17, 3:36pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: >> -- Subject: Re: [patch] changing lua_Number to int64_t >> >> | > 1. Lua 5.3 will have 64 bit integer support as standard, which will >> | > make interop and reuse between kernel and userspace code much easier, >> | > iff we use int64_t >> | >> | If they are using int64_t for integers, I think it is a good reason to us >> to >> | stick to int64_t. >> >> This is not relevant. The numeric type will still be double, so forget >> about compatibility between kernel and userland. There is no need for >> the interpreter to use a fixed width type, but rather it is convenient >> to use the largest numeric type the machine can represent. > > There will be two numeric types as standard, int64_t and double. It > should be possible to compile the kernel Lua with only int64_t and no > double support I would think, so integer only userland programs would > be compatible which is a very useful feature. But the semantics of the > Lua integer type will be such that it wraps at 64 bit, unlike some > hypothetical larger type (that doesn't yet exist and which the kernel > doesn't yet use). Well, I don't think I fully understood that; mainly because I'm not aware about Lua 5.3. It will provide two number types for the scripts? Or you are just talking about lua_Integer type on the C-side. Lua 5.1 already has a lua_Integer type that is defined as ptrdiff_t. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 7:56 PM, Christos Zoulas wrote: > On Nov 17, 3:36pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: > -- Subject: Re: [patch] changing lua_Number to int64_t > > | > 1. Lua 5.3 will have 64 bit integer support as standard, which will > | > make interop and reuse between kernel and userspace code much easier, > | > iff we use int64_t > | > | If they are using int64_t for integers, I think it is a good reason to us to > | stick to int64_t. > > This is not relevant. The numeric type will still be double, so forget > about compatibility between kernel and userland. There is no need for > the interpreter to use a fixed width type, but rather it is convenient > to use the largest numeric type the machine can represent. There will be two numeric types as standard, int64_t and double. It should be possible to compile the kernel Lua with only int64_t and no double support I would think, so integer only userland programs would be compatible which is a very useful feature. But the semantics of the Lua integer type will be such that it wraps at 64 bit, unlike some hypothetical larger type (that doesn't yet exist and which the kernel doesn't yet use). Justin
Re: [patch] changing lua_Number to int64_t
On Nov 17, 3:36pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | > 1. Lua 5.3 will have 64 bit integer support as standard, which will | > make interop and reuse between kernel and userspace code much easier, | > iff we use int64_t | | If they are using int64_t for integers, I think it is a good reason to us to | stick to int64_t. This is not relevant. The numeric type will still be double, so forget about compatibility between kernel and userland. There is no need for the interpreter to use a fixed width type, but rather it is convenient to use the largest numeric type the machine can represent. christos
Re: [patch] changing lua_Number to int64_t
On Sat, Nov 16, 2013 at 9:25 PM, Lourival Vieira Neto wrote: > (...) I moved strtoimax.c to common/libc. Don't know if > someone sees a problem on that. BTW, is it OK? Could someone review this? Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 3:10 PM, Justin Cormack wrote: > On Sun, Nov 17, 2013 at 4:52 PM, Lourival Vieira Neto > wrote: >> On Sun, Nov 17, 2013 at 2:02 PM, Christos Zoulas wrote: >>> On Nov 17, 10:46am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: >>> -- Subject: Re: [patch] changing lua_Number to int64_t >>> >>> | On Sun, Nov 17, 2013 at 7:37 AM, Marc Balmer wrote: >>> | > Am 17.11.13 04:49, schrieb Terry Moore: >>> | >> I believe that if you want the Lua scripts to be portable across NetBSD >>> | >> deployments, you should choose a well-known fixed width. >>> | > >>> | > I don't see this as very important. Lua scripts will hardly depend on >>> | > the size of an integer. >>> | >>> | But they could. I think that the script programmers should know if the >>> | numeric data type is enough for their usage (e.g., time diffs). >>> >>> By making it the biggest type possible, you never need to be worried. >> >> Right.. you just convinced me.. if no one opposes, I'll change that to >> intmax_t and get rid of PRI/SCNd64 =). > > 1. Lua 5.3 will have 64 bit integer support as standard, which will > make interop and reuse between kernel and userspace code much easier, > iff we use int64_t If they are using int64_t for integers, I think it is a good reason to us to stick to int64_t. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 4:52 PM, Lourival Vieira Neto wrote: > On Sun, Nov 17, 2013 at 2:02 PM, Christos Zoulas wrote: >> On Nov 17, 10:46am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: >> -- Subject: Re: [patch] changing lua_Number to int64_t >> >> | On Sun, Nov 17, 2013 at 7:37 AM, Marc Balmer wrote: >> | > Am 17.11.13 04:49, schrieb Terry Moore: >> | >> I believe that if you want the Lua scripts to be portable across NetBSD >> | >> deployments, you should choose a well-known fixed width. >> | > >> | > I don't see this as very important. Lua scripts will hardly depend on >> | > the size of an integer. >> | >> | But they could. I think that the script programmers should know if the >> | numeric data type is enough for their usage (e.g., time diffs). >> >> By making it the biggest type possible, you never need to be worried. > > Right.. you just convinced me.. if no one opposes, I'll change that to > intmax_t and get rid of PRI/SCNd64 =). 1. Lua 5.3 will have 64 bit integer support as standard, which will make interop and reuse between kernel and userspace code much easier, iff we use int64_t 2. Code will have to handle use in the kernel of uint64_t which will potentially behave differently with Lua number type being int64_t vs if it was sat int128_t which might happen, such that its unlikely to be tested properly. There is no existing system where intmax_t is not int64_t so this breakage is untestable. Justin
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 2:02 PM, Christos Zoulas wrote: > On Nov 17, 10:46am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: > -- Subject: Re: [patch] changing lua_Number to int64_t > > | On Sun, Nov 17, 2013 at 7:37 AM, Marc Balmer wrote: > | > Am 17.11.13 04:49, schrieb Terry Moore: > | >> I believe that if you want the Lua scripts to be portable across NetBSD > | >> deployments, you should choose a well-known fixed width. > | > > | > I don't see this as very important. Lua scripts will hardly depend on > | > the size of an integer. > | > | But they could. I think that the script programmers should know if the > | numeric data type is enough for their usage (e.g., time diffs). > > By making it the biggest type possible, you never need to be worried. Right.. you just convinced me.. if no one opposes, I'll change that to intmax_t and get rid of PRI/SCNd64 =). Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Nov 17, 10:46am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | On Sun, Nov 17, 2013 at 7:37 AM, Marc Balmer wrote: | > Am 17.11.13 04:49, schrieb Terry Moore: | >> I believe that if you want the Lua scripts to be portable across NetBSD | >> deployments, you should choose a well-known fixed width. | > | > I don't see this as very important. Lua scripts will hardly depend on | > the size of an integer. | | But they could. I think that the script programmers should know if the | numeric data type is enough for their usage (e.g., time diffs). By making it the biggest type possible, you never need to be worried. christos
Re: [patch] changing lua_Number to int64_t
On Nov 17, 10:36am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | I mean know it as a script programmer. I think that would be helpful | to know the exact lua_Number width when you are writing a script. | AFAIK, you don't have sizeof functionality from Lua. So, IMHO, | lua_Number width should be fixed and documented. Lua should provide manifest constants for it (like INTMAX_MAX). Otherwise you'd be making assumptions christos
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 12:58 PM, Christos Zoulas wrote: > On Nov 17, 1:36am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: > -- Subject: Re: [patch] changing lua_Number to int64_t > > | > Bigger is better. And you can use %jd to print which is a big win. > | > | I agree that bigger is better and %jd is much better then "%" PRI/SCN. > | But don't you think that to know the exact width is even better? > > Why? You can always compute it at runtime if you need to. Yes, but I think that it is not a common practice in Lua. AFAIK, Lua has only math.huge [1] to tell what is the biggest number value available (what isn't the same of having the lua_Number width). Thus, we would need to provide an interface to return this information. I'd prefer to have it defined in compile time independently of platform. Anyway, as I stated before, I have no strong feelings about that.. if everybody else prefer intmax_t, I have no objection. [1] http://www.lua.org/manual/5.1/manual.html#pdf-math.huge Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Nov 17, 1:36am, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | > Bigger is better. And you can use %jd to print which is a big win. | | I agree that bigger is better and %jd is much better then "%" PRI/SCN. | But don't you think that to know the exact width is even better? Why? You can always compute it at runtime if you need to. christos
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 10:10 AM, Justin Cormack wrote: > On Sun, Nov 17, 2013 at 11:30 AM, Alexander Nasonov wrote: >> Mouse wrote: >>> Also, using an exact-width type assumes that the hardware/compiler in >>> question _has_ such a type. >>> >>> It's possible that lua, NetBSD, or the combination of the two is >>> willing to write off portability to machines where one or both of those >>> potential portability issues becomes actual. But that seems to be >>> asking for trouble to me; history is full of "but nobody will ever want >>> to port this to one of _those_" that come back to bite people. >> >> I was perfectly fine with long long because it's long enough to >> represent all integers in range [-2^53-1, 2^53-1]. >> >> As Marc pointed out, Lua has a single numeric type which is double >> by default. Many Lua libraries don't need FP and they use a subset of >> exactly representable integers (not all of them do range checks, though). >> Extending the range when porting from userspace to kernel will decrease >> "the pain factor" of porting. > > The range [-2^53-1, 2^53-1] is not sufficient - in kernel you need to > be able to deal with the longest type the kernel uses, it is > incredibly annoying to have to use userdata to deal with off_t or > gratuitously hope that losing some bits is ok (as happens with Lua in > userspace now). As the widest type in the kernel is int64_t thats > what Lua should use. (The issue of uint64_t is left as an exercise for > the Lua programmer). When/if the kernel uses something longer then Lua > can change, but using intmax_t is not useful as the kernel is explicit > about sizes. Indeed. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 9:30 AM, Alexander Nasonov wrote: > Mouse wrote: >> Also, using an exact-width type assumes that the hardware/compiler in >> question _has_ such a type. >> >> It's possible that lua, NetBSD, or the combination of the two is >> willing to write off portability to machines where one or both of those >> potential portability issues becomes actual. But that seems to be >> asking for trouble to me; history is full of "but nobody will ever want >> to port this to one of _those_" that come back to bite people. > > I was perfectly fine with long long because it's long enough to > represent all integers in range [-2^53-1, 2^53-1]. > > As Marc pointed out, Lua has a single numeric type which is double > by default. Many Lua libraries don't need FP and they use a subset of > exactly representable integers (not all of them do range checks, though). > Extending the range when porting from userspace to kernel will decrease > "the pain factor" of porting. I think that it is not the main point here, to decrease the "pain factor" of porting. Porting Lua user-space libraries should be painful enough independently of lua_Number type. IMHO, the main point here is to define a lua_Number type adjusted to kernel needs. As, we have already broke compatibility with user space libraries (for several factors, not only the missing of floating point number), I think it shouldn't matter much now. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 8:52 AM, Alexander Nasonov wrote: > Marc Balmer wrote: >> The basic issue here is that Lua has only _one_ numerical data type, >> which is an integral type in kernel, but a floating point type in userspace. > > Right, not everyone here knows this I guess. Thanks for making it clear. Sorry, I thought that it was clear. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 7:37 AM, Marc Balmer wrote: > Am 17.11.13 04:49, schrieb Terry Moore: >> I believe that if you want the Lua scripts to be portable across NetBSD >> deployments, you should choose a well-known fixed width. > > I don't see this as very important. Lua scripts will hardly depend on > the size of an integer. But they could. I think that the script programmers should know if the numeric data type is enough for their usage (e.g., time diffs). Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 7:35 AM, Marc Balmer wrote: > Am 17.11.13 04:36, schrieb Lourival Vieira Neto: >> On Sat, Nov 16, 2013 at 10:44 PM, Christos Zoulas >> wrote: >>> On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: >>> -- Subject: Re: [patch] changing lua_Number to int64_t >>> >>> | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas >>> wrote: >>> | > In article <52872b0c.5080...@msys.ch>, Marc Balmer >>> wrote: >>> | >>Changing the number type to int64_t is certainly a good idea. Two >>> | >>questions, however: >>> | > >>> | > Why not intmax_t? >>> | >>> | My only argument is that int64_t has a well-defined width and, AFAIK, >>> | intmax_t could vary. But I have no strong feelings about this. Do you >>> | think intmax_t would be better? >>> >>> Bigger is better. And you can use %jd to print which is a big win. >> >> I agree that bigger is better and %jd is much better then "%" PRI/SCN. >> But don't you think that to know the exact width is even better? > > You can always use sizeof if the need to know the size arises. I mean know it as a script programmer. I think that would be helpful to know the exact lua_Number width when you are writing a script. AFAIK, you don't have sizeof functionality from Lua. So, IMHO, lua_Number width should be fixed and documented. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, 17 Nov 2013, Mouse wrote: sizeof returns the number of bytes used to store an object. This is only loosely related to the number of data bits in the object; the latter is no more than sizeof the object times CHAR_BIT, but it may be lower. Also, using an exact-width type assumes that the hardware/compiler in question _has_ such a type. Yes, that's true of C. It's possible that lua, NetBSD, or the combination of the two is willing to write off portability to machines where one or both of those potential portability issues becomes actual. But that seems to be asking for trouble to me; history is full of "but nobody will ever want to port this to one of _those_" that come back to bite people. NetBSD already assumes that char is exactly 8 bits, and that integer types with exactly 16, 32, and 64 bits exist. Adding more instances of the same assumptions doesn't seem like a big problem to me. If there's ever a need to port to a machine where those assumptions do not hold, then we can worry about it at that time, but I susect that it will be possible to change to using things like int_least64_t (for a type with no less than 64 bits) instead of int64_t (for a type with exactly 64 bits). --apb (Alan Barrett)
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 11:30 AM, Alexander Nasonov wrote: > Mouse wrote: >> Also, using an exact-width type assumes that the hardware/compiler in >> question _has_ such a type. >> >> It's possible that lua, NetBSD, or the combination of the two is >> willing to write off portability to machines where one or both of those >> potential portability issues becomes actual. But that seems to be >> asking for trouble to me; history is full of "but nobody will ever want >> to port this to one of _those_" that come back to bite people. > > I was perfectly fine with long long because it's long enough to > represent all integers in range [-2^53-1, 2^53-1]. > > As Marc pointed out, Lua has a single numeric type which is double > by default. Many Lua libraries don't need FP and they use a subset of > exactly representable integers (not all of them do range checks, though). > Extending the range when porting from userspace to kernel will decrease > "the pain factor" of porting. The range [-2^53-1, 2^53-1] is not sufficient - in kernel you need to be able to deal with the longest type the kernel uses, it is incredibly annoying to have to use userdata to deal with off_t or gratuitously hope that losing some bits is ok (as happens with Lua in userspace now). As the widest type in the kernel is int64_t thats what Lua should use. (The issue of uint64_t is left as an exercise for the Lua programmer). When/if the kernel uses something longer then Lua can change, but using intmax_t is not useful as the kernel is explicit about sizes. Justin
Re: [patch] changing lua_Number to int64_t
Mouse wrote: > Also, using an exact-width type assumes that the hardware/compiler in > question _has_ such a type. > > It's possible that lua, NetBSD, or the combination of the two is > willing to write off portability to machines where one or both of those > potential portability issues becomes actual. But that seems to be > asking for trouble to me; history is full of "but nobody will ever want > to port this to one of _those_" that come back to bite people. I was perfectly fine with long long because it's long enough to represent all integers in range [-2^53-1, 2^53-1]. As Marc pointed out, Lua has a single numeric type which is double by default. Many Lua libraries don't need FP and they use a subset of exactly representable integers (not all of them do range checks, though). Extending the range when porting from userspace to kernel will decrease "the pain factor" of porting. Alex
Re: [patch] changing lua_Number to int64_t
Marc Balmer wrote: > The basic issue here is that Lua has only _one_ numerical data type, > which is an integral type in kernel, but a floating point type in userspace. Right, not everyone here knows this I guess. Thanks for making it clear. > > Also, some libraries work in both spaces. Lua binding for proplib(3) is > > a good example. > > Yes, if one keeps above issue in mind, a script can be written for both > use cases. speaking of a proplib(3) binding, is that code available > somewhere? I'm not aware of any binding for proplib(3). It would nice to have it in NetBSD. Alex
Re: [patch] changing lua_Number to int64_t
Am 17.11.13 11:18, schrieb Alexander Nasonov: > Lourival Vieira Neto wrote: >> I don't think that keeping compatibility with userspace Lua is the >> right argument. We already have lost this kind of compatibility by >> using an integer type for lua_Number. Expecting that kernel lua_Number >> works just like userspace lua_Numbers could lead to misunderstandings. > > Portability with userspace will arise as soon as you decide to import > some Lua library to kernel space. If kernel number type is wide enough > to represent all exactly representable integers from userspace, you have > less worries. The basic issue here is that Lua has only _one_ numerical data type, which is an integral type in kernel, but a floating point type in userspace. > Also, some libraries work in both spaces. Lua binding for proplib(3) is > a good example. Yes, if one keeps above issue in mind, a script can be written for both use cases. speaking of a proplib(3) binding, is that code available somewhere? > >>> I don't see a need for bigger type unless mainstream Lua switches to >>> long double. I don't expect it to happen any time soon. >> >> Why it should bother if Lua switches to a bigger floating-point type? > > See above. If they switch to long double, they will extend integer > arithmetic range. > >>> PPS "%"PRId64 may break in C++11, space between the literals should fix it. >> >> I don't think that are plans to change kernel language to C++ ;-), >> however doesn't hurt to write it in "clean" C. >> >> Just for curiosity.. do you know why it is not allowed in C++11? > > I'm not sure why. It may be related to user-defined literals: > http://akrzemi1.wordpress.com/2012/08/12/user-defined-literals-part-i/
Re: [patch] changing lua_Number to int64_t
Lourival Vieira Neto wrote: > I don't think that keeping compatibility with userspace Lua is the > right argument. We already have lost this kind of compatibility by > using an integer type for lua_Number. Expecting that kernel lua_Number > works just like userspace lua_Numbers could lead to misunderstandings. Portability with userspace will arise as soon as you decide to import some Lua library to kernel space. If kernel number type is wide enough to represent all exactly representable integers from userspace, you have less worries. Also, some libraries work in both spaces. Lua binding for proplib(3) is a good example. > > I don't see a need for bigger type unless mainstream Lua switches to > > long double. I don't expect it to happen any time soon. > > Why it should bother if Lua switches to a bigger floating-point type? See above. If they switch to long double, they will extend integer arithmetic range. > > PPS "%"PRId64 may break in C++11, space between the literals should fix it. > > I don't think that are plans to change kernel language to C++ ;-), > however doesn't hurt to write it in "clean" C. > > Just for curiosity.. do you know why it is not allowed in C++11? I'm not sure why. It may be related to user-defined literals: http://akrzemi1.wordpress.com/2012/08/12/user-defined-literals-part-i/ Alex
Re: [patch] changing lua_Number to int64_t
>> I agree that bigger is better and %jd is much better then "%" >> PRI/SCN. But don't you think that to know the exact width is even >> better? > You can always use sizeof if the need to know the size arises. sizeof returns the number of bytes used to store an object. This is only loosely related to the number of data bits in the object; the latter is no more than sizeof the object times CHAR_BIT, but it may be lower. Also, using an exact-width type assumes that the hardware/compiler in question _has_ such a type. It's possible that lua, NetBSD, or the combination of the two is willing to write off portability to machines where one or both of those potential portability issues becomes actual. But that seems to be asking for trouble to me; history is full of "but nobody will ever want to port this to one of _those_" that come back to bite people. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: [patch] changing lua_Number to int64_t
Am 17.11.13 06:30, schrieb Terry Moore: >> From: Lourival Vieira Neto [mailto:lourival.n...@gmail.com] >>> Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, > but >>> you may find if not careful that the compiled binary is not loadable on >>> machines with different choices for LP64, ILP32, etc. This is somewhat >>> independent of the choice of lua_Number mapping. >> >> Yes, Lua bytecode isn't portable in fact. BTW, loading Lua bytecode is >> not recommended by Lua team. It isn't safe. > > It's not *much* less safe than compiling and executing a string in the > kernel. The only additional attack surfaces are that you can write things > that the compiler wouldn't write. This can (1) cause a crash at load time, > or it can (2) cause surprising behavior later. The problem is that malicious bytecode in 5.1 is possible. That is why there is a guad against loading bytecode. > I suspect that anyone who would want to allow Lua in the kernel would be > somewhat indifferent to type-2 safety issues. Only if loading precompiled chunks, which btw needs a custom version of luac. You can not compile bytecode for in-kernel use using the standard luac(1) program. If you don't trust the input > string, you had better not give it to the kernel (whether or not it's > compiled). Type-1 safety issues ought to be fixed, as careful examination of > the input data only slows down the load process -- it has no run-time > effects. > > Given that Lua byte codes can be translated into C arrays and then loaded > via the API, portability issues can creep in through the back door. > > Best regards, > --Terry >
Re: [patch] changing lua_Number to int64_t
Am 17.11.13 04:49, schrieb Terry Moore: > I believe that if you want the Lua scripts to be portable across NetBSD > deployments, you should choose a well-known fixed width. I don't see this as very important. Lua scripts will hardly depend on the size of an integer. > Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, but > you may find if not careful that the compiled binary is not loadable on > machines with different choices for LP64, ILP32, etc. This is somewhat > independent of the choice of lua_Number mapping. Bytecode is not portable. Not even between userland and kernel Lua on the same machine. > > --Terry > >> -Original Message- >> From: tech-kern-ow...@netbsd.org [mailto:tech-kern-ow...@netbsd.org] On >> Behalf Of Lourival Vieira Neto >> Sent: Saturday, November 16, 2013 22:36 >> To: Christos Zoulas >> Cc: tech-kern@netbsd.org >> Subject: Re: [patch] changing lua_Number to int64_t >> >> On Sat, Nov 16, 2013 at 10:44 PM, Christos Zoulas >> wrote: >>> On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) > wrote: >>> -- Subject: Re: [patch] changing lua_Number to int64_t >>> >>> | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas >> wrote: >>> | > In article <52872b0c.5080...@msys.ch>, Marc Balmer >> wrote: >>> | >>Changing the number type to int64_t is certainly a good idea. Two >>> | >>questions, however: >>> | > >>> | > Why not intmax_t? >>> | >>> | My only argument is that int64_t has a well-defined width and, AFAIK, >>> | intmax_t could vary. But I have no strong feelings about this. Do you >>> | think intmax_t would be better? >>> >>> Bigger is better. And you can use %jd to print which is a big win. >> >> I agree that bigger is better and %jd is much better then "%" PRI/SCN. >> But don't you think that to know the exact width is even better? >> >> Regards, >> -- >> Lourival Vieira Neto >
Re: [patch] changing lua_Number to int64_t
Am 17.11.13 00:47, schrieb Alexander Nasonov: > PS Why do you still use a shadow copy of luaconf.h? Please add your > changes to the main luaconf.h. If you guard your kernel changes properly > with _KERNEL, they will not affect userspace. I totally agree. It was probably just an oversight when I committed the code. It does, however, not make much sense to fix a lot in 5.1, since the move to 5.2 should happen soon.
Re: [patch] changing lua_Number to int64_t
Am 17.11.13 04:36, schrieb Lourival Vieira Neto: > On Sat, Nov 16, 2013 at 10:44 PM, Christos Zoulas wrote: >> On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: >> -- Subject: Re: [patch] changing lua_Number to int64_t >> >> | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas >> wrote: >> | > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: >> | >>Changing the number type to int64_t is certainly a good idea. Two >> | >>questions, however: >> | > >> | > Why not intmax_t? >> | >> | My only argument is that int64_t has a well-defined width and, AFAIK, >> | intmax_t could vary. But I have no strong feelings about this. Do you >> | think intmax_t would be better? >> >> Bigger is better. And you can use %jd to print which is a big win. > > I agree that bigger is better and %jd is much better then "%" PRI/SCN. > But don't you think that to know the exact width is even better? You can always use sizeof if the need to know the size arises.
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 3:30 AM, Terry Moore wrote: >> From: Lourival Vieira Neto [mailto:lourival.n...@gmail.com] >> > Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, > but >> > you may find if not careful that the compiled binary is not loadable on >> > machines with different choices for LP64, ILP32, etc. This is somewhat >> > independent of the choice of lua_Number mapping. >> >> Yes, Lua bytecode isn't portable in fact. BTW, loading Lua bytecode is >> not recommended by Lua team. It isn't safe. > > It's not *much* less safe than compiling and executing a string in the > kernel. The only additional attack surfaces are that you can write things > that the compiler wouldn't write. This can (1) cause a crash at load time, > or it can (2) cause surprising behavior later. > > I suspect that anyone who would want to allow Lua in the kernel would be > somewhat indifferent to type-2 safety issues. If you don't trust the input > string, you had better not give it to the kernel (whether or not it's > compiled). Type-1 safety issues ought to be fixed, as careful examination of > the input data only slows down the load process -- it has no run-time > effects. > > Given that Lua byte codes can be translated into C arrays and then loaded > via the API, portability issues can creep in through the back door. > > Best regards, > --Terry Sorry. I think I didn't make myself clear here. By 'it isn't safe' I meant that loading bytecode is not _reliable_, unless you have compiled it in the same Lua build that you are using to run. Otherwise, It can lead to malfunctioning both in kernel or user space. I was not referring myself to possible attacks. Note, that this is not all about portability among platforms, bytecode is not portable among different builds of Lua interpreter. In fact, Lua interpreter has no compromise with bytecode exchanging; the authors' idea is to distribute Lua source code instead of bytecode (AFAIK). I support that is not a good idea to compile a Lua script in some Lua interpreter build and then run it on another one (even on the same platform). Moreover, there is no significative difference in the time cost to load strings or byte codes. Lua compiling time is really fast (even comparing to the loading time of byte code). Regards, -- Lourival Vieira Neto
RE: [patch] changing lua_Number to int64_t
> From: Lourival Vieira Neto [mailto:lourival.n...@gmail.com] > > Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, but > > you may find if not careful that the compiled binary is not loadable on > > machines with different choices for LP64, ILP32, etc. This is somewhat > > independent of the choice of lua_Number mapping. > > Yes, Lua bytecode isn't portable in fact. BTW, loading Lua bytecode is > not recommended by Lua team. It isn't safe. It's not *much* less safe than compiling and executing a string in the kernel. The only additional attack surfaces are that you can write things that the compiler wouldn't write. This can (1) cause a crash at load time, or it can (2) cause surprising behavior later. I suspect that anyone who would want to allow Lua in the kernel would be somewhat indifferent to type-2 safety issues. If you don't trust the input string, you had better not give it to the kernel (whether or not it's compiled). Type-1 safety issues ought to be fixed, as careful examination of the input data only slows down the load process -- it has no run-time effects. Given that Lua byte codes can be translated into C arrays and then loaded via the API, portability issues can creep in through the back door. Best regards, --Terry
RE: [patch] changing lua_Number to int64_t
I believe that if you want the Lua scripts to be portable across NetBSD deployments, you should choose a well-known fixed width. Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, but you may find if not careful that the compiled binary is not loadable on machines with different choices for LP64, ILP32, etc. This is somewhat independent of the choice of lua_Number mapping. --Terry > -Original Message- > From: tech-kern-ow...@netbsd.org [mailto:tech-kern-ow...@netbsd.org] On > Behalf Of Lourival Vieira Neto > Sent: Saturday, November 16, 2013 22:36 > To: Christos Zoulas > Cc: tech-kern@netbsd.org > Subject: Re: [patch] changing lua_Number to int64_t > > On Sat, Nov 16, 2013 at 10:44 PM, Christos Zoulas > wrote: > > On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: > > -- Subject: Re: [patch] changing lua_Number to int64_t > > > > | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas > wrote: > > | > In article <52872b0c.5080...@msys.ch>, Marc Balmer > wrote: > > | >>Changing the number type to int64_t is certainly a good idea. Two > > | >>questions, however: > > | > > > | > Why not intmax_t? > > | > > | My only argument is that int64_t has a well-defined width and, AFAIK, > > | intmax_t could vary. But I have no strong feelings about this. Do you > > | think intmax_t would be better? > > > > Bigger is better. And you can use %jd to print which is a big win. > > I agree that bigger is better and %jd is much better then "%" PRI/SCN. > But don't you think that to know the exact width is even better? > > Regards, > -- > Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sun, Nov 17, 2013 at 1:49 AM, Terry Moore wrote: > I believe that if you want the Lua scripts to be portable across NetBSD > deployments, you should choose a well-known fixed width. Indeed. > Watch out, by the way, for compiled scripts; I have not checked Lua 5.x, but > you may find if not careful that the compiled binary is not loadable on > machines with different choices for LP64, ILP32, etc. This is somewhat > independent of the choice of lua_Number mapping. Yes, Lua bytecode isn't portable in fact. BTW, loading Lua bytecode is not recommended by Lua team. It isn't safe. Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sat, Nov 16, 2013 at 10:44 PM, Christos Zoulas wrote: > On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: > -- Subject: Re: [patch] changing lua_Number to int64_t > > | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas > wrote: > | > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: > | >>Changing the number type to int64_t is certainly a good idea. Two > | >>questions, however: > | > > | > Why not intmax_t? > | > | My only argument is that int64_t has a well-defined width and, AFAIK, > | intmax_t could vary. But I have no strong feelings about this. Do you > | think intmax_t would be better? > > Bigger is better. And you can use %jd to print which is a big win. I agree that bigger is better and %jd is much better then "%" PRI/SCN. But don't you think that to know the exact width is even better? Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sat, Nov 16, 2013 at 9:47 PM, Alexander Nasonov wrote: > Lourival Vieira Neto wrote: >> On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas wrote: >> > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: >> >>Changing the number type to int64_t is certainly a good idea. Two >> >>questions, however: >> > >> > Why not intmax_t? >> >> My only argument is that int64_t has a well-defined width and, AFAIK, >> intmax_t could vary. But I have no strong feelings about this. Do you >> think intmax_t would be better? > > int64_t should be enough to cover a range of exactly representable > integers in userspace Lua program where lua_Number is double. I don't think that keeping compatibility with userspace Lua is the right argument. We already have lost this kind of compatibility by using an integer type for lua_Number. Expecting that kernel lua_Number works just like userspace lua_Numbers could lead to misunderstandings. > I don't see a need for bigger type unless mainstream Lua switches to > long double. I don't expect it to happen any time soon. Why it should bother if Lua switches to a bigger floating-point type? > PS Why do you still use a shadow copy of luaconf.h? Please add your > changes to the main luaconf.h. Only because it is the way that it was committed. I didn't take time yet to unify that. But you are right, I'll try to do that in the future. > If you guard your kernel changes properly > with _KERNEL, they will not affect userspace. Yes, I know. I think the guards are just fine. > PPS "%"PRId64 may break in C++11, space between the literals should fix it. I don't think that are plans to change kernel language to C++ ;-), however doesn't hurt to write it in "clean" C. Just for curiosity.. do you know why it is not allowed in C++11? Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Nov 16, 9:30pm, lourival.n...@gmail.com (Lourival Vieira Neto) wrote: -- Subject: Re: [patch] changing lua_Number to int64_t | On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas wrote: | > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: | >>Changing the number type to int64_t is certainly a good idea. Two | >>questions, however: | > | > Why not intmax_t? | | My only argument is that int64_t has a well-defined width and, AFAIK, | intmax_t could vary. But I have no strong feelings about this. Do you | think intmax_t would be better? Bigger is better. And you can use %jd to print which is a big win. christos
Re: [patch] changing lua_Number to int64_t
Lourival Vieira Neto wrote: > On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas wrote: > > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: > >>Changing the number type to int64_t is certainly a good idea. Two > >>questions, however: > > > > Why not intmax_t? > > My only argument is that int64_t has a well-defined width and, AFAIK, > intmax_t could vary. But I have no strong feelings about this. Do you > think intmax_t would be better? int64_t should be enough to cover a range of exactly representable integers in userspace Lua program where lua_Number is double. I don't see a need for bigger type unless mainstream Lua switches to long double. I don't expect it to happen any time soon. PS Why do you still use a shadow copy of luaconf.h? Please add your changes to the main luaconf.h. If you guard your kernel changes properly with _KERNEL, they will not affect userspace. PPS "%"PRId64 may break in C++11, space between the literals should fix it. Alex
Re: [patch] changing lua_Number to int64_t
On Sat, Nov 16, 2013 at 8:52 PM, Christos Zoulas wrote: > In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: >>Changing the number type to int64_t is certainly a good idea. Two >>questions, however: > > Why not intmax_t? My only argument is that int64_t has a well-defined width and, AFAIK, intmax_t could vary. But I have no strong feelings about this. Do you think intmax_t would be better? Regards, -- Lourival Vieira Neto
Re: [patch] changing lua_Number to int64_t
On Sat, Nov 16, 2013 at 6:21 AM, Marc Balmer wrote:
> Changing the number type to int64_t is certainly a good idea. Two
> questions, however:
>
> 1) Why do you remove the sys/modules/lua/inttypes.g file?
Because this place holder is no long necessary. I have just replaced
that for sys/inttypes.h (adding -I${S}/sys to CPPFLAGS).
> 2) In sys/modules/lua/luaconf.h, lua_str2number is still #defined as
> stroll(), which assumes long long. Shouldn't that be changed as well to
> a function taking int64_t as argument?
Yes, my bad; sorry, I forgot about str2number. I've adjusted the
patch. Now, I'm using (int64_t) strtoimax() instead of strtoll(). I
think it should be alright because intmax_t is always greater or equal
than int64_t. I moved strtoimax.c to common/libc. Don't know if
someone sees a problem on that.
The patch is attached.
Regards,
--
Lourival Vieira Neto
lua_kernel_int64_t-3.patch
Description: Binary data
Re: [patch] changing lua_Number to int64_t
In article <52872b0c.5080...@msys.ch>, Marc Balmer wrote: >Changing the number type to int64_t is certainly a good idea. Two >questions, however: Why not intmax_t? christos
Re: [patch] changing lua_Number to int64_t
Changing the number type to int64_t is certainly a good idea. Two questions, however: 1) Why do you remove the sys/modules/lua/inttypes.g file? 2) In sys/modules/lua/luaconf.h, lua_str2number is still #defined as stroll(), which assumes long long. Shouldn't that be changed as well to a function taking int64_t as argument?
Re: [patch] changing lua_Number to int64_t
fixed a little issue.. (s/LUA_NUMBER_SCAN "%"PRId64/LUA_NUMBER_SCAN "%"SCNd64).. sorry about that.. =/ On Sat, Nov 16, 2013 at 12:55 AM, Lourival Vieira Neto wrote: > *is > > On Sat, Nov 16, 2013 at 12:53 AM, Lourival Vieira Neto > wrote: >> Hi Folks, >> >> Here are a patch that changes lua_Number from 'long long' to 'int64_t'. >> >> Regards, >> -- >> Lourival Vieira Neto > > -- > Lourival Vieira Neto -- Lourival Vieira Neto lua_kernel_int64_t-2.patch Description: Binary data
Re: [patch] changing lua_Number to int64_t
*is On Sat, Nov 16, 2013 at 12:53 AM, Lourival Vieira Neto wrote: > Hi Folks, > > Here are a patch that changes lua_Number from 'long long' to 'int64_t'. > > Regards, > -- > Lourival Vieira Neto -- Lourival Vieira Neto
[patch] changing lua_Number to int64_t
Hi Folks, Here are a patch that changes lua_Number from 'long long' to 'int64_t'. Regards, -- Lourival Vieira Neto lua_kernel_int64_t.patch Description: Binary data
