Re: [rust-dev] Integer overflow, round -2147483648

[Gábor Lehel]

Thanks to everyone for the excellent feedback. I've submitted an RFC
incorporating many of the ideas from the discussion:

https://github.com/rust-lang/rfcs/pull/146


On Fri, Jun 27, 2014 at 5:58 PM, Patrick Walton 
wrote:

> On 6/27/14 1:31 AM, Igor Bukanov wrote:
>
>> This bug would be harmless in safe code in Rust as exploiting it
>> requires array access without bound checking.
>>
>
> Correct. This is a prime example of what I was talking about in my earlier
> message: weaponizing integer overflows is much more difficult in a
> memory-safe language.
>
> Patrick
>
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/27/14 1:31 AM, Igor Bukanov wrote:

This bug would be harmless in safe code in Rust as exploiting it
requires array access without bound checking.


Correct. This is a prime example of what I was talking about in my 
earlier message: weaponizing integer overflows is much more difficult in 
a memory-safe language.


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Igor Bukanov]

This bug would be harmless in safe code in Rust as exploiting it
requires array access without bound checking.

On 27 June 2014 07:07, Tony Arcieri  wrote:
> Thought I'd just throw this one on the fire ;)
>
> http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.html
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 27/06/14 01:45 AM, Gregory Maxwell wrote:
> On Thu, Jun 26, 2014 at 10:30 PM, Daniel Micay  wrote:
>> It's a perfect example of a case where this feature wouldn't have
>> helped. Performance critical loops with years of micro-optimization are
>> not going to use checked arithmetic types. Every branch that the
>> programmer thinks can be avoided will be avoided.
> 
> Checked integer operation during tests would potentially have detected
> this even where the tests were not quite good enough to usefully
> trigger the out of bounds memory access, even given your argument that
> the tests would be off in production.
> 
> (We had bugs like that in the development of the opus specification
> which were detected by Regehr's interger overflow checker but didn't
> trigger valgrind for inputs probable enough for the fuzzer to reach.)

If you had actually written a test to pass >16M of zeroes to it on
32-bit, and terabytes of data on 64-bit. It wouldn't have ever been
caught on 64-bit hardware.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 27/06/14 01:38 AM, Tony Arcieri wrote:
> On Thu, Jun 26, 2014 at 10:30 PM, Daniel Micay  > wrote:
> 
> It's a perfect example of a case where this feature wouldn't have
> helped. Performance critical loops with years of micro-optimization are
> not going to use checked arithmetic types. Every branch that the
> programmer thinks can be avoided will be avoided.
> 
> 
> I know you're not fans of a compiler flag, but a checked debug build
> with traps you can optimize out in an optimized build could've caught it.
> 
> As it were, this is what Swift provides (in addition to explicit
> overflow operators) 

It doesn't happen on inputs that aren't pathological, a debug build with
checked overflow wouldn't have caught this. No one had considered this
issue and there was no test for it.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Gregory Maxwell]

On Thu, Jun 26, 2014 at 10:30 PM, Daniel Micay  wrote:
> It's a perfect example of a case where this feature wouldn't have
> helped. Performance critical loops with years of micro-optimization are
> not going to use checked arithmetic types. Every branch that the
> programmer thinks can be avoided will be avoided.

Checked integer operation during tests would potentially have detected
this even where the tests were not quite good enough to usefully
trigger the out of bounds memory access, even given your argument that
the tests would be off in production.

(We had bugs like that in the development of the opus specification
which were detected by Regehr's interger overflow checker but didn't
trigger valgrind for inputs probable enough for the fuzzer to reach.)
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Thu, Jun 26, 2014 at 10:30 PM, Daniel Micay 
wrote:

> It's a perfect example of a case where this feature wouldn't have
> helped. Performance critical loops with years of micro-optimization are
> not going to use checked arithmetic types. Every branch that the
> programmer thinks can be avoided will be avoided.


I know you're not fans of a compiler flag, but a checked debug build with
traps you can optimize out in an optimized build could've caught it.

As it were, this is what Swift provides (in addition to explicit overflow
operators)

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 27/06/14 01:07 AM, Tony Arcieri wrote:
> Thought I'd just throw this one on the fire ;)
> 
> http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.html

It's a perfect example of a case where this feature wouldn't have
helped. Performance critical loops with years of micro-optimization are
not going to use checked arithmetic types. Every branch that the
programmer thinks can be avoided will be avoided.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

Thought I'd just throw this one on the fire ;)

http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.html
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Florian Zeitz]

On 26.06.2014 00:04, Vadim Chugunov wrote:
> I wasn't thinking of subtyping u32w to u32.  I actually don't think that
> u32 should be convertible to u32w (or vice-versa) without an explicit
> cast.   For array slices, it would have to be a transmute-like function,
> e.g. fn as_wrapping<'a>(s:&'a [u32])->&'a [u32w] { unsafe { transmute(s) } }
> 
> 
FWIW, as I understand it RFC PR #91
 would nicely deal with such
cases, by making `u32` and `u32w` coercible. I don't think there is any
reason for `u32w` not to be just a newtype of `u32` with appropriate
arithmetic trait impls.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Vadim Chugunov]

I wasn't thinking of subtyping u32w to u32.  I actually don't think that
u32 should be convertible to u32w (or vice-versa) without an explicit
cast.   For array slices, it would have to be a transmute-like function,
e.g. fn as_wrapping<'a>(s:&'a [u32])->&'a [u32w] { unsafe { transmute(s) } }
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

On Tue, Jun 24, 2014 at 6:06 PM, Vadim Chugunov  wrote:
>
> On Tue, Jun 24, 2014 at 5:41 PM, Daniel Micay 
>> wrote:
>>
>>> On 24/06/14 08:39 PM, Vadim Chugunov wrote:
>>> > I mostly agree, though  for #1, I think that new int types would be
>>> more
>>> > appropriate.   A set of special operators seems like an overkill for a
>>> > relatively infrequently used functionality.  Annotations are too broad
>>> > (what if I need to do both wrapping and non-wrapping calculations in
>>> the
>>> > same scope?).
>>>
>>> Introducing new types would make the language more painful to use, and
>>> it would be difficult to determine the correct types to use at API
>>> boundaries. It would be a large backwards compatibility hazard among
>>> other issues, and would introduce performance overhead due to issues
>>> like `&[u32]` and `&[u32c]` being different types.
>>
>>
> Easy: don't use normal ints in the API, Cast to wrapping ints when
> needed.I don't expect the slice scenario to come up often, but if it
> does, these slice types can be transmuted into each other.
>

I'm hoping for illumination on that last point from someone who understands
the type system. If, say, u32w subtypes u32, then a u32w collection would
not be assignable to/from a u32 collection. It's the
covariant/contravariant mess. [Mathematicians don't pick understandable
names like "visitor pattern."]

-- 
   Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Robert O'Callahan]

On Wed, Jun 25, 2014 at 6:58 AM, Daniel Micay  wrote:

> Rust has been consistently opposed to adding compiler switches changing
> the meaning of the code. The metadata belongs *in the code* itself, and
> you are free to flip wrapping on/off for whatever reason in the code
> itself.
>

If, for performance reasons, a developer tells the compiler it can assume
certain integer arithmetic operations do not overflow at run-time, that
does not change the meaning of the code.

So, let's define the semantics of integer arithmetic as non-wrapping by
default. Disable run-time overflow checks in default build configurations.
Add Swift-style explicit wrapping operators.

This would give us the immediate benefits Greg pointed out: debug builds
and analysis tools become more effective at finding overflow bugs, because
we would have distinguished acceptable from erroneous overflow at the
language level. This would also make it possible to enable run-time integer
overflow checking by individual projects or developers, or by default in
some future version of Rust, with minimal compatibility impact.

Rob
-- 
Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Vadim Chugunov]

On Tue, Jun 24, 2014 at 5:48 PM, Jerry Morrison  wrote:

> Yeah. And would programmers also have to convert each literal, like in the
> Java-ish hashCode() example:
>
> result = (wint) 31 * result + (wint) areaCode;
>
> because adding a non-wraparound integer and a wraparound integer is
> ambiguous?
>

Literal type could be inferred, just like it is inferred now for regular
ints.



> Hey, it's "just" 5 more arithmetic operators. (A building architect once
> said, "'Just' is a 4-letter word.")
>
>
> On Tue, Jun 24, 2014 at 5:41 PM, Daniel Micay 
> wrote:
>
>> On 24/06/14 08:39 PM, Vadim Chugunov wrote:
>> > I mostly agree, though  for #1, I think that new int types would be more
>> > appropriate.   A set of special operators seems like an overkill for a
>> > relatively infrequently used functionality.  Annotations are too broad
>> > (what if I need to do both wrapping and non-wrapping calculations in the
>> > same scope?).
>>
>> Introducing new types would make the language more painful to use, and
>> it would be difficult to determine the correct types to use at API
>> boundaries. It would be a large backwards compatibility hazard among
>> other issues, and would introduce performance overhead due to issues
>> like `&[u32]` and `&[u32c]` being different types.
>
>
Easy: don't use normal ints in the API, Cast to wrapping ints when
needed.I don't expect the slice scenario to come up often, but if it
does, these slice types can be transmuted into each other.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 08:39 PM, Vadim Chugunov wrote:
> I mostly agree, though  for #1, I think that new int types would be more
> appropriate.   A set of special operators seems like an overkill for a
> relatively infrequently used functionality.  Annotations are too broad
> (what if I need to do both wrapping and non-wrapping calculations in the
> same scope?).

You can also wrap a single operation in a block, like `let z =
#[wrap(no)] { x + y }`. Rust just needs to start accepting attributes on
blocks.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

Yeah. And would programmers also have to convert each literal, like in the
Java-ish hashCode() example:

result = (wint) 31 * result + (wint) areaCode;

because adding a non-wraparound integer and a wraparound integer is
ambiguous?

Hey, it's "just" 5 more arithmetic operators. (A building architect once
said, "'Just' is a 4-letter word.")


On Tue, Jun 24, 2014 at 5:41 PM, Daniel Micay  wrote:

> On 24/06/14 08:39 PM, Vadim Chugunov wrote:
> > I mostly agree, though  for #1, I think that new int types would be more
> > appropriate.   A set of special operators seems like an overkill for a
> > relatively infrequently used functionality.  Annotations are too broad
> > (what if I need to do both wrapping and non-wrapping calculations in the
> > same scope?).
>
> Introducing new types would make the language more painful to use, and
> it would be difficult to determine the correct types to use at API
> boundaries. It would be a large backwards compatibility hazard among
> other issues, and would introduce performance overhead due to issues
> like `&[u32]` and `&[u32c]` being different types.
>
>


-- 
   Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 08:39 PM, Vadim Chugunov wrote:
> I mostly agree, though  for #1, I think that new int types would be more
> appropriate.   A set of special operators seems like an overkill for a
> relatively infrequently used functionality.  Annotations are too broad
> (what if I need to do both wrapping and non-wrapping calculations in the
> same scope?).

Introducing new types would make the language more painful to use, and
it would be difficult to determine the correct types to use at API
boundaries. It would be a large backwards compatibility hazard among
other issues, and would introduce performance overhead due to issues
like `&[u32]` and `&[u32c]` being different types.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Vadim Chugunov]

I mostly agree, though  for #1, I think that new int types would be more
appropriate.   A set of special operators seems like an overkill for a
relatively infrequently used functionality.  Annotations are too broad
(what if I need to do both wrapping and non-wrapping calculations in the
same scope?).


On Tue, Jun 24, 2014 at 5:03 PM, Jerry Morrison  wrote:

>
> On Tue, Jun 24, 2014 at 11:58 AM, Daniel Micay 
> wrote:
>
>> That's why I support adding attributes but turning wrapping on overflow
>>  on and off for a scope. You can indicate whether wrapping is considered
>> correct in that scope, meaning you either expect it to wrap or you have
>> audited it (just as you would an `unsafe` block) and determined that it
>> can not overflow. This doesn't introduce implementation defined
>> behaviour or new language dialects via compiler flags.
>>
>> Rust has been consistently opposed to adding compiler switches changing
>> the meaning of the code. The metadata belongs *in the code* itself, and
>> you are free to flip wrapping on/off for whatever reason in the code
>> itself.
>
>
> I agree except there are 3 cases even if the compiler treats two of them
> the same. Whatever order Gábor listed them, there's:
>
> (1) Wraparound is correct here, e.g. in a hash function. This is important
> information to programmers, debuggers, analysis tools, and testing tools
> even if the compiler treats it identically with #2.
>
> (2) [Default] Wraparound would be incorrect but don't check at runtime to
> save the performance costs. This includes the audited case -- Tools should
> still consider overflow here bad. (What's the distinction between "bug" and
> "error"?) Even we audited this code, maybe we goofed in our audit and maybe
> later code changes invalidated some of our assumptions.
>
> (3) Wraparound is incorrect and do check at runtime. Hopefully the
> language would not require precise exceptions for these integer overflow
> checks and future development work would test techniques like AIR.
>
>
> On ergonomics:
>
> * It sounds good to specify #2 and #3 with annotations so the choice can
> be made and changed on a whole scope, and yes, so the information is in the
> source code for all programmers and tools to see.
>
> Q: Is there a practical way to scope it larger than a function?
> Q: Could testing tools temporarily change the annotation's behavior to
> make #[unchecked] act as #[checked]?
> Q: Could debugging tools temporarily change the annotation's behavior into
> a precise exception to aid debugging?
>
> * #1 is separate -- that code won't change to/from #2 or #3. An annotation
> sounds reasonable although wraparound operators sound better since they're
> more visible and not accidentally changed when editing annotations.
>
> --
>Jerry
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

On Tue, Jun 24, 2014 at 11:58 AM, Daniel Micay 
wrote:

> That's why I support adding attributes but turning wrapping on overflow
> on and off for a scope. You can indicate whether wrapping is considered
> correct in that scope, meaning you either expect it to wrap or you have
> audited it (just as you would an `unsafe` block) and determined that it
> can not overflow. This doesn't introduce implementation defined
> behaviour or new language dialects via compiler flags.
>
> Rust has been consistently opposed to adding compiler switches changing
> the meaning of the code. The metadata belongs *in the code* itself, and
> you are free to flip wrapping on/off for whatever reason in the code
> itself.


I agree except there are 3 cases even if the compiler treats two of them
the same. Whatever order Gábor listed them, there's:

(1) Wraparound is correct here, e.g. in a hash function. This is important
information to programmers, debuggers, analysis tools, and testing tools
even if the compiler treats it identically with #2.

(2) [Default] Wraparound would be incorrect but don't check at runtime to
save the performance costs. This includes the audited case -- Tools should
still consider overflow here bad. (What's the distinction between "bug" and
"error"?) Even we audited this code, maybe we goofed in our audit and maybe
later code changes invalidated some of our assumptions.

(3) Wraparound is incorrect and do check at runtime. Hopefully the language
would not require precise exceptions for these integer overflow checks and
future development work would test techniques like AIR.


On ergonomics:

* It sounds good to specify #2 and #3 with annotations so the choice can be
made and changed on a whole scope, and yes, so the information is in the
source code for all programmers and tools to see.

Q: Is there a practical way to scope it larger than a function?
Q: Could testing tools temporarily change the annotation's behavior to make
#[unchecked] act as #[checked]?
Q: Could debugging tools temporarily change the annotation's behavior into
a precise exception to aid debugging?

* #1 is separate -- that code won't change to/from #2 or #3. An annotation
sounds reasonable although wraparound operators sound better since they're
more visible and not accidentally changed when editing annotations.

-- 
   Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 03:33 PM, Thad Guidry wrote:
> I completely agree with Daniel in all points on this thread.  (he
> aggressively states over and over his stance and the teams concerning
> the goals of Rust. The team has not deviated from their objective of the
> Rust model. Kudos.)

Well, I don't speak for anyone but myself.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Thad Guidry]

I completely agree with Daniel in all points on this thread.  (he
aggressively states over and over his stance and the teams concerning the
goals of Rust. The team has not deviated from their objective of the Rust
model. Kudos.)

I do not need compiler switches nor do I want them.
I want the control defined in the code by me.
Giving me that control for wrapping on/off for a scope is the way forward.
Whatever the names end up being...

+1 for wrapping on/off for a scope.  Get 'er done.

(and thanks for keeping my memory safe)



On Tue, Jun 24, 2014 at 2:01 PM, Daniel Micay  wrote:

> On 24/06/14 02:34 PM, Daniel Micay wrote:
> >
> > You haven't explained how this is going to cause security issues in
> > Rust, when the language is guaranteed to be memory safe outside of
> > `unsafe` blocks. The `unsafe` blocks are low-level, performance critical
> > code where unnecessary overflow checks are always going to be
> > acceptable, so the feature has next to no value in terms of memory
> safety.
>
> s/acceptable/unacceptable/
>
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>


-- 
-Thad
+ThadGuidry 
Thad on LinkedIn 
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 02:34 PM, Daniel Micay wrote:
> 
> You haven't explained how this is going to cause security issues in
> Rust, when the language is guaranteed to be memory safe outside of
> `unsafe` blocks. The `unsafe` blocks are low-level, performance critical
> code where unnecessary overflow checks are always going to be
> acceptable, so the feature has next to no value in terms of memory safety.

s/acceptable/unacceptable/



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 02:51 PM, Gregory Maxwell wrote:
> On Tue, Jun 24, 2014 at 11:39 AM, Daniel Micay  wrote:
>> A language full of implementation defined behaviour and language
>> dialects via compiler switches has no place in 2014.
> 
> This seems to be getting a by high spirited here.
> 
> Am I supposted to respond in kind? "A language where common idiomatic
> code cannot express where integer overflow is correct or incorrect has
> no place in 2014. Even C does better than this."

That's why I support adding attributes but turning wrapping on overflow
on and off for a scope. You can indicate whether wrapping is considered
correct in that scope, meaning you either expect it to wrap or you have
audited it (just as you would an `unsafe` block) and determined that it
can not overflow. This doesn't introduce implementation defined
behaviour or new language dialects via compiler flags.

Rust has been consistently opposed to adding compiler switches changing
the meaning of the code. The metadata belongs *in the code* itself, and
you are free to flip wrapping on/off for whatever reason in the code itself.

> I don't really think such dramatic language is helpful. There are real
> tradeoffs in this space and some of them are mutually exclusive.
> Different people for rational reasons prioritize different criteria
> over each other, that doesn't make anyone wrong. There is no need to
> 'win' an argument, or even argue at all. Presumably everyone can hear
> ideas and preferences in this space without the discussion getting
> that intense.

Yes, there are real tradeoffs to make here. However, some people feel
the need to spread misinformation to back up their points and pretend to
compromise and address the issues raised by the other side while
ignoring them. It gives the appearance to someone not reading the whole
context of the thread that they have been addressed when in fact they
have not.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Gregory Maxwell]

On Tue, Jun 24, 2014 at 11:39 AM, Daniel Micay  wrote:
> A language full of implementation defined behaviour and language
> dialects via compiler switches has no place in 2014.

This seems to be getting a by high spirited here.

Am I supposted to respond in kind? "A language where common idiomatic
code cannot express where integer overflow is correct or incorrect has
no place in 2014. Even C does better than this."

I don't really think such dramatic language is helpful. There are real
tradeoffs in this space and some of them are mutually exclusive.
Different people for rational reasons prioritize different criteria
over each other, that doesn't make anyone wrong. There is no need to
'win' an argument, or even argue at all. Presumably everyone can hear
ideas and preferences in this space without the discussion getting
that intense.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 11:12 AM, Gregory Maxwell wrote:
> On Mon, Jun 23, 2014 at 10:00 PM, Daniel Micay  wrote:
>> I don't understand why this would be better than either `checked {}` or
>> checked operators along with an opt-in lint to catch unchecked
>> operators. It's far better than simply saying stuff is unspecified and
>> not actually attempting to tackle the problem. If you can't depend on
>> the semantics then there's no reason to have the feature.
> 
> Because there are many places where the runtime cost is unacceptable—
> as you've pointed out yourself. But running tests and debug builds
> with that cost is seldom an unacceptable hit and can improve code
> quality a lot, plus static analysis can make use of the additional
> information.  But all of this is unworkable if the code is full of
> harmless overflows because overflow was a perfectly acceptable thing
> to do in all cases.
> 
> The behavior of non-checked non-wrapping types could be specified
> in-so-far as they _either_ wrap or fault, up to the implementation...
> and a checked{} block could force the latter.

A language full of implementation defined behaviour and language
dialects via compiler switches has no place in 2014.

A top-level `#![checked]` flag with the ability to turn it off in
performance critical scopes will work fine if the claims about compiler
optimization passes here are true. I don't see a reason to have a
compiler switch when most of the people who want this feature are so
certain that it can be made cheap.

>> Simply making it unspecified makes 0% of the operations checked because
> few people will ever pay the huge performance cost to enable them,
> 
> I would expect that virtually everyone would run enable them in
> testing/QA builds, and all static analysis tools would make use of the
> additional information (e.g. that if it can prove that some series of
> operations will cause an overflow, thats a bug).
> 
>> This proposal is just the same stuff that has already been proposed
> without any response to most of the issues raised by those on the other
> side of the fence
> 
> It seemed pretty clear to me that he was attempting to try to balance
> the concerns. Perhaps he failed, but you spent half your message
> dismissing his attempted compromises before saying he made none at
> all.

He wasn't trying to balance the concerns, he was presenting the same
proposal yet again and not addressing the concerns that have been
raised. There were no attempts at compromising, it's the same thing as
the other proposals but presented as if it's a compromise.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 10:57 AM, Lars Bergstrom wrote:
>> On Jun 23, 2014, at 7:16 PM, John Regehr  wrote:
>>
>>> I do think Rust should exposed either `checked { }` or operators for
>>> checked arithmetic along with an opt-in lint to deny the unchecked
>>> operators. You can opt-out of a lint for a function/impl/module after
>>> opting into it at a higher scope.
>>>
>>> I'm just making it clear that doing this by default would make Rust
>>> slower than Java by default, and I think that would kill off interest in
>>> the language. I know I wouldn't be interested anymore.
>>
>> Sure, I think there are a lot of reasonable options here, and I agree that 
>> speed and predictability are super important for Rust.
>>
>> One thing I personally think is very important (not for 1.0, but eventually) 
>> is to make it possible -- no need for this to be mandatory -- to get 
>> overflow checking for the default integer type.  I'm happy to use a special 
>> compiler flag or whatever to get this.  The only controversial thing this 
>> requires from the core language is a way for me to tell the compiler which 
>> integers (a tiny subset, typically) should have wrapping behavior.
> 
> If the compiler option to have integer overflow checking were available, I'm 
> 99% sure that we'd require having it enabled for Servo and all of its 
> dependencies. We would probably only relax that requirement within the 
> equivalent of an `unsafe` block where the performance had shown up as an 
> issue, leaving it clearly labeled so that we could easily identify any code 
> that touches it.

This has nothing to do with `unsafe`, considering it to be `unsafe`
would be incorrect. I doubt Servo would use it, because at that point I
don't understand why you wouldn't just be writing the browser in Scala
as it would perform better. There has been a strong consensus to avoid
jamming things unrelated to memory safety into `unsafe`, such as
catching reference cycles.

>> I realize that safe integers are available and that operator overloading 
>> goes a lot ways towards making these palatable, but the fact is that 
>> everyone is an optimist when it comes to integer overflow bugs.  People just 
>> do not think they're going to get bitten.
> 
> At a quick glance, in Mozilla's bugzilla there were 88 public browser bugs 
> resolved/verified related to integer overflow issues + 14 currently under 
> investigation. I don't have access to the tippy-top secret databases of 
> security vulnerabilities, but I'd be pretty shocked if there weren't a few 
> that would get past Rust's memory safety model (e.g., because they're passed 
> down to WebCL, WebRTC libraries, a graphics driver, etc.). A quick glance at 
> the WebKit bugzilla showed 11 similarly resolved bugs and a couple currently 
> under investigation.

We've have very few memory safety bugs caused by integer overflow, and I
haven't seen these occur in places where the performance hit of the
integer overflow checking would have been acceptable.

The only one I can think of off the top of my head is the size overflow
in the ~[T] `push` function. The overflow check there had to be done
very carefully, as one check reduces the performance of pushing to a
vector by ~20% but adding in two checks resulted in a >200% loss in
performance. There are numerous calculations in that function, so
automatic integer overflow checking would destroy it.

On the other hand, we've had countless memory safety bugs caused by
exceptions. The difficulty of writing exception safe code is high, and
it's likely the most common safety issue in the Rust codebase. Yet I
don't see the Servo developers arguing against having exceptions in
Rust, in fact they're the strongest supporters of the feature.

> While that isn't a lot, if I have to choose between the Servo team spending 
> time casually optimizing corner cases or adding an LLVM pass (in ways we 
> already know - this isn't "A Sufficiently Smart Compiler"-level hackery here) 
> vs. handling panicked high-priority security issues in a browser we are 
> trying to claim is "safe," I'll pick the former every time.

You can continue living in your fantasy world, but in the real world no
compiler pass is going to make a language with overflow checking perform
as well as Java/Scala. You can keep repeating this misinformation, but
it doesn't make it any more true.

You haven't explained how this is going to cause security issues in
Rust, when the language is guaranteed to be memory safe outside of
`unsafe` blocks. The `unsafe` blocks are low-level, performance critical
code where unnecessary overflow checks are always going to be
acceptable, so the feature has next to no value in terms of memory safety.

It would help in identifying correctness issues in other cases, but it
wouldn't actually be any more correct to throw an exception than to
overflow. In either case, it's a bug, and in neither case is it a memory
safety issue.



signature.asc
Description: OpenPGP digital signature
_

Re: [rust-dev] Integer overflow, round -2147483648

[Gregory Maxwell]

On Mon, Jun 23, 2014 at 10:00 PM, Daniel Micay  wrote:
> I don't understand why this would be better than either `checked {}` or
> checked operators along with an opt-in lint to catch unchecked
> operators. It's far better than simply saying stuff is unspecified and
> not actually attempting to tackle the problem. If you can't depend on
> the semantics then there's no reason to have the feature.

Because there are many places where the runtime cost is unacceptable—
as you've pointed out yourself. But running tests and debug builds
with that cost is seldom an unacceptable hit and can improve code
quality a lot, plus static analysis can make use of the additional
information.  But all of this is unworkable if the code is full of
harmless overflows because overflow was a perfectly acceptable thing
to do in all cases.

The behavior of non-checked non-wrapping types could be specified
in-so-far as they _either_ wrap or fault, up to the implementation...
and a checked{} block could force the latter.

> Simply making it unspecified makes 0% of the operations checked because
few people will ever pay the huge performance cost to enable them,

I would expect that virtually everyone would run enable them in
testing/QA builds, and all static analysis tools would make use of the
additional information (e.g. that if it can prove that some series of
operations will cause an overflow, thats a bug).

> This proposal is just the same stuff that has already been proposed
without any response to most of the issues raised by those on the other
side of the fence

It seemed pretty clear to me that he was attempting to try to balance
the concerns. Perhaps he failed, but you spent half your message
dismissing his attempted compromises before saying he made none at
all.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Lars Bergstrom]

> On Jun 23, 2014, at 7:16 PM, John Regehr  wrote:
> 
>> I do think Rust should exposed either `checked { }` or operators for
>> checked arithmetic along with an opt-in lint to deny the unchecked
>> operators. You can opt-out of a lint for a function/impl/module after
>> opting into it at a higher scope.
>> 
>> I'm just making it clear that doing this by default would make Rust
>> slower than Java by default, and I think that would kill off interest in
>> the language. I know I wouldn't be interested anymore.
> 
> Sure, I think there are a lot of reasonable options here, and I agree that 
> speed and predictability are super important for Rust.
> 
> One thing I personally think is very important (not for 1.0, but eventually) 
> is to make it possible -- no need for this to be mandatory -- to get overflow 
> checking for the default integer type.  I'm happy to use a special compiler 
> flag or whatever to get this.  The only controversial thing this requires 
> from the core language is a way for me to tell the compiler which integers (a 
> tiny subset, typically) should have wrapping behavior.

If the compiler option to have integer overflow checking were available, I'm 
99% sure that we'd require having it enabled for Servo and all of its 
dependencies. We would probably only relax that requirement within the 
equivalent of an `unsafe` block where the performance had shown up as an issue, 
leaving it clearly labeled so that we could easily identify any code that 
touches it.

> I realize that safe integers are available and that operator overloading goes 
> a lot ways towards making these palatable, but the fact is that everyone is 
> an optimist when it comes to integer overflow bugs.  People just do not think 
> they're going to get bitten.

At a quick glance, in Mozilla's bugzilla there were 88 public browser bugs 
resolved/verified related to integer overflow issues + 14 currently under 
investigation. I don't have access to the tippy-top secret databases of 
security vulnerabilities, but I'd be pretty shocked if there weren't a few that 
would get past Rust's memory safety model (e.g., because they're passed down to 
WebCL, WebRTC libraries, a graphics driver, etc.). A quick glance at the WebKit 
bugzilla showed 11 similarly resolved bugs and a couple currently under 
investigation.

While that isn't a lot, if I have to choose between the Servo team spending 
time casually optimizing corner cases or adding an LLVM pass (in ways we 
already know - this isn't "A Sufficiently Smart Compiler"-level hackery here) 
vs. handling panicked high-priority security issues in a browser we are trying 
to claim is "safe," I'll pick the former every time.
- Lars

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 01:55 AM, Jerry Morrison wrote:
> 
> 
> On Mon, Jun 23, 2014 at 10:32 PM, Daniel Micay  > wrote:
> 
> On 24/06/14 01:17 AM, Jerry Morrison wrote:
> >
> > Does `checked { }` mean all functions within that scope use
> > checked-integer arithmetic? This sounds great to me.
> 
> It would only apply to local operations. It's not possible to alter the
> behaviour of functions in general because of crates. It would also be
> incorrect if they required wrapping semantics.
> 
> > One detail: There should be a way to explicitly specify
> > wraparound-arithmetic, e.g. wraparound-arithmetic operators. Lint
> would
> > never complain about that.
> 
> I think the scope-based solution would probably be cleaner in that case:
> 
> #[checked]
> fn foo() { ... }
> 
> #[unchecked]
> fn bar() { #[checked] { ... }  }
> 
> The lint would warn for operations in a scope without an explicit
> decision one way or the other. You could even put `#[checked]` at module
> / crate level and override it in inner scopes.
> 
> I think that's as a sane solution, as opposed to introducing dialects
> with compiler switches which I really dislike.
> 
> 
> Attributes do seem saner than compiler switches.
> 
> Intentional-wraparound arithmetic is different than
> unchecked-for-speed-and-lint-could-complain, and probably only applies
> to fixed sized, unsigned integers. Would you use wraparound operators
> like &+ for those?

Well, perhaps a better naming convention is `#[wrapping(no)]` (with
checks) and `#[wrapping(yes)]`. I don't really like the idea of having
language support for a no-op convention.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

On Mon, Jun 23, 2014 at 10:32 PM, Daniel Micay 
wrote:

> On 24/06/14 01:17 AM, Jerry Morrison wrote:
> >
> > Does `checked { }` mean all functions within that scope use
> > checked-integer arithmetic? This sounds great to me.
>
> It would only apply to local operations. It's not possible to alter the
> behaviour of functions in general because of crates. It would also be
> incorrect if they required wrapping semantics.
>
> > One detail: There should be a way to explicitly specify
> > wraparound-arithmetic, e.g. wraparound-arithmetic operators. Lint would
> > never complain about that.
>
> I think the scope-based solution would probably be cleaner in that case:
>
> #[checked]
> fn foo() { ... }
>
> #[unchecked]
> fn bar() { #[checked] { ... }  }
>
> The lint would warn for operations in a scope without an explicit
> decision one way or the other. You could even put `#[checked]` at module
> / crate level and override it in inner scopes.
>
> I think that's as a sane solution, as opposed to introducing dialects
> with compiler switches which I really dislike.
>

Attributes do seem saner than compiler switches.

Intentional-wraparound arithmetic is different than
unchecked-for-speed-and-lint-could-complain, and probably only applies to
fixed sized, unsigned integers. Would you use wraparound operators like &+
for those?

-- 
   Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 01:34 AM, Daniel Micay wrote:
> On 24/06/14 01:22 AM, comex wrote:
>> On Tue, Jun 24, 2014 at 1:17 AM, Jerry Morrison  wrote:
>>> Does `checked { }` mean all functions within that scope use checked-integer
>>> arithmetic? This sounds great to me.
>>
>> Bikeshed: If this happens there should also be a module-level
>> attribute alternative to avoid unnecessary indentation.
> 
> I think it would actually work best as a lint-like attribute you could
> apply at any item level or to a block. It would flip the behaviour of
> the operators for that scope, and you could override it any number of
> times in inner scopes.

By 'flip' I mean *override*, you'd have both #[checked] and
#[unchecked], although preferably with clearer names.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 01:22 AM, comex wrote:
> On Tue, Jun 24, 2014 at 1:17 AM, Jerry Morrison  wrote:
>> Does `checked { }` mean all functions within that scope use checked-integer
>> arithmetic? This sounds great to me.
> 
> Bikeshed: If this happens there should also be a module-level
> attribute alternative to avoid unnecessary indentation.

I think it would actually work best as a lint-like attribute you could
apply at any item level or to a block. It would flip the behaviour of
the operators for that scope, and you could override it any number of
times in inner scopes.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 24/06/14 01:17 AM, Jerry Morrison wrote:
> 
> Does `checked { }` mean all functions within that scope use
> checked-integer arithmetic? This sounds great to me.

It would only apply to local operations. It's not possible to alter the
behaviour of functions in general because of crates. It would also be
incorrect if they required wrapping semantics.

> One detail: There should be a way to explicitly specify
> wraparound-arithmetic, e.g. wraparound-arithmetic operators. Lint would
> never complain about that.

I think the scope-based solution would probably be cleaner in that case:

#[checked]
fn foo() { ... }

#[unchecked]
fn bar() { #[checked] { ... }  }

The lint would warn for operations in a scope without an explicit
decision one way or the other. You could even put `#[checked]` at module
/ crate level and override it in inner scopes.

I think that's as a sane solution, as opposed to introducing dialects
with compiler switches which I really dislike.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Tue, Jun 24, 2014 at 1:17 AM, Jerry Morrison  wrote:
> Does `checked { }` mean all functions within that scope use checked-integer
> arithmetic? This sounds great to me.

Bikeshed: If this happens there should also be a module-level
attribute alternative to avoid unnecessary indentation.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

On Mon, Jun 23, 2014 at 1:49 PM, Daniel Micay  wrote:

> On 23/06/14 04:00 PM, Gregory Maxwell wrote:
> > On Mon, Jun 23, 2014 at 12:50 PM, Daniel Micay 
> wrote:
> >> The discussion here is about checking for both signed / unsigned integer
> >> overflow, as in passing both `-fsanitize=signed-integer-overflow` and
> >> `-fsanitize=unsigned-integer-overflow`. Rust has defined signed overflow
> >> already so it doesn't make sense to just check for that.
> >
> > The undefinedness of just signed overflow in C has shown itself to be
> > useful from a performance perspective and, paradoxically now that
> > better testing tools exist, from a correctness perspective.
>
> I already mentioned the issue of undefined overflow, and how using
> inbounds pointer arithmetic is both higher-level (iterators) and just as
> fast. It doesn't cover every case, but it covers enough of them that the
> use case for undefined signed overflow is reasonable small.
>
> Undefined behaviour on overflow is also a memory safety issue, while
> wrapping on overflow is not. You can claim that it could cause memory
> safety issues via incorrect unsafe code, but low-level code is going to
> be using wrapping or undefined semantics for performance regardless of
> the default.
>
> > I think a lot the discussion here has been about having checked types
> > and making them a default, not in forcing all possible usage into
> > them.  If only making the signed type checked had much better
> > performance characteristics  then it ought to be considered.
>
> Slower performance than Java by default would kill off nearly all
> interest in Rust, and would make the claim that it's a viable C
> replacement even less true than it already is.
>
> I don't understand what the problem would be with my proposal to have
> either `checked { }` or checked operators + a lint for unchecked usage.
>

Does `checked { }` mean all functions within that scope use
checked-integer arithmetic? This sounds great to me.

One detail: There should be a way to explicitly specify
wraparound-arithmetic, e.g. wraparound-arithmetic operators. Lint would
never complain about that.


> John was kind enough to post numbers for each of many microbenchmarks
> > instead of a range. Beyond the signed vs signed+unsigned do you have
> > any additional idea why his numbers would be lower than yours?
>
> If you read my response, you'll see that I mentioned the impact of the
> architecture on the results. I also mentioned that the code bloat issue
> does not impact microbenchmarks as they fit entirely into the L1 cache
> with or without the overflow checks. If we're going to play the game of
> picking and choosing benchmarks, I can demonstrate cases where the
> overhead is 1000-2000%.
>
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 11:58 PM, François-Xavier Bourlet wrote:
> In short:
>  - everybody wants checked integer arithmetic because it helps to
> write better code (thanks to compile time and runtime errors)
>  - nobody wants to pay the price in performances, but maybe in the
> future, hardware++ will make it easier... or so on...
> 
> What about:
>  - Defining "safe" integer in Rust with strict and well defined rules.
>  - Enforcing the rules is an incremental process.
> 
> At first, Rust will not enforce any rules, maybe in debug mode or what
> not for who wants to pay the cost. One could add an attribute on
> blocks to enforce runtime overflow check at all cost, when you really,
> really need safety no matter the speed tradeoff. And in the future,
> the compiler & standard library are free to effectively enforce more
> of the rules, as performances tradeoff mitigate.
> 
> Its kinda like when C/C++ compiler started warning that not all cases
> where handled in switch/cases on enums (yes I know our case is
> different on many level).
> 
> What is important, is that the rules of the different safe integer
> types are well defined, even if not necessarily enforced yet.
> 
> If you think about it, its better than nothing.
> 
> The goal is not to solve everything right now, but at least to define
> the scope of the problem when it is still possible, before everybody
> uses "unsafe integers", so it can be implemented in the future.

I don't understand why this would be better than either `checked {}` or
checked operators along with an opt-in lint to catch unchecked
operators. It's far better than simply saying stuff is unspecified and
not actually attempting to tackle the problem. If you can't depend on
the semantics then there's no reason to have the feature.

Simply making it unspecified makes 0% of the operations checked because
few people will ever pay the huge performance cost to enable them, while
adding usable checked arithmetic would actually be valuable.

This proposal is just the same stuff that has already been proposed
without any response to most of the issues raised by those on the other
side of the fence. It's introducing unspecified behehaviour / language
dialects, the problems associated with fragmenting code into various
integer types, and the other issues that were raised. This seems
strictly worse than an operator or scope-based solution to me.

Increasing the complexity of the language with no clear goals is not
"better than nothing". It's not somehow going to become 'free' in the
near future, and there's no point in trying to design for 25 years down
the road at the expense of making it worse today.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[François-Xavier Bourlet]

In short:
 - everybody wants checked integer arithmetic because it helps to
write better code (thanks to compile time and runtime errors)
 - nobody wants to pay the price in performances, but maybe in the
future, hardware++ will make it easier... or so on...

What about:
 - Defining "safe" integer in Rust with strict and well defined rules.
 - Enforcing the rules is an incremental process.

At first, Rust will not enforce any rules, maybe in debug mode or what
not for who wants to pay the cost. One could add an attribute on
blocks to enforce runtime overflow check at all cost, when you really,
really need safety no matter the speed tradeoff. And in the future,
the compiler & standard library are free to effectively enforce more
of the rules, as performances tradeoff mitigate.

Its kinda like when C/C++ compiler started warning that not all cases
where handled in switch/cases on enums (yes I know our case is
different on many level).

What is important, is that the rules of the different safe integer
types are well defined, even if not necessarily enforced yet.

If you think about it, its better than nothing.

The goal is not to solve everything right now, but at least to define
the scope of the problem when it is still possible, before everybody
uses "unsafe integers", so it can be implemented in the future.

On Mon, Jun 23, 2014 at 7:52 PM, Benjamin Striegel
 wrote:
>> the fact is that everyone is an optimist when it comes to integer overflow
>> bugs.  People just do not think they're going to get bitten.
>
> I agree, and I don't think anyone else here is going to try to argue that
> this doesn't cause real bugs. As so often seems to be the case, language
> design amounts to deciding which unfortunate tradeoffs you are willing to
> live with.
>
>
> On Mon, Jun 23, 2014 at 8:16 PM, John Regehr  wrote:
>>>
>>> I do think Rust should exposed either `checked { }` or operators for
>>> checked arithmetic along with an opt-in lint to deny the unchecked
>>> operators. You can opt-out of a lint for a function/impl/module after
>>> opting into it at a higher scope.
>>>
>>> I'm just making it clear that doing this by default would make Rust
>>> slower than Java by default, and I think that would kill off interest in
>>> the language. I know I wouldn't be interested anymore.
>>
>>
>> Sure, I think there are a lot of reasonable options here, and I agree that
>> speed and predictability are super important for Rust.
>>
>> One thing I personally think is very important (not for 1.0, but
>> eventually) is to make it possible -- no need for this to be mandatory -- to
>> get overflow checking for the default integer type.  I'm happy to use a
>> special compiler flag or whatever to get this.  The only controversial thing
>> this requires from the core language is a way for me to tell the compiler
>> which integers (a tiny subset, typically) should have wrapping behavior.
>>
>> I realize that safe integers are available and that operator overloading
>> goes a lot ways towards making these palatable, but the fact is that
>> everyone is an optimist when it comes to integer overflow bugs.  People just
>> do not think they're going to get bitten.
>>
>> Finally, I'll note that certain optimizations such as array bounds check
>> removal and some loop optimziations actually get better then integers cannot
>> wrap.  Clearly we would not expect, in general, for these benefits to make
>> up for the costs of overflow checking.
>>
>>
>> John
>> ___
>> Rust-dev mailing list
>> Rust-dev@mozilla.org
>> https://mail.mozilla.org/listinfo/rust-dev
>
>
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Benjamin Striegel]

> the fact is that everyone is an optimist when it comes to integer
overflow bugs.  People just do not think they're going to get bitten.

I agree, and I don't think anyone else here is going to try to argue that
this doesn't cause real bugs. As so often seems to be the case, language
design amounts to deciding which unfortunate tradeoffs you are willing to
live with.


On Mon, Jun 23, 2014 at 8:16 PM, John Regehr  wrote:

> I do think Rust should exposed either `checked { }` or operators for
>> checked arithmetic along with an opt-in lint to deny the unchecked
>> operators. You can opt-out of a lint for a function/impl/module after
>> opting into it at a higher scope.
>>
>> I'm just making it clear that doing this by default would make Rust
>> slower than Java by default, and I think that would kill off interest in
>> the language. I know I wouldn't be interested anymore.
>>
>
> Sure, I think there are a lot of reasonable options here, and I agree that
> speed and predictability are super important for Rust.
>
> One thing I personally think is very important (not for 1.0, but
> eventually) is to make it possible -- no need for this to be mandatory --
> to get overflow checking for the default integer type.  I'm happy to use a
> special compiler flag or whatever to get this.  The only controversial
> thing this requires from the core language is a way for me to tell the
> compiler which integers (a tiny subset, typically) should have wrapping
> behavior.
>
> I realize that safe integers are available and that operator overloading
> goes a lot ways towards making these palatable, but the fact is that
> everyone is an optimist when it comes to integer overflow bugs.  People
> just do not think they're going to get bitten.
>
> Finally, I'll note that certain optimizations such as array bounds check
> removal and some loop optimziations actually get better then integers
> cannot wrap.  Clearly we would not expect, in general, for these benefits
> to make up for the costs of overflow checking.
>
>
> John
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 08:16 PM, John Regehr wrote:
>> I do think Rust should exposed either `checked { }` or operators for
>> checked arithmetic along with an opt-in lint to deny the unchecked
>> operators. You can opt-out of a lint for a function/impl/module after
>> opting into it at a higher scope.
>>
>> I'm just making it clear that doing this by default would make Rust
>> slower than Java by default, and I think that would kill off interest in
>> the language. I know I wouldn't be interested anymore.
> 
> Sure, I think there are a lot of reasonable options here, and I agree
> that speed and predictability are super important for Rust.
> 
> One thing I personally think is very important (not for 1.0, but
> eventually) is to make it possible -- no need for this to be mandatory
> -- to get overflow checking for the default integer type.  I'm happy to
> use a special compiler flag or whatever to get this.  The only
> controversial thing this requires from the core language is a way for me
> to tell the compiler which integers (a tiny subset, typically) should
> have wrapping behavior.
> 
> I realize that safe integers are available and that operator overloading
> goes a lot ways towards making these palatable, but the fact is that
> everyone is an optimist when it comes to integer overflow bugs.  People
> just do not think they're going to get bitten.
> 
> Finally, I'll note that certain optimizations such as array bounds check
> removal and some loop optimziations actually get better then integers
> cannot wrap.  Clearly we would not expect, in general, for these
> benefits to make up for the costs of overflow checking.

They'll get better if they're simply guaranteed not to wrap, but not if
there are checks enforcing that at runtime. It can only get worse with
runtime enforcement. LLVM becomes unable to hoist out a bounds check in
many cases if there's a separate failure path for overflow checking.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[John Regehr]

I do think Rust should exposed either `checked { }` or operators for
checked arithmetic along with an opt-in lint to deny the unchecked
operators. You can opt-out of a lint for a function/impl/module after
opting into it at a higher scope.

I'm just making it clear that doing this by default would make Rust
slower than Java by default, and I think that would kill off interest in
the language. I know I wouldn't be interested anymore.


Sure, I think there are a lot of reasonable options here, and I agree 
that speed and predictability are super important for Rust.


One thing I personally think is very important (not for 1.0, but 
eventually) is to make it possible -- no need for this to be mandatory 
-- to get overflow checking for the default integer type.  I'm happy to 
use a special compiler flag or whatever to get this.  The only 
controversial thing this requires from the core language is a way for me 
to tell the compiler which integers (a tiny subset, typically) should 
have wrapping behavior.


I realize that safe integers are available and that operator overloading 
goes a lot ways towards making these palatable, but the fact is that 
everyone is an optimist when it comes to integer overflow bugs.  People 
just do not think they're going to get bitten.


Finally, I'll note that certain optimizations such as array bounds check 
removal and some loop optimziations actually get better then integers 
cannot wrap.  Clearly we would not expect, in general, for these 
benefits to make up for the costs of overflow checking.


John
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 06:34 PM, comex wrote:
> On Mon, Jun 23, 2014 at 5:38 PM, Benjamin Striegel
>  wrote:
>> I'd like to also note that Apple has no external incentive to improve Swift.
>> Objective-C was a dead language before Apple's fiat rocketed it into the
>> position of world's third-most-popular programming language. Regardless of
>> Swift's implementation or design decisions, it *will* be one of the most
>> popular languages in the world come this time next year (likely accompanied
>> by Objective-C's meteoric descent). If Swift were a fusion of RPG and
>> Malbolge with an implementation written in INTERCAL, this fact would not
>> change (thankfully, the Swift designers have better taste). Why bother
>> straining yourself to satisfy a captive audience, when your only real
>> competitor is whatever dialect of Java that Dalvik supports?
> 
> For one thing, Swift ought to be an appealing potential competitor for
> Apple's internal high-performance frameworks, which are currently all
> C++.

Perhaps with the -Ofast hack...



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 06:08 PM, Tony Arcieri wrote:
> On Mon, Jun 23, 2014 at 3:07 PM, Daniel Micay  > wrote:
> 
> The language shouldn't be designed around the hypothetical good will of
> a corporation. Anyway, I don't know why Swift would have the high-level
> SIL IR layer if that's not where they plan on doing these optimizations.
> 
> To flip the question around: what's wrong with Swift's approach?

Nothing is wrong with performing optimizations on a high-level IR, it's
more realistic than doing it in LLVM IR. It adds complexity and hurts
compile-time, but it's a pragmatic approach. However, there are no plans
to have an optimization layer in Rust's frontend.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 5:38 PM, Benjamin Striegel
 wrote:
> I'd like to also note that Apple has no external incentive to improve Swift.
> Objective-C was a dead language before Apple's fiat rocketed it into the
> position of world's third-most-popular programming language. Regardless of
> Swift's implementation or design decisions, it *will* be one of the most
> popular languages in the world come this time next year (likely accompanied
> by Objective-C's meteoric descent). If Swift were a fusion of RPG and
> Malbolge with an implementation written in INTERCAL, this fact would not
> change (thankfully, the Swift designers have better taste). Why bother
> straining yourself to satisfy a captive audience, when your only real
> competitor is whatever dialect of Java that Dalvik supports?

For one thing, Swift ought to be an appealing potential competitor for
Apple's internal high-performance frameworks, which are currently all
C++.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Mon, Jun 23, 2014 at 3:08 PM, Tony Arcieri  wrote:

> To flip the question around: what's wrong with Swift's approach?
>

Or perhaps to ask a less pointed question, what's wrong with Swift's
approach besides making the slow operators the default?

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Mon, Jun 23, 2014 at 3:07 PM, Daniel Micay  wrote:

> The language shouldn't be designed around the hypothetical good will of
> a corporation. Anyway, I don't know why Swift would have the high-level
> SIL IR layer if that's not where they plan on doing these optimizations.
>

To flip the question around: what's wrong with Swift's approach?

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 05:57 PM, Tony Arcieri wrote:
> On Mon, Jun 23, 2014 at 2:44 PM, Daniel Micay  > wrote:
> 
> Rust is a performance-centric systems language, Swift is not.
> 
> 
> You say that, but I don't see how it applies to my argument. 2 of the 3
> options I proposed are purely additive changes to Rust that would not
> affect at all how it works today, besides adding new functionality. I'm
> just proposing that the new functionality mirror what Swift is doing, so
> that optimizations that are added to LLVM by Apple/Swift can be
> leveraged by Rust too.
> 
> Make sense?

The language shouldn't be designed around the hypothetical good will of
a corporation. Anyway, I don't know why Swift would have the high-level
SIL IR layer if that's not where they plan on doing these optimizations.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Mon, Jun 23, 2014 at 2:44 PM, Daniel Micay  wrote:

> Rust is a performance-centric systems language, Swift is not.
>

You say that, but I don't see how it applies to my argument. 2 of the 3
options I proposed are purely additive changes to Rust that would not
affect at all how it works today, besides adding new functionality. I'm
just proposing that the new functionality mirror what Swift is doing, so
that optimizations that are added to LLVM by Apple/Swift can be leveraged
by Rust too.

Make sense?

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 05:38 PM, Benjamin Striegel wrote:
>> I feel like Rust might be missing out on the free lunch I expect Swift
> to provide
> 
> I think that it may be unfounded to expect Swift to spur drastic
> improvements to any aspect of LLVM. Apple is already the biggest
> benefactor of LLVM, which powers the C compiler their OS is built with,
> the Objective-C language that all their apps are written in, the IDE
> used to write those apps, and the Javascript engine that powers their
> web browser. Despite Swift's arrival, I wouldn't expect any greater
> investment in LLVM than Apple does currently (which is a shame, given
> the bazillion dollars burning a hole in their pocket).
> 
> I'd like to also note that Apple has no external incentive to improve
> Swift. Objective-C was a dead language before Apple's fiat rocketed it
> into the position of world's third-most-popular programming language.
> Regardless of Swift's implementation or design decisions, it *will* be
> one of the most popular languages in the world come this time next year
> (likely accompanied by Objective-C's meteoric descent). If Swift were a
> fusion of RPG and Malbolge with an implementation written in INTERCAL,
> this fact would not change (thankfully, the Swift designers have better
> taste). Why bother straining yourself to satisfy a captive audience,
> when your only real competitor is whatever dialect of Java that Dalvik
> supports?

Apple has also shown that they will keep their code proprietary to
maintain a competitive advantage. This is what they did with their
64-bit ARM backend, but eventually relented and landed it upstream
because the 'community' version was pulling ahead.

They have their own SIL IR for Swift, and doing these optimizations at
the SIL layer would be easier. It's harder to do these at a low-level
and they can simply sit back and benefit from those improvements without
giving away their own secret sauce. This is Apple we're talking about
after all... Objective-C was originally only open-source because it was
implemented on top of GCC and they were forced into it.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 05:08 PM, Tony Arcieri wrote:
> On Mon, Jun 23, 2014 at 1:32 PM, Daniel Micay  > wrote:
> 
> It would be an enormous mistake to ship a language with region typing /
> move semantics and worse before than Java.
> 
> 
> You keep saying that, but if the argument is to use Swift's approach, i.e.:
> 
> Non-overflow operators: + - * / %
> Overflow operators: &+ &- &* &/ &%
> 
> Or let's swap Swift's defaults if you so desire:
> 
> Overflow operators: + - * / %
> Non-overflow operators: &+ &- &* &/ &%
>  
> Or even change the syntax if you so desire:
> 
> Overflow operators: + - * / %
> Non-Overflow operators: +~ -~ *~ /~ %~
> 
> ...then any arguments about performance are really a false dichotomy.
> It's just a question of syntax and defaults. Want to perform well at
> TIOBE or other (micro)benchmarks? Use the overflow operators! Want to
> write safe code? Use the checked overflow operators. I really think Rust
> should support both approaches, be it implemented through a type or
> operator or what have you. I'm not here to bikeshed that. I just want to
> make sure both approaches have a first class position in the language,
> and would generally prefer but don't insist upon checked overflow being
> the default.
> 
> If the Rust developers insist on choosing overflow operators as the One
> True Way To Do Math, well, that's your prerogative. I will probably
> still choose Rust over Swift. But then I feel like Rust might be missing
> out on the free lunch I expect Swift to provide, which is sad if that's
> the way the cookie crumbles...

Rust is a performance-centric systems language, Swift is not. The
language chooses performance above other considerations like complexity
(move semantics, lifetimes, large APIs) and correctness elsewhere. It
has memory safety in safe code as a hard requirement, but anything else
is missing the point of the language.

An opt-in lint works well for catching accidental usage of the unchecked
operators, but it doesn't work well for catching accidental usage of the
checked ones. The one with the normal math syntax is the 'default' and
is the one that people are going to use when not specifically thinking
about it.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Benjamin Striegel]

> I feel like Rust might be missing out on the free lunch I expect Swift to
provide

I think that it may be unfounded to expect Swift to spur drastic
improvements to any aspect of LLVM. Apple is already the biggest benefactor
of LLVM, which powers the C compiler their OS is built with, the
Objective-C language that all their apps are written in, the IDE used to
write those apps, and the Javascript engine that powers their web browser.
Despite Swift's arrival, I wouldn't expect any greater investment in LLVM
than Apple does currently (which is a shame, given the bazillion dollars
burning a hole in their pocket).

I'd like to also note that Apple has no external incentive to improve
Swift. Objective-C was a dead language before Apple's fiat rocketed it into
the position of world's third-most-popular programming language. Regardless
of Swift's implementation or design decisions, it *will* be one of the most
popular languages in the world come this time next year (likely accompanied
by Objective-C's meteoric descent). If Swift were a fusion of RPG and
Malbolge with an implementation written in INTERCAL, this fact would not
change (thankfully, the Swift designers have better taste). Why bother
straining yourself to satisfy a captive audience, when your only real
competitor is whatever dialect of Java that Dalvik supports?


On Mon, Jun 23, 2014 at 5:08 PM, Tony Arcieri  wrote:

> On Mon, Jun 23, 2014 at 1:32 PM, Daniel Micay 
> wrote:
>
>> It would be an enormous mistake to ship a language with region typing /
>>  move semantics and worse before than Java.
>
>
> You keep saying that, but if the argument is to use Swift's approach, i.e.:
>
> Non-overflow operators: + - * / %
>  Overflow operators: &+ &- &* &/ &%
>
> Or let's swap Swift's defaults if you so desire:
>
> Overflow operators: + - * / %
> Non-overflow operators: &+ &- &* &/ &%
>
> Or even change the syntax if you so desire:
>
> Overflow operators: + - * / %
> Non-Overflow operators: +~ -~ *~ /~ %~
>
> ...then any arguments about performance are really a false dichotomy. It's
> just a question of syntax and defaults. Want to perform well at TIOBE or
> other (micro)benchmarks? Use the overflow operators! Want to write safe
> code? Use the checked overflow operators. I really think Rust should
> support both approaches, be it implemented through a type or operator or
> what have you. I'm not here to bikeshed that. I just want to make sure both
> approaches have a first class position in the language, and would generally
> prefer but don't insist upon checked overflow being the default.
>
> If the Rust developers insist on choosing overflow operators as the One
> True Way To Do Math, well, that's your prerogative. I will probably still
> choose Rust over Swift. But then I feel like Rust might be missing out on
> the free lunch I expect Swift to provide, which is sad if that's the way
> the cookie crumbles...
>
> --
> Tony Arcieri
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 05:11 PM, Gregory Maxwell wrote:
> 
> Calling things 'slower than java' is a little bit hyperbole with the
> actual numbers posted here. But I agree any non-trivial slowdown by
> default would adversely impact adoption, I don't consider that
> desirable.

It's really not hyperbole. Java's inner loops are only 10-30% slower
than C, so Rust with checked overflow would be slower. It already has
the issue of bounds checks and checks on the arithmetic leading up to
the bounds checks would make it even harder to hoist these out of loops.

>> I don't understand what the problem would be with my proposal to have
>> either `checked { }` or checked operators + a lint for unchecked usage.
> 
> My /own/ desire there doesn't even want either of those things, though
> I agree they could also be useful.
> 
> With the performance concerns aside, my reason for commenting was
> wanting the programmers intention to be well specified enough in
> widely deployed software that strong static and debug-build dynamic
> checking are able to suss out all aspects of software correctness, not
> just memory safety.
> 
> I think it will be unfortunate if rust takes a step back from C by
> resulting in an ecosystem where common idiomatic rust code is less
> amenability to tools that help software authors find incorrect code by
> reasoning from or trapping on overflow behavior.

The need to distinguish between wrapping as a bug and wrapping as an
error case would make Rust more difficult to write. I can see the
appeal, but I don't think it's worth the cost. I think Rust is already
pushing the complexity / noise limit that people will tolerate just to
achieve memory safety without losing references as values or using a
garbage collector, so I'm generally against adding more pain for the
sake of catching regular bugs.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 04:58 PM, Patrick Walton wrote:
> On 6/23/14 1:55 PM, Daniel Micay wrote:
>> It's not much a systems language if it's slower than an inner loop in a
>> JavaScript program without going out of your way to avoid the overhead.
> 
> I agree with your general concerns, but I should nitpick that it won't
> be slower than JavaScript, since JS needs the overflow checks too. :)
> (And engine implementers; e.g. Vyacheslav Egorov, have noted that
> they're a performance problem in JS.)
> 
> Patrick

I'll go with Scala (and Java) as my point of comparison then :P.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

O Mon, Jun 23, 2014 at 2:08 PM, Tony Arcieri  wrote:

> Want to perform well at TIOBE or other (micro)benchmarks? Use the overflow
> operators!
>

I meant http://shootout.alioth.debian.org here, but yeah, I hope you get
the idea ;)

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Gregory Maxwell]

On Mon, Jun 23, 2014 at 1:49 PM, Daniel Micay  wrote:
> I already mentioned the issue of undefined overflow, and how using
> inbounds pointer arithmetic is both higher-level (iterators) and just as
> fast. It doesn't cover every case, but it covers enough of them that the
> use case for undefined signed overflow is reasonable small.

Yes, I think I'm adequately convinced that failure to undefined
overflow is not performance doom, there are other ways to get the same
performance... though they may result in different idiomatic high
performance code than people are writing today. Some code style
evolution is unavoidable I guess.

> Undefined behavior on overflow is also a memory safety issue, while
> wrapping on overflow is not. You can claim that it could cause memory
> safety issues via incorrect unsafe code, but low-level code is going to
> be using wrapping or undefined semantics for performance regardless of
> the default.

It depends on what exactly what is allowed to be done with the undefinedness.

> Slower performance than Java by default would kill off nearly all
> interest in Rust, and would make the claim that it's a viable C
> replacement even less true than it already is.

Calling things 'slower than java' is a little bit hyperbole with the
actual numbers posted here. But I agree any non-trivial slowdown by
default would adversely impact adoption, I don't consider that
desirable.

> I don't understand what the problem would be with my proposal to have
> either `checked { }` or checked operators + a lint for unchecked usage.

My /own/ desire there doesn't even want either of those things, though
I agree they could also be useful.

With the performance concerns aside, my reason for commenting was
wanting the programmers intention to be well specified enough in
widely deployed software that strong static and debug-build dynamic
checking are able to suss out all aspects of software correctness, not
just memory safety.

I think it will be unfortunate if rust takes a step back from C by
resulting in an ecosystem where common idiomatic rust code is less
amenability to tools that help software authors find incorrect code by
reasoning from or trapping on overflow behavior.

>> John was kind enough to post numbers for each of many microbenchmarks
>> instead of a range. Beyond the signed vs signed+unsigned do you have
>> any additional idea why his numbers would be lower than yours?
>
> If you read my response, you'll see that I mentioned the impact of the
> architecture on the results. I also mentioned that the code bloat issue
> does not impact microbenchmarks as they fit entirely into the L1 cache
> with or without the overflow checks. If we're going to play the game of
> picking and choosing benchmarks, I can demonstrate cases where the
> overhead is 1000-2000%.

I'd somehow missed the code bloat point. Sorry about that.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Mon, Jun 23, 2014 at 1:32 PM, Daniel Micay  wrote:

> It would be an enormous mistake to ship a language with region typing /
>  move semantics and worse before than Java.


You keep saying that, but if the argument is to use Swift's approach, i.e.:

Non-overflow operators: + - * / %
Overflow operators: &+ &- &* &/ &%

Or let's swap Swift's defaults if you so desire:

Overflow operators: + - * / %
Non-overflow operators: &+ &- &* &/ &%

Or even change the syntax if you so desire:

Overflow operators: + - * / %
Non-Overflow operators: +~ -~ *~ /~ %~

...then any arguments about performance are really a false dichotomy. It's
just a question of syntax and defaults. Want to perform well at TIOBE or
other (micro)benchmarks? Use the overflow operators! Want to write safe
code? Use the checked overflow operators. I really think Rust should
support both approaches, be it implemented through a type or operator or
what have you. I'm not here to bikeshed that. I just want to make sure both
approaches have a first class position in the language, and would generally
prefer but don't insist upon checked overflow being the default.

If the Rust developers insist on choosing overflow operators as the One
True Way To Do Math, well, that's your prerogative. I will probably still
choose Rust over Swift. But then I feel like Rust might be missing out on
the free lunch I expect Swift to provide, which is sad if that's the way
the cookie crumbles...

-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/23/14 2:04 PM, comex wrote:

On Mon, Jun 23, 2014 at 4:49 PM, Daniel Micay  wrote:

I don't understand what the problem would be with my proposal to have
either `checked { }` or checked operators + a lint for unchecked usage.


I don't see 'checked { }' anywhere in the discussion before this
message...  sounds like it should be doable as a macro too.


That's what I was alluding to with "lexical scoping like C#", but I 
probably didn't make that clear enough for people who don't obsessively 
study these problems in other languages like I do. :)


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 4:49 PM, Daniel Micay  wrote:
> I don't understand what the problem would be with my proposal to have
> either `checked { }` or checked operators + a lint for unchecked usage.

I don't see 'checked { }' anywhere in the discussion before this
message...  sounds like it should be doable as a macro too.

I think it would be nice to at least make checked arithmetic *really
easy* like that.  For example, I was just analyzing an integer
overflow vulnerability in a tricky C++ function in a low-level
component that parses binary files.  If the component were written in
Rust, because it's low-level, normal memory safety checks might or
might not be sufficient - but since most of it is not terribly
performance sensitive, a good hardening approach would be to liberally
add checked {} around such functions or even the whole file.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/23/14 1:55 PM, Daniel Micay wrote:

It's not much a systems language if it's slower than an inner loop in a
JavaScript program without going out of your way to avoid the overhead.


I agree with your general concerns, but I should nitpick that it won't 
be slower than JavaScript, since JS needs the overflow checks too. :) 
(And engine implementers; e.g. Vyacheslav Egorov, have noted that 
they're a performance problem in JS.)


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 03:59 PM, Cameron Zwarich wrote:
> On Jun 22, 2014, at 4:12 PM, Patrick Walton  wrote:
> 
>> On 6/22/14 2:12 PM, Cameron Zwarich wrote:
>>> For some applications, Rust’s bounds checks and the inability of rustc
>>> to eliminate them in nontrivial cases will already be too much of a
>>> performance sacrifice. What do we say to those people? Is it just that
>>> memory safety is important because of its security implications, and
>>> other forms of program correctness are not?
>>>
>>> I am wary of circling around on this topic again, but I feel that the
>>> biggest mistake in this discussion js that checked overflow in a
>>> language requires a potential trap on every single integer operation.
>>> Languages like Ada (and Swift, to a lesser extent), allow for slightly
>>> imprecise exceptions in the case of integer overflow.
>>
>> I believe that it is possible that the overhead of integer overflow will be 
>> negligible in the future, and that paper is exciting to me too! But I feel 
>> that:
>>
>> 1. Integer overflow is primarily a security concern when it compromises 
>> memory safety. Quoting OWASP [1], emphasis mine:
>>
>> "An integer overflow condition exists when an integer, which has not been 
>> properly sanity checked, is used in the *determination of an offset or size 
>> for memory allocation, copying, concatenation, or similarly*.”
> 
> It doesn’t sound like that definition would consider this bug an “integer 
> overflow condition”, but it certainly seems like one to me:
> 
> http://minimaxir.com/2013/05/stones-of-jordan/

Rust's usage of `unsafe` is restricted to low-level performance critical
code and wrapping existing C libraries. I doubt that checked overflow is
going to be used in the former case since it's performance critical code
at the core of the standard libraries and in the latter case it's up to
the C code to get it right.

>> 3. The As-If-Infinitely-Ranged paper is research. Like all research, the 
>> risk of adopting integer overflow checks is somewhat high; it might still 
>> not work out to be acceptable in practice when we've exhausted all potential 
>> compiler optimizations that it allows. That risk has to be compared against 
>> the potential reward, which is likely to be lesser in Rust than in C because 
>> of the reasons outlined in (1) and (2).
> 
> Ada adopted a similar model before many of the people working on Rust were 
> even born, so the basic idea isn’t research. The researchy aspect of the AIR 
> paper is retroactively applying it to C. There are probably some unanswered 
> questions regarding aggressive optimizations in the face of this model, but 
> the same goes for many other design choices of Rust.
> 
>> 5. It's not clear to me that integer overflow cannot be added backwards 
>> compatibly: we can lexically scope checked arithmetic in much the same way 
>> C# lexically scopes unchecked arithmetic. It's not in line with Rust's 
>> philosophy of safe-by-default, but saying that we might introduce a safer 
>> opt-in version of Rust in the future strikes me as a fairly pragmatic 
>> compromise, and one that is not without successful precedent: for example, 
>> C, has, for all intents and purposes, successfully shed the baggage of its 
>> "variables without an annotated type default to int" design mistake via a 
>> bog-standard compiler warning.
> 
> I had to deal with bugs caused by K&R C last year, and ABIs for new 
> architectures like ARM64 are designed to be compatible with K&R C, even if it 
> complicates the implementation, so I don’t necessarily agree that the removal 
> of baggage has been quick.
> 
> We can probably correct integer operations later (even if it breaks backwards 
> compatibility to some extent), but it’s disappointing that a systems language 
> from ~2013 can’t learn from all of the correctness lessons of a systems 
> language from 1983.

It's not much a systems language if it's slower than an inner loop in a
JavaScript program without going out of your way to avoid the overhead.

I don't see why having either `checked { }` or explicitly checked
operators along with a lint is a problem. It will be backwards
compatible and won't interfere with the out-of-the-box performance, and
there won't be any significant drawbacks for people who want the feature.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 3:50 PM, Daniel Micay  wrote:
> I doubt it, since Swift has a high level IR above LLVM IR and the
> implementation isn't open-source. The language-specific optimizations
> like removing overflow / bounds checks based on type system rules will
> almost certainly be done on the high-level SIL IR, not at the LLVM IR
> layer where most of the information is already lost.

I don't know how SIL works or how this is actually implemented, but
the only part of the type system I can think of that matters for
overflow checks is aliasing, which LLVM IR definitely wants to know
about.  It's not like Swift's reachability rules are as sophisticated
as Rust's...
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 04:00 PM, Gregory Maxwell wrote:
> On Mon, Jun 23, 2014 at 12:50 PM, Daniel Micay  wrote:
>> The discussion here is about checking for both signed / unsigned integer
>> overflow, as in passing both `-fsanitize=signed-integer-overflow` and
>> `-fsanitize=unsigned-integer-overflow`. Rust has defined signed overflow
>> already so it doesn't make sense to just check for that.
> 
> The undefinedness of just signed overflow in C has shown itself to be
> useful from a performance perspective and, paradoxically now that
> better testing tools exist, from a correctness perspective.

I already mentioned the issue of undefined overflow, and how using
inbounds pointer arithmetic is both higher-level (iterators) and just as
fast. It doesn't cover every case, but it covers enough of them that the
use case for undefined signed overflow is reasonable small.

Undefined behaviour on overflow is also a memory safety issue, while
wrapping on overflow is not. You can claim that it could cause memory
safety issues via incorrect unsafe code, but low-level code is going to
be using wrapping or undefined semantics for performance regardless of
the default.

> I think a lot the discussion here has been about having checked types
> and making them a default, not in forcing all possible usage into
> them.  If only making the signed type checked had much better
> performance characteristics  then it ought to be considered.

Slower performance than Java by default would kill off nearly all
interest in Rust, and would make the claim that it's a viable C
replacement even less true than it already is.

I don't understand what the problem would be with my proposal to have
either `checked { }` or checked operators + a lint for unchecked usage.

> John was kind enough to post numbers for each of many microbenchmarks
> instead of a range. Beyond the signed vs signed+unsigned do you have
> any additional idea why his numbers would be lower than yours?

If you read my response, you'll see that I mentioned the impact of the
architecture on the results. I also mentioned that the code bloat issue
does not impact microbenchmarks as they fit entirely into the L1 cache
with or without the overflow checks. If we're going to play the game of
picking and choosing benchmarks, I can demonstrate cases where the
overhead is 1000-2000%.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 04:01 PM, John Regehr wrote:
>> I doubt it, since Swift has a high level IR above LLVM IR and the
>> implementation isn't open-source. The language-specific optimizations
>> like removing overflow / bounds checks based on type system rules will
>> almost certainly be done on the high-level SIL IR, not at the LLVM IR
>> layer where most of the information is already lost.
> 
> No, the overflow checks in Swift will turn into LLVM intrinsics such as
> llvm.sadd.with.overflow.  This is also what Rust would use.  Then, the
> IR-level optimizations will try to remove these.  LLVM already has a
> not-great integer range analysis that could be beefed up significantly
> without a whole lot of trouble.

I already exposed these intrinsics as Rust intrinsics and you can make a
user-defined integer type using these for the overloaded operators.

> Of course there's nothing stopping higher-level analyses from also
> attempting to avoid unnecessary overflow checks but I doubt that much or
> any of this is being done.

I don't think they'd have their own high-level IR if they didn't plan on
doing those kinds of optimizations. It's a compile-time hit so it's
there for a reason, and the only reasons I can think of are leveraging
the type system's design to eliminate bounds checks, overflow checks and
to perform devirtualization.

>> Rust 1.0 will be released in about 6 months, and these improvements
>> aren't going to happen in that time. It's a language for the present,
>> not one written for a fantasy architecture / compiler backend in 2025.
> 
> I wasn't arguing that Rust 1.0 should trap on integer overflow, I was
> arguing that the overhead of overflow checking isn't as high as you say.
>  But I'd be happy to look at real data.

I do think Rust should exposed either `checked { }` or operators for
checked arithmetic along with an opt-in lint to deny the unchecked
operators. You can opt-out of a lint for a function/impl/module after
opting into it at a higher scope.

I'm just making it clear that doing this by default would make Rust
slower than Java by default, and I think that would kill off interest in
the language. I know I wouldn't be interested anymore.

This conversation could just as easily be about using compiler
optimizations instead of exposing move semantics and lifetimes in the
type system.

Affine types and region typing is a lot of complexity to tackle an issue
that many people would claim a compiler could do. I don't see a claim
that it can eliminate the overhead of integer overflow checks or bounds
checks to be any less extraordinary, as if it was really as easy as
people are making it out to be then it would already work after all
these years.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 04:10 PM, Tony Arcieri wrote:
> On Monday, June 23, 2014, Daniel Micay  > wrote:
> 
> Rust is not a language designed for an imaginary sufficiently smart
> compiler. It targets real architectures and the real LLVM backend.
> 
> 
> I hate to keep throwing hypotheticals at you, but here's another one:
> what if Apple invests in LLVM development which improves the performance
> of Swift semantics?
> 
> To be clear, Swift semantics eschew the dichotomy of checked overflow vs
> the fast path. Swift offers both, but makes the "safe" operators the
> default.
> 
> With safe operators as the default, and Swift being marketed as a
> performance-oriented language, Apple is incentivized to optimize Swift's
> overflow handling for performance. They have the resources, expertise,
> and connections to make these sorts of changes to LLVM.
> 
> I know it's a gamble, but if you "borrowed" Swift's semantics, wouldn't
> you potentially reap a free lunch from what Apple contributes upstream
> to LLVM to better optimize Swift?
> 
> I don't have the crystal ball. Maybe Apple won't submit anything
> upstream to LLVM in this regard. Maybe Swift will be a dud.
> 
> But if Swift succeds, and Rust were to adopt similar semantics, and
> Apple were to submit its LLVM optimizations for this upstream, I feel
> like Rust could reap many of the benefits too.

Swift isn't aimed at the same niche as Rust. It doesn't have references
at values, and you can't take references into values or arbitrary other
parts of data. It only has value types and atomically reference counted
reference types. It's in the same tier as C#, and isn't trying to
compete with the performance of C++ code.

It would be an enormous mistake to ship a language with region typing /
move semantics and worse before than Java. I can't see a language like
that succeeding at all. If you want Swift, feel free to use Swift.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Tony Arcieri]

On Monday, June 23, 2014, Daniel Micay  wrote:

> Rust is not a language designed for an imaginary sufficiently smart
> compiler. It targets real architectures and the real LLVM backend.
>

I hate to keep throwing hypotheticals at you, but here's another one: what
if Apple invests in LLVM development which improves the performance of
Swift semantics?

To be clear, Swift semantics eschew the dichotomy of checked overflow vs
the fast path. Swift offers both, but makes the "safe" operators the
default.

With safe operators as the default, and Swift being marketed as a
performance-oriented language, Apple is incentivized to optimize Swift's
overflow handling for performance. They have the resources, expertise, and
connections to make these sorts of changes to LLVM.

I know it's a gamble, but if you "borrowed" Swift's semantics, wouldn't you
potentially reap a free lunch from what Apple contributes upstream to LLVM
to better optimize Swift?

I don't have the crystal ball. Maybe Apple won't submit anything upstream
to LLVM in this regard. Maybe Swift will be a dud.

But if Swift succeds, and Rust were to adopt similar semantics, and Apple
were to submit its LLVM optimizations for this upstream, I feel like Rust
could reap many of the benefits too.


-- 
Tony Arcieri
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[John Regehr]

I doubt it, since Swift has a high level IR above LLVM IR and the
implementation isn't open-source. The language-specific optimizations
like removing overflow / bounds checks based on type system rules will
almost certainly be done on the high-level SIL IR, not at the LLVM IR
layer where most of the information is already lost.


No, the overflow checks in Swift will turn into LLVM intrinsics such as 
llvm.sadd.with.overflow.  This is also what Rust would use.  Then, the 
IR-level optimizations will try to remove these.  LLVM already has a 
not-great integer range analysis that could be beefed up significantly 
without a whole lot of trouble.


Of course there's nothing stopping higher-level analyses from also 
attempting to avoid unnecessary overflow checks but I doubt that much or 
any of this is being done.



Rust 1.0 will be released in about 6 months, and these improvements
aren't going to happen in that time. It's a language for the present,
not one written for a fantasy architecture / compiler backend in 2025.


I wasn't arguing that Rust 1.0 should trap on integer overflow, I was 
arguing that the overhead of overflow checking isn't as high as you say. 
 But I'd be happy to look at real data.


John
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Gregory Maxwell]

On Mon, Jun 23, 2014 at 12:50 PM, Daniel Micay  wrote:
> The discussion here is about checking for both signed / unsigned integer
> overflow, as in passing both `-fsanitize=signed-integer-overflow` and
> `-fsanitize=unsigned-integer-overflow`. Rust has defined signed overflow
> already so it doesn't make sense to just check for that.

The undefinedness of just signed overflow in C has shown itself to be
useful from a performance perspective and, paradoxically now that
better testing tools exist, from a correctness perspective.

I think a lot the discussion here has been about having checked types
and making them a default, not in forcing all possible usage into
them.  If only making the signed type checked had much better
performance characteristics  then it ought to be considered.

John was kind enough to post numbers for each of many microbenchmarks
instead of a range. Beyond the signed vs signed+unsigned do you have
any additional idea why his numbers would be lower than yours?
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Cameron Zwarich]

On Jun 22, 2014, at 4:12 PM, Patrick Walton  wrote:

> On 6/22/14 2:12 PM, Cameron Zwarich wrote:
>> For some applications, Rust’s bounds checks and the inability of rustc
>> to eliminate them in nontrivial cases will already be too much of a
>> performance sacrifice. What do we say to those people? Is it just that
>> memory safety is important because of its security implications, and
>> other forms of program correctness are not?
>> 
>> I am wary of circling around on this topic again, but I feel that the
>> biggest mistake in this discussion js that checked overflow in a
>> language requires a potential trap on every single integer operation.
>> Languages like Ada (and Swift, to a lesser extent), allow for slightly
>> imprecise exceptions in the case of integer overflow.
> 
> I believe that it is possible that the overhead of integer overflow will be 
> negligible in the future, and that paper is exciting to me too! But I feel 
> that:
> 
> 1. Integer overflow is primarily a security concern when it compromises 
> memory safety. Quoting OWASP [1], emphasis mine:
> 
> "An integer overflow condition exists when an integer, which has not been 
> properly sanity checked, is used in the *determination of an offset or size 
> for memory allocation, copying, concatenation, or similarly*.”

It doesn’t sound like that definition would consider this bug an “integer 
overflow condition”, but it certainly seems like one to me:

http://minimaxir.com/2013/05/stones-of-jordan/

> 3. The As-If-Infinitely-Ranged paper is research. Like all research, the risk 
> of adopting integer overflow checks is somewhat high; it might still not work 
> out to be acceptable in practice when we've exhausted all potential compiler 
> optimizations that it allows. That risk has to be compared against the 
> potential reward, which is likely to be lesser in Rust than in C because of 
> the reasons outlined in (1) and (2).

Ada adopted a similar model before many of the people working on Rust were even 
born, so the basic idea isn’t research. The researchy aspect of the AIR paper 
is retroactively applying it to C. There are probably some unanswered questions 
regarding aggressive optimizations in the face of this model, but the same goes 
for many other design choices of Rust.

> 5. It's not clear to me that integer overflow cannot be added backwards 
> compatibly: we can lexically scope checked arithmetic in much the same way C# 
> lexically scopes unchecked arithmetic. It's not in line with Rust's 
> philosophy of safe-by-default, but saying that we might introduce a safer 
> opt-in version of Rust in the future strikes me as a fairly pragmatic 
> compromise, and one that is not without successful precedent: for example, C, 
> has, for all intents and purposes, successfully shed the baggage of its 
> "variables without an annotated type default to int" design mistake via a 
> bog-standard compiler warning.

I had to deal with bugs caused by K&R C last year, and ABIs for new 
architectures like ARM64 are designed to be compatible with K&R C, even if it 
complicates the implementation, so I don’t necessarily agree that the removal 
of baggage has been quick.

We can probably correct integer operations later (even if it breaks backwards 
compatibility to some extent), but it’s disappointing that a systems language 
from ~2013 can’t learn from all of the correctness lessons of a systems 
language from 1983.

Cameron
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 03:15 PM, John Regehr wrote:
>> Using checked overflow will
>> reduce the performance of most code with non-trivial usage of integer
>> arithmetic by 30-70%.
> 
> No, this view is overly pessimistic.

My numbers are based on real measurements, and are accurate. You can
pick and choose different benchmarks where the performance hit isn't as
bad, but it doesn't make my numbers overly pessimistic. The performance
hit will depend a lot on the architecture, and microbenchmarks can't
measure the cost of the bloated code because it will all fit in the L1
cache regardless.

> The last time we checked, Clang with the integer sanitizer turned on had
> a little less than 30% overhead for SPEC CINT 2006, on average.  Here
> are the actual slowdowns:
> 
>   400.perlbench   42.8%
>   401.bzip2   44.4%
>   403.gcc 12.7%
>   429.mcf 11.3%
>   445.gobmk   42.0%
>   456.hmmer   36.5%
>   458.sjeng   36.7%
>   462.libquantum  36.9%
>   464.h264ref 122.0%
>   471.omnetpp 4.8%
>   473.astar   16.1%
>   483.xalancbmk   12.4%
>   433.milc22.7%
>   444.namd15.5%
>   447.dealII  52.5%
>   450.soplex  17.5%
>   453.povray  11.0%
>   470.lbm 13.3%
>   482.sphinx3 34.3%
> 
> This was on some sort of Core i7.

It will be significantly worse on ARM and older x86 CPUs. A modern x86
CPU eliminates much of the overhead from the branches themselves, but
ARM CPUs are much worse at this and there are plenty of in-order CPUs
without the ability to do this at all.

Another issue is that even with checked arithmetic provided by the
architecture, Rust couldn't produce code with those instructions by
default because it targets the baseline architecture.

> Now consider that:
> 
> - This isn't only checking for signed overflows, it's checking for lossy
> casts, shift past bitwidth, etc. -- the average overhead goes down to
> 20% if we only check for C/C++ undefined behaviors

The discussion here is about checking for both signed / unsigned integer
overflow, as in passing both `-fsanitize=signed-integer-overflow` and
`-fsanitize=unsigned-integer-overflow`. Rust has defined signed overflow
already so it doesn't make sense to just check for that.

> - LLVM does a crap job in removing overflow checks; there's a ton of
> room for improvement, and I believe this will start happening now due to
> Swift

I doubt it, since Swift has a high level IR above LLVM IR and the
implementation isn't open-source. The language-specific optimizations
like removing overflow / bounds checks based on type system rules will
almost certainly be done on the high-level SIL IR, not at the LLVM IR
layer where most of the information is already lost.

Rust 1.0 will be released in about 6 months, and these improvements
aren't going to happen in that time. It's a language for the present,
not one written for a fantasy architecture / compiler backend in 2025.

It's not going to be the last programming language, and hurting it in
the present based on biased predictions of the future is only going to
result in no one using the language.

If there was really a ton of low-hanging fruit, I expect it would have
been significantly improved by now. A claim that there's a lot of room
for improvement isn't worth anything. A working implementation is the
only thing that matters, and it needs to retain the good compile-time.

> - We designed the integer sanitizer to be a debugger, not a production
> tool, it has precise exception semantics which suppresses a lot of
> integer optimizations; a more relaxed exception model like AIR/Ada would
> permit most of LLVM's integer optimizations to keep working

I don't believe that LLVM will be capable of optimizing away most of the
overhead either way. LLVM is pretty much just an inlining machine with
good x86 code generation and register allocation. It isn't even capable
of eliminating a null check when the proof that it's null is in another
basic block because the value propagation is so incredibly bad.

The optimization capabilities of LLVM are greatly exaggerated. There's
essentially no interprocedural optimization or non-trivial alias
analysis, and it's next to impossible to preserve the high level type
system invariants.

We've made the mistake of making assumptions about LLVM's capabilities
before, by depending on optimizations that are actually implemented
(unlike the ones discussed here) but don't work in edge cases. Those
edge cases are surprisingly common, and Rust is often significantly
slower than C because some of the basic abstractions like iterators
depend on reasonable value propagation.

Rust is not a language designed for an imaginary sufficiently smart
compiler. It targets real architectures and the real LLVM backend. The
only numbers that matter are the ones you can measure, and those numbers
aren't going to drastically change in the 6 months before the 1.0 relea

Re: [rust-dev] Integer overflow, round -2147483648

[John Regehr]

Using checked overflow will
reduce the performance of most code with non-trivial usage of integer
arithmetic by 30-70%.


No, this view is overly pessimistic.

The last time we checked, Clang with the integer sanitizer turned on had 
a little less than 30% overhead for SPEC CINT 2006, on average.  Here 
are the actual slowdowns:


  400.perlbench   42.8%
  401.bzip2   44.4%
  403.gcc 12.7%
  429.mcf 11.3%
  445.gobmk   42.0%
  456.hmmer   36.5%
  458.sjeng   36.7%
  462.libquantum  36.9%
  464.h264ref 122.0%
  471.omnetpp 4.8%
  473.astar   16.1%
  483.xalancbmk   12.4%
  433.milc22.7%
  444.namd15.5%
  447.dealII  52.5%
  450.soplex  17.5%
  453.povray  11.0%
  470.lbm 13.3%
  482.sphinx3 34.3%

This was on some sort of Core i7.

Now consider that:

- This isn't only checking for signed overflows, it's checking for lossy 
casts, shift past bitwidth, etc. -- the average overhead goes down to 
20% if we only check for C/C++ undefined behaviors


- LLVM does a crap job in removing overflow checks; there's a ton of 
room for improvement, and I believe this will start happening now due to 
Swift


- We designed the integer sanitizer to be a debugger, not a production 
tool, it has precise exception semantics which suppresses a lot of 
integer optimizations; a more relaxed exception model like AIR/Ada would 
permit most of LLVM's integer optimizations to keep working


John Regehr
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[John Regehr]

 > Ada's approach to integer overflows is substantially similar to AIR
Isn't Ada's response to overflow implementation-defined?


Sort of.

First, the standard seems to require a Constraint_Error when signed 
integer overflow happens.  For example, on page 47 of the ADA 2012 standard:


"For a signed integer type, the exception Constraint_Error is raised by 
the execution of an operation that cannot deliver the correct result 
because it is outside the base range of the type."


  http://www.ada-auth.org/standards/12rm/RM-Final.pdf

On the other hand, other parts of the standard provide significant 
wiggle room, giving a design that I believe is more or less equivalent 
to AIR.  The discussion here is the best that I know of:


  http://gcc.gnu.org/ml/gcc/2009-07/msg00486.html

I believe that Ada's design point is a solid one.

John Regehr
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 12:16 PM, SiegeLord wrote:
> On 06/22/2014 11:32 AM, Benjamin Striegel wrote:
>> This is a mistaken assumption. Systems programming exists on the extreme
>> end of the programming spectrum where edge cases are the norm, not the
>> exception, and where 80/20 does not apply.
> 
> Even in systems programming not every line is going to be critical for
> performance. There is still going to be a distribution of some lines
> just taking more time than others. Additionally, in a single project,
> there's a nontrivial cost in using Rust for the 20% of code that's fast
> and using some other language for the remaining 80%. How are you going
> to transfer Rust's trait abstractions to, e.g., Python?

Rust's design rejects this kind of reasoning about performance not being
relevant to 80% of the code. The language forces you to deal with
ownership and lifetimes throughout the code, even though it's not going
to speed up most inner loops. It's there to improve the performance
characteristics if the language as a whole. If you don't care about
that, a high-level garbage collected language like F# is a better choice.

>> If you don't require absolute speed, why are you using Rust?
> 
> Because it's a nice, general purpose language? Systems programming
> language is a statement about capability, not a statement about the sole
> type of programming the language supports.

Rust supports checked overflow. It has operator overloading and
user-defined types.

> C++ can be and is used effectively in applications where speed is of the
> essence and in applications where speed doesn't matter. Is Rust going to
> be purposefully less generally useful than C++? There's always this talk
> of "C++ programmers won't use Rust because of reason X". Which C++
> programmers? In my experience the vast majority of C++ programmers don't
> push C++ to its performance limits. Are they using the wrong language
> for the job? I don't think so as there are many reasons to use C++
> beside its speed potential.

If you're using C++ and you don't care about performance or code size,
then you're using the wrong tool for the job. C++ doesn't have support
for checked overflow anyway, and there's a significant difference
between pushing the language to the performance limit and opting into a
50% performance hit. Checked overflow turns an overflow bug where the
program continues running into the program exiting abnormally, so it's
still a bug either way. We're not talking about a situation where
performance is being lost for the sake of correctness.

> Rust will never become popular if it caters to the tiny percentage of
> C++ users who care about the last few percent of speed while alienating
> everybody else (via language features or statements like yours). The
> better goal is a) enable both styles of programming b) make the
> super-fast style easy enough so that everybody uses it.

The two choices here aren't 'fast' and 'super-fast'. We're not talking
about the "last few percent of speed" here. Using checked overflow will
reduce the performance of most code with non-trivial usage of integer
arithmetic by 30-70%. The only saving grace is that many applications
use floating point arithmetic, and this would still be unchecked. Even
if the program is bounded by memory bandwidth, the stalls from all of
the extra icache churn are going to have an impact.

Supporting checked overflow well without changing the defaults is fine,
but that hasn't been what I've seen proposed here. I think Rust is
already providing a significant improvement over C by making signed
integer overflow defined, and we do pay in terms of some missed
optimizations from that alone. However, our pointer arithmetic was
switched to being inbounds like C, and you can often just use pointers
instead of relying on integer.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Benjamin Striegel]

> Ada's approach to integer overflows is substantially similar to AIR

Isn't Ada's response to overflow implementation-defined?


On Mon, Jun 23, 2014 at 11:37 AM, John Regehr  wrote:

>  I would think that something simple like
>>
>>let mut sum = 0;
>>for x in some_int_array.iter() {
>>sum += x;
>>}
>>
>> would be very hard to vectorise with unwinding integer operations.
>>
>
> It sounds like there are two problems.  First, you need to give up on
> precise exceptions.  So the code becomes something line:
>
>
> let mut sum = 0;
> overflow = false
> for x in some_int_array.iter() {
>   (sum,o) = x+sum
>   overflow |= o
> }
> if (overflow) deal with it
>
> The other problem is that as far as I know AVX doesn't store the overflow
> bits for integer vector operations and recovering these bits another way
> isn't necessarily cheap.
>
> In an earlier mail Patrick said that AIR integers are research and using
> research results is risky.  This is true but keep in mind that Ada's
> approach to integer overflows is substantially similar to AIR, and Ada is
> not a research language.
>
> John
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[John Regehr]

I would think that something simple like

   let mut sum = 0;
   for x in some_int_array.iter() {
   sum += x;
   }

would be very hard to vectorise with unwinding integer operations.


It sounds like there are two problems.  First, you need to give up on 
precise exceptions.  So the code becomes something line:


let mut sum = 0;
overflow = false
for x in some_int_array.iter() {
  (sum,o) = x+sum
  overflow |= o
}
if (overflow) deal with it

The other problem is that as far as I know AVX doesn't store the overflow 
bits for integer vector operations and recovering these bits another way 
isn't necessarily cheap.


In an earlier mail Patrick said that AIR integers are research and using 
research results is risky.  This is true but keep in mind that Ada's 
approach to integer overflows is substantially similar to AIR, and Ada is 
not a research language.


John
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Huon Wilson]

On 23/06/14 14:46, comex wrote:

On Mon, Jun 23, 2014 at 12:35 AM, Daniel Micay  wrote:

An operation that can unwind isn't pure. It impedes code motion such as
hoisting operations out of a loop, which is very important for easing
the performance issues caused by indexing bounds checks. LLVM doesn't
model the `nounwind` effect on functions simply for fun.

No it doesn't!  Or maybe it does today, but an unwindable operation is
guaranteed to be repeatable without consequence, which I'd like to
think can account for most cases where operations are hoisted out of
loops (again, could be wrong :), and not to modify any memory (unless
it traps, but in Rust at that point you are guaranteed to be exiting
the function immediately).



I would think that something simple like

  let mut sum = 0;
  for x in some_int_array.iter() {
  sum += x;
  }

would be very hard to vectorise with unwinding integer operations.


Huon
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 01:16 AM, Cameron Zwarich wrote:
> On Jun 22, 2014, at 10:00 PM, Daniel Micay  > wrote:
> 
>> On 23/06/14 12:49 AM, Cameron Zwarich wrote:
>>> On Jun 22, 2014, at 9:35 PM, Daniel Micay >> 
>>> > wrote:
>>>
 An operation that can unwind isn't pure. It impedes code motion such as
 hoisting operations out of a loop, which is very important for easing
 the performance issues caused by indexing bounds checks. LLVM doesn't
 model the `nounwind` effect on functions simply for fun.
>>>
>>> It gets easier to optimize if you adopt a less precise model of
>>> exceptions. For example, you could pick a model where you preserve
>>> control dependence and externally visible side effects, but allow
>>> reordering in other cases. This does get tricky if destructors
>>> themselves have externally visible side effects that are dependent on
>>> intervening stores that can be elided.
>>>
>>> This probably requires whole-program compilation with some knowledge of
>>> externally visible side effects, or more restrictions placed on
>>> destructors than there are currently. It also is hard to make work with
>>> unsafe code, since unsafe code might require exact placement of
>>> unwinding for memory safety in destructors.
>>>
>>> Cameron
>>
>> Adding restrictions to destructors sounds like adding an effects system
>> to Rust. I think the trait system will get in the way of an attempt to
>> do that. For example, should a trait like `Eq` use pure methods? If
>> they're not pure, then no implementation can be considered pure in
>> generic code. Anything using that generic code can't be considered pure,
>> and so on.
> 
> We already have a need for limiting what destructors can do:
> 
> https://github.com/rust-lang/rust/issues/14875

I don't really see how it would be possible to fix that. It also causes
the failing while failing issue where we currently abort, so the claim
that Rust supports isolated task failure doesn't really pass the sniff
test. The poisoning of RWLock / Mutex is necessary (otherwise we might
as well just have try-catch and call it exceptions) and erodes the
"isolation" part too.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 01:15 AM, comex wrote:
> On Mon, Jun 23, 2014 at 1:05 AM, Daniel Micay  wrote:
>> The call we make to perform unwinding isn't a pure function though, it's
>> aware of the context causing it to unwind. As long as we're supporting
>> stuff like backtraces outside of a debugger, that's going to be the case.
> 
> It does not seem essential to me that backtraces be 100% accurate in
> optimized builds - at least judging by how people manage to get by
> with the utter crap that is most compilers' optimized debugging info
> :P
> 
> Not sure exactly what you're referring to, though.

I'm referring the code performing failure in Rust being impure (not
readnone or readonly) in addition to being able to unwind (not nounwind).



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Cameron Zwarich]

On Jun 22, 2014, at 10:00 PM, Daniel Micay  wrote:

> On 23/06/14 12:49 AM, Cameron Zwarich wrote:
>> On Jun 22, 2014, at 9:35 PM, Daniel Micay > > wrote:
>> 
>>> An operation that can unwind isn't pure. It impedes code motion such as
>>> hoisting operations out of a loop, which is very important for easing
>>> the performance issues caused by indexing bounds checks. LLVM doesn't
>>> model the `nounwind` effect on functions simply for fun.
>> 
>> It gets easier to optimize if you adopt a less precise model of
>> exceptions. For example, you could pick a model where you preserve
>> control dependence and externally visible side effects, but allow
>> reordering in other cases. This does get tricky if destructors
>> themselves have externally visible side effects that are dependent on
>> intervening stores that can be elided.
>> 
>> This probably requires whole-program compilation with some knowledge of
>> externally visible side effects, or more restrictions placed on
>> destructors than there are currently. It also is hard to make work with
>> unsafe code, since unsafe code might require exact placement of
>> unwinding for memory safety in destructors.
>> 
>> Cameron
> 
> Adding restrictions to destructors sounds like adding an effects system
> to Rust. I think the trait system will get in the way of an attempt to
> do that. For example, should a trait like `Eq` use pure methods? If
> they're not pure, then no implementation can be considered pure in
> generic code. Anything using that generic code can't be considered pure,
> and so on.

We already have a need for limiting what destructors can do:

https://github.com/rust-lang/rust/issues/14875

In the case of optimizations, at least you always have the conservative option 
of preserving the exact ordering of unwinding if you can’t prove that it 
doesn’t matter.

Cameron___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 1:05 AM, Daniel Micay  wrote:
> The call we make to perform unwinding isn't a pure function though, it's
> aware of the context causing it to unwind. As long as we're supporting
> stuff like backtraces outside of a debugger, that's going to be the case.

It does not seem essential to me that backtraces be 100% accurate in
optimized builds - at least judging by how people manage to get by
with the utter crap that is most compilers' optimized debugging info
:P

Not sure exactly what you're referring to, though.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 12:46 AM, comex  wrote:
> Second of all, it may be possible to do the checks more efficiently.
> I should look at clang's assembly output.

Specifically, for the record, I propose using MPX in the future as a
poor man's version of an overflow check instruction.  For adding
32-bit integers, you would check whether the result is > 2^32 with a
preset bound register (BNDCU instruction, 4 bytes) - admittedly, Rust
doesn't use those often... similar goes for 64-bit integers to which a
constant is being added (assuming you can hoist the job of loading the
negations of those constants into bound registers).  For arbitrary
64+64, you'd have to load one of the inputs into a bounds register,
which could be more expensive.

I have no idea how this will perform when it's released in Skylake;
it's more bytes than a jump, but it avoids cluttering the branch
predictor.  Anyway, it's not exactly far future.

(I am also curious about the possibility of improvements on current
architectures by doing things like deferring overflow checks... maybe
it's not possible in genera though.)
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 12:46 AM, comex wrote:
> On Mon, Jun 23, 2014 at 12:35 AM, Daniel Micay  wrote:
>> An operation that can unwind isn't pure. It impedes code motion such as
>> hoisting operations out of a loop, which is very important for easing
>> the performance issues caused by indexing bounds checks. LLVM doesn't
>> model the `nounwind` effect on functions simply for fun.
> 
> No it doesn't!  Or maybe it does today, but an unwindable operation is
> guaranteed to be repeatable without consequence, which I'd like to
> think can account for most cases where operations are hoisted out of
> loops (again, could be wrong :), and not to modify any memory (unless
> it traps, but in Rust at that point you are guaranteed to be exiting
> the function immediately).

The call we make to perform unwinding isn't a pure function though, it's
aware of the context causing it to unwind. As long as we're supporting
stuff like backtraces outside of a debugger, that's going to be the case.

If it was pure beyond not being `nounwind` then LLVM would have more
freedom to move it around, but it would still cause more problems than
using `llvm.trap` (abort).

> In particular, the only reason I can see why an impure operation would
> require repeating a bounds check is if the compiler thinks it could
> modify the size of the array, or the index, or something else in
> memory.  But it cannot.
> 
> Yeah, yeah, Rust is designed for 2014 not 2024, and I admit "LLVM
> cannot do this right now" is a perfectly good reason in this context.
> But I want to differentiate the different arguments being made here.
>
>> Unwinding is a stateful effect, and any code that would be otherwise
>> pure is no longer pure if it has the potential to unwind. It's not
>> simply a matter of unwinding or not unwinding, the compiler needs to
>> ensure that the source of the unwinding is the same.
> 
> Only if it crosses initialization of objects with destructors.  It
> doesn't matter if the stack trace is off.
> 
>> I provided an example demonstrating the cost with `clang`.
> 
> First of all, that's including actual bounds checks as opposed to
> merely assuming impurity/unwinding, no?  Again, simply to
> differentiate the arguments...

It's just a measurement of the overflow checks on signed/unsigned
integers. It does introduce a form of impurity since it's aborting in
branches, but it's a simpler form of impurity than unwinding where the
code continues running based on the context where it was thrown.

> Second of all, it may be possible to do the checks more efficiently.
> I should look at clang's assembly output.

LLVM actually has intrinsics for checked arithmetic to avoid screwups in
code generation. I don't think it's going to get any better on x86 at a
code generation level.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 12:49 AM, Cameron Zwarich wrote:
> On Jun 22, 2014, at 9:35 PM, Daniel Micay  > wrote:
> 
>> An operation that can unwind isn't pure. It impedes code motion such as
>> hoisting operations out of a loop, which is very important for easing
>> the performance issues caused by indexing bounds checks. LLVM doesn't
>> model the `nounwind` effect on functions simply for fun.
> 
> It gets easier to optimize if you adopt a less precise model of
> exceptions. For example, you could pick a model where you preserve
> control dependence and externally visible side effects, but allow
> reordering in other cases. This does get tricky if destructors
> themselves have externally visible side effects that are dependent on
> intervening stores that can be elided.
> 
> This probably requires whole-program compilation with some knowledge of
> externally visible side effects, or more restrictions placed on
> destructors than there are currently. It also is hard to make work with
> unsafe code, since unsafe code might require exact placement of
> unwinding for memory safety in destructors.
> 
> Cameron

Adding restrictions to destructors sounds like adding an effects system
to Rust. I think the trait system will get in the way of an attempt to
do that. For example, should a trait like `Eq` use pure methods? If
they're not pure, then no implementation can be considered pure in
generic code. Anything using that generic code can't be considered pure,
and so on.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Cameron Zwarich]

On Jun 22, 2014, at 9:35 PM, Daniel Micay  wrote:

> An operation that can unwind isn't pure. It impedes code motion such as
> hoisting operations out of a loop, which is very important for easing
> the performance issues caused by indexing bounds checks. LLVM doesn't
> model the `nounwind` effect on functions simply for fun.

It gets easier to optimize if you adopt a less precise model of exceptions. For 
example, you could pick a model where you preserve control dependence and 
externally visible side effects, but allow reordering in other cases. This does 
get tricky if destructors themselves have externally visible side effects that 
are dependent on intervening stores that can be elided.

This probably requires whole-program compilation with some knowledge of 
externally visible side effects, or more restrictions placed on destructors 
than there are currently. It also is hard to make work with unsafe code, since 
unsafe code might require exact placement of unwinding for memory safety in 
destructors.

Cameron___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 12:35 AM, Daniel Micay  wrote:
> An operation that can unwind isn't pure. It impedes code motion such as
> hoisting operations out of a loop, which is very important for easing
> the performance issues caused by indexing bounds checks. LLVM doesn't
> model the `nounwind` effect on functions simply for fun.

No it doesn't!  Or maybe it does today, but an unwindable operation is
guaranteed to be repeatable without consequence, which I'd like to
think can account for most cases where operations are hoisted out of
loops (again, could be wrong :), and not to modify any memory (unless
it traps, but in Rust at that point you are guaranteed to be exiting
the function immediately).

In particular, the only reason I can see why an impure operation would
require repeating a bounds check is if the compiler thinks it could
modify the size of the array, or the index, or something else in
memory.  But it cannot.

Yeah, yeah, Rust is designed for 2014 not 2024, and I admit "LLVM
cannot do this right now" is a perfectly good reason in this context.
But I want to differentiate the different arguments being made here.

> Unwinding is a stateful effect, and any code that would be otherwise
> pure is no longer pure if it has the potential to unwind. It's not
> simply a matter of unwinding or not unwinding, the compiler needs to
> ensure that the source of the unwinding is the same.

Only if it crosses initialization of objects with destructors.  It
doesn't matter if the stack trace is off.

> I provided an example demonstrating the cost with `clang`.

First of all, that's including actual bounds checks as opposed to
merely assuming impurity/unwinding, no?  Again, simply to
differentiate the arguments...

Second of all, it may be possible to do the checks more efficiently.
I should look at clang's assembly output.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Mon, Jun 23, 2014 at 12:10 AM, Daniel Micay  wrote:
> Rust only gains efficiency from the type system, not any form of thread
> safety that's not available to a garbage collected language. It's able
> to send `Box` data structures without performing a deep copy.

Which is the garbage collected language that provides this type of
safety, along with a relatively functional view of the world (while
still preferably being imperative at heart), reasonably fast generics,
zero startup overhead (different from runtime overhead), macros,
encouraging message passing but preferably supporting shared memory
(sorry OCaml), a user community - oh, and interoperability with Rust
code, since it's perfectly reasonable to want a fast core with slower
addons?

Today, if not for those things, I would pick a different language,
because of rustc's slowness... but I'm sticking around.  Of course, it
doesn't mean that Rust has to compromise for my or anyone else's use
cases, but I don't want to just go use another language if it can be
avoided.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 23/06/14 12:17 AM, comex wrote:
> On Sun, Jun 22, 2014 at 5:05 PM, Daniel Micay  wrote:
>> Anyway, no one has offered an explanation of how they're
>> planning on integrating this into LLVM and how they propose turning a
>> trapping operation into unwinding across various platforms.
> 
> Isn't that what asynchronous unwind tables are for?  The actual
> platform glue isn't that difficult on the platforms I know of - at the
> cost of making interoperability with C programs that use signal
> handlers tend to break, but that's not fatal.

Asynchronous unwind tables are the low-level implementation detail for
providing this support. However, that doesn't mean it has well-defined
behaviour in LLVM IR. LLVM has modelling of effects like unwinding, and
it does assume that unwinding cannot simply occur anywhere. If it had to
make that assumption, it would be far less capable of performing
optimizations.

In the case of MPX, you could add new LLVM intrinsics and provide a hard
guarantee that whatever signal handler glue you use to turn MPX into
unwinding *never* occurs outside of those LLVM intrinsics.

> On Reddit you mentioned that unwinding impedes optimization.  But I'd
> still like to hear more specifics... what sort of optimizations?  I
> believe you if you say that simply allowing unwinding (without
> considering the cost of actually trapping*) significantly impedes
> performance today, but can some of that can be fixed?  In terms of
> true limitations, I can think of:

An operation that can unwind isn't pure. It impedes code motion such as
hoisting operations out of a loop, which is very important for easing
the performance issues caused by indexing bounds checks. LLVM doesn't
model the `nounwind` effect on functions simply for fun.

> - No more speculative execution - 'foo ? x + 2 : y + 4' now must be a
> branch instead of a cmov.  But how common is this in practice?  'foo ?
> x + 2 : x + 4' can of course be done with a cmov.  Bigger statements
> turn into branches anyway in practice.
> 
> - Can't reorder arithmetic with respect to stores that could be
> visible to destructors.  Does it matter?  It's still fine to eliminate
> common subexpressions as long as they're only computed when one of
> their instances should be.
> 
> My intuition could be wildly off, but I don't see why this should be
> such a big deal.  And the value many people seem to assign to the
> issue (albeit which I'm not presupposing is merited) suggests that
> making any useful improvements to LLVM in this case should be doable.
> (Easy to say without volunteering to do it myself, I know...)

Unwinding is a stateful effect, and any code that would be otherwise
pure is no longer pure if it has the potential to unwind. It's not
simply a matter of unwinding or not unwinding, the compiler needs to
ensure that the source of the unwinding is the same.

An abort (`llvm.trap`) or undefined behaviour is similar in that it adds
an effect and prevents the compiler from doing as much with the code.
However, the unwinding invariants are harder to maintain and LLVM
doesn't even take on this problem.

> Even if this turns out to be a bad idea in the end, I'm not sure this
> thread has enough of a real, measured idea of the costs involved yet,
> and again, people seem to think it's important enough that I don't
> think it's worth giving up without that.  Yes, even though 1.0 is
> soon.  Just my two cents.

I provided an example demonstrating the cost with `clang`.




signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Cameron Zwarich]

On Jun 22, 2014, at 8:52 PM, Patrick Walton  wrote:

> On 6/22/14 8:46 PM, Daniel Micay wrote:
>> It's for faster (but not free) array bounds checking. I don't think Rust
>> will be able to use it because it unwinds on out-of-bounds rather than
>> aborting, and it will be difficult to turn the OS support (perhaps
>> SIGFPE / SIGSEGV on *nix) into well defined unwinding in LLVM.
> 
> GCJ did it. Presumably JavaScriptCore does it too. How?

If you’re referring to JSC’s use of LLVM, IIRC (this is based on conversations 
with some of the people that did it, not looking at the code myself) they added 
support for on-stack replacement via stack maps and traps to a runtime to 
LLVM’s JIT. Failure of bounds checks is just treated like any other failure 
that falls off the optimized path and performs an OSR exit.

Cameron
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[comex]

On Sun, Jun 22, 2014 at 5:05 PM, Daniel Micay  wrote:
> Anyway, no one has offered an explanation of how they're
> planning on integrating this into LLVM and how they propose turning a
> trapping operation into unwinding across various platforms.

Isn't that what asynchronous unwind tables are for?  The actual
platform glue isn't that difficult on the platforms I know of - at the
cost of making interoperability with C programs that use signal
handlers tend to break, but that's not fatal.

On Reddit you mentioned that unwinding impedes optimization.  But I'd
still like to hear more specifics... what sort of optimizations?  I
believe you if you say that simply allowing unwinding (without
considering the cost of actually trapping*) significantly impedes
performance today, but can some of that can be fixed?  In terms of
true limitations, I can think of:

- No more speculative execution - 'foo ? x + 2 : y + 4' now must be a
branch instead of a cmov.  But how common is this in practice?  'foo ?
x + 2 : x + 4' can of course be done with a cmov.  Bigger statements
turn into branches anyway in practice.

- Can't reorder arithmetic with respect to stores that could be
visible to destructors.  Does it matter?  It's still fine to eliminate
common subexpressions as long as they're only computed when one of
their instances should be.

My intuition could be wildly off, but I don't see why this should be
such a big deal.  And the value many people seem to assign to the
issue (albeit which I'm not presupposing is merited) suggests that
making any useful improvements to LLVM in this case should be doable.
(Easy to say without volunteering to do it myself, I know...)

Even if this turns out to be a bad idea in the end, I'm not sure this
thread has enough of a real, measured idea of the costs involved yet,
and again, people seem to think it's important enough that I don't
think it's worth giving up without that.  Yes, even though 1.0 is
soon.  Just my two cents.

* for now, anyway, since I'm responding to a post about how this isn't
feasible  for future architectures...
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 06:43 PM, Clark Gaebel wrote:
> I think a reasonable middle ground is to have checked operators that
> look a little funny. Kind of like swift, but in reverse:
> 
>> malloc((number_of_elements +~ 12) *~ size_of::())
> 
> Where adding a ~ to the end of an operator makes it check for overflow.
> This would certainly look nicer than stuff like:
> 
>> malloc(number_of_elements.checked_add(12).checked_mul(size_of::()))
> 
> lying around in low level data structures code.
> 
> It also keeps the default fast, which is very important.
> 
>   - Clark

Along with an opt-in lint to warn about usage of the unchecked
operators, I think that's the way to go. Since it would be backwards
compatible, it doesn't need to be done before 1.0.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 11:52 PM, Patrick Walton wrote:
> On 6/22/14 8:46 PM, Daniel Micay wrote:
>> It's for faster (but not free) array bounds checking. I don't think Rust
>> will be able to use it because it unwinds on out-of-bounds rather than
>> aborting, and it will be difficult to turn the OS support (perhaps
>> SIGFPE / SIGSEGV on *nix) into well defined unwinding in LLVM.
> 
> GCJ did it. Presumably JavaScriptCore does it too. How?
> 
> Patrick

AFAIK, LLVM doesn't support unwinding from an asynchronous signal
handler with defined behaviour.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 07:24 PM, Andrew Poelstra wrote:
> On Sun, Jun 22, 2014 at 05:26:47PM -0400, Daniel Micay wrote:
>>
>> Rust's design is based on the assumption that performance cannot be
>> achieved simply by having highly optimized inner loops. It takes a whole
>> program approach to performance by exposing references as first-class
>> values and enforcing safety via type-checked lifetimes.
>>
>> You can write an efficient low-level loop in Haskell or Swift, but you
>> can't build high level safe abstractions without paying a runtime cost.
>>
>> If someone isn't interested in this approach, then I have a hard time
>> understanding why they would be using Rust.
>>
> 
> This is a bit of an aside, but the reason to use Rust if you don't care
> about performance is that it not only tracks lifetimes, but tracks
> ownership as well. As a side-effect, tracking lifetimes then becomes
> much easier, as does eliminating manual memory management.

I don't understand what you mean here. Rust's lifetimes don't represent
object lifetimes, they represent minimum reference lifetimes. Move
semantics are a way of passing on the responsibility of calling the
destructor.

There is no tracking of ownership in the type system, as shown by the
existence of types like `Rc`. With `Rc`, the programmer is
responsible for ensuring that there are no ownership cycles because the
compiler is unable to track this, it can only deal with destructors at a
scope-based level.

A managed language will not destroy an object until all references to
the object are gone. There is no need to track reference lifetimes,
because the references extend the object's lifetime as long as they
exist (they *are* essentially the object).

> This happens to also make sharing data across threads safer as well. (See
> for example basically anything Niko has written on the subject.)

Thread safety is provided via the Send trait, not anything to do with
lifetimes. A garbage collected language could (many do) provide the same
thread safety semantics as Rust. Value types, immutable reference types
and synchronized reference types would be sendable.

Rust only gains efficiency from the type system, not any form of thread
safety that's not available to a garbage collected language. It's able
to send `Box` data structures without performing a deep copy.

In the future, it will be possible to share disjoint `&mut T` between
parallel jobs (but not normal tasks) but this is also just an efficiency
improvement over other safe alternatives.

> This, along with immutable-by-default and an aversion to magic implicit
> behaviour (e.g. the Rc::clone() argument in the next thread), result in
> a language which is extremely easy to analyze. It makes things harder to
> write, but much easier to read.

It has immutable variables by default, but the contents can still be
mutable via internal (Cell, RefCell, RWLock, Mutex, atomics) or external
(&mut, slice::MutItems) mutability at a type level.

Removing the variable-level immutability was proposed, and it could
still happen at this point:

http://smallcultfollowing.com/babysteps/blog/2014/05/13/focusing-on-ownership/

It's true that there are no *user-defined* copy/move constructors, but
`Gc` *does* have magical implicit glue code running on assigning and
parameter passing. The language just doesn't allow you to do the same
thing in your own code.

> Perhaps you can point out another language that does all this. You
> definitely can't point out another language that does all this, -and-
> has the massive community around it that Rust does.
> 
> 
> So there is a strong reason to use Rust even if you don't care too much
> about performance.
> 
> 



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/22/14 8:52 PM, Patrick Walton wrote:

On 6/22/14 8:46 PM, Daniel Micay wrote:

It's for faster (but not free) array bounds checking. I don't think Rust
will be able to use it because it unwinds on out-of-bounds rather than
aborting, and it will be difficult to turn the OS support (perhaps
SIGFPE / SIGSEGV on *nix) into well defined unwinding in LLVM.


GCJ did it. Presumably JavaScriptCore does it too. How?


Also this doesn't seem that difficult to me. Replace 
array-bounds-checked loads and stores with a 
@llvm.load.with.bounds.check() (or whatever) intrinsic and have LLVM 
consider that as unwindable.


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/22/14 8:46 PM, Daniel Micay wrote:

It's for faster (but not free) array bounds checking. I don't think Rust
will be able to use it because it unwinds on out-of-bounds rather than
aborting, and it will be difficult to turn the OS support (perhaps
SIGFPE / SIGSEGV on *nix) into well defined unwinding in LLVM.


GCJ did it. Presumably JavaScriptCore does it too. How?

Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 08:25 PM, Jerry Morrison wrote:
> 
> The post also links to Wikipedia on Intel MPX
> : Intel is adding x86 extensions
> to aid memory safety! I think it's for array bounds checking, but the
> article is unclear.

It's for faster (but not free) array bounds checking. I don't think Rust
will be able to use it because it unwinds on out-of-bounds rather than
aborting, and it will be difficult to turn the OS support (perhaps
SIGFPE / SIGSEGV on *nix) into well defined unwinding in LLVM.

> (BTW is there a use for /signed/ wraparound?)

Yes, but it's not as common so C leaves it as undefined for portability
to architectures not using two's complement arithmetic.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 07:26 PM, Gábor Lehel wrote:
>
> Coincidentally, I didn't propose it. I proposed one implementation with
> owned allocation (which we already have) in the `prelude` and one with
> tracing GC allocation in the `gc` module.

Then I don't understand why we would have one for task-local garbage
collection and task-local reference counting. There can never be cycles
here and non-atomic reference counting will be faster along with
maintaining scope-based destruction semantics. I don't see the use case
for duplicating these types with versions wrapped in a specific smart
pointer type.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Daniel Micay]

On 22/06/14 07:43 PM, Vadim Chugunov wrote:
> Makes sense, but I am curious, how do you see adding this post-1.0?  
> Would you:
> - add overflow-checked int types and tell everybody to use them instead
> of the default ones from that point on
> - declare that in Rust2 integers are overflow-checked, and have
> everybody port their Rust1 code.  (Well, in reality, I would expect that
> most existing code would just continue to work, but some testing will be
> needed).
> Both sound somewhat painful.

It could be done by adding overflow-checked arithmetic operators. A lint
could then be added to warn about uses of the unchecked operators.

The lint would be opt-in, so you could flip it on at the crate level and
then disable it for audited functions.

It would be backwards compatible (post 1.0 feature) and wouldn't change
the meaning of any existing code by introducing dialects via compiler
switches.



signature.asc
Description: OpenPGP digital signature
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Patrick Walton]

On 6/22/14 5:34 PM, Vadim Chugunov wrote:

Modern C++ compilers often have a bunch of runtime checks (stack
overflow protectors, iterator invalidation detectors, and so on) that
may be enabled or disabled, and nobody bats an eye at that.


I'm not so sure. C++ is notoriously bad at dynamic linking, and these 
switches are part of the reason—code with `-fno-exceptions` can't be 
safely linked with code compiled without, for example...


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Vadim Chugunov]

On Sun, Jun 22, 2014 at 2:31 PM, Daniel Micay  wrote:

> On 22/06/14 05:09 PM, Rick Richardson wrote:
> > Apologies if this has been suggested, but would it be possible to have a
> > compiler switch that can add runtime checks and abort on
> > overflow/underflow/carry for debugging purposes, but the default
> > behavior is no check?  IMO this would be the best of both worlds,
> > because I would assume that one would really only care about checked
> > math during testing and dev.
>
> You would need to build an entirely separate set of standard libraries
> with checked overflow. Adding new dialects of the language via compiler
> switches is never the right answer. It seems that every time an issue
> like this comes up, people propose making a compiler switch as the option.
>

> If we had compiler switches for abort vs. unwinding, no tracing gc
> support vs. tracing gc support, no integer overflow checks vs. integer
> overflow checks and more, we would have a truly ridiculous number of
> language dialects. I think even 2 dialects is too much...
>

I am very skeptical about the idea having a single set of standard
libraries that are good for everyone.  If Rust ever gains any degree of
mainstream adoption, I predict that there *will* be switches to disable
bounds checks, stack overflow checks, etc, because different applications
have different needs.

Modern C++ compilers often have a bunch of runtime checks (stack overflow
protectors, iterator invalidation detectors, and so on) that may be enabled
or disabled, and nobody bats an eye at that.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Jerry Morrison]

Apologies for adding heat to the discussion. The industry needs to make
progress on security/safety-critical systems, and Rust does a great deal
for that by bringing memory safety to a C/C++ alternative.

This post about hardware traps for integer overflow
 is useful, even besides the issue of
the full cost of traps. The post and its discussion explore alternatives.
It links to this conference article about Understanding Integer Overflow in
C/C++ , which studied
overflow bugs, vulnerabilities, and some overflow-checking techniques. One
technique averaged 44% cost, ranging 0 to 191%. Another averaged 30%,
ranging 0 - 95%. They discovered that "intentional uses of wraparound
behaviors are more common than expected" but hard to find.

The post also links to Wikipedia on Intel MPX
: Intel is adding x86 extensions to
aid memory safety! I think it's for array bounds checking, but the article
is unclear.

If the choice is a Rust with inconvenient and often unused overflow safety
that kills off C/C++ vs. a Rust with overflow safety that doesn't, I vote
for the former.

The open design questions are ergonomics of checked and wraparound
arithmetic. It'd be great if uses of wraparound arithmetic were explicit.



On Sun, Jun 22, 2014 at 2:05 PM, Daniel Micay  wrote:

> A `Vec` and `Vec` would be entirely distinct types. ...
>

THAT is a knockout blow for the distinct-types design for wrapping integers.

(BTW is there a use for *signed* wraparound?)


On Sun, Jun 22, 2014 at 2:09 PM, Rick Richardson 
 wrote:

> ... I would assume that one would really only care about checked math
> during testing and dev.
>

Alas, that doesn't block security exploits.


On Sun, Jun 22, 2014 at 2:26 PM, Daniel Micay  wrote:

> Rust's design is based on the assumption that performance cannot be
> achieved simply by having highly optimized inner loops. It takes a whole
> program approach to performance by exposing references as first-class
> values and enforcing safety via type-checked lifetimes.
>
> You can write an efficient low-level loop in Haskell or Swift, but you
> can't build high level safe abstractions without paying a runtime cost.
>
> If someone isn't interested in this approach, then I have a hard time
> understanding why they would be using Rust.
>

To move a lot of code from C/C++ to a place that has memory safety and less
error-prone 1970's technology with 1960's macro processing and linking.

For real-time programs, which need predictable response times whether or
not they need high performance (often fixable with faster hardware). Many
of these are embedded devices with small memory.

Maybe Swift will work there but (1) it'll have to go beyond Apple hardware
and OSs, and (2) reference counting has unpredictable delays when freeing a
large network of objects.



On Sun, Jun 22, 2014 at 2:23 PM, Daniel Micay  wrote:

> The set of design compromises made by the language is ill-suited to a
> use case where performance isn't critical.


Is there a web page that lays out Rust's goals, design trade-offs, and
intended uses?


On Sun, Jun 22, 2014 at 2:12 PM, Cameron Zwarich 
 wrote:

> Languages like Ada (and Swift, to a lesser extent), allow for slightly
> imprecise exceptions in the case of integer overflow. ... the “As
> Infinitely Ranged” model ...
>


> I know that some people don’t want to require new compiler techniques (or
> are afraid of relying on something outside of the scope of what LLVM can
> handle today)
>

>From TFA:

> AIR integers do not require precise traps and consequently do not break or
> inhibit most existing optimizations.


Hopefully Apple will contribute its Swift compiler to the LLVM source tree.
That includes overflow detection. Also if needed, they have the ability to
customize their ARM chips and some sway with Intel.


On Sun, Jun 22, 2014 at 2:31 PM, Daniel Micay  wrote:

> If we had compiler switches for ... I think even 2 dialects is too much...
>

Yes. When one C dialect calls another via dynamic loading, bad things
happen, and there's no debugger on the robot. (Speaking from experience.)


-- 
   Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Vadim Chugunov]

Makes sense, but I am curious, how do you see adding this post-1.0?   Would
you:
- add overflow-checked int types and tell everybody to use them instead of
the default ones from that point on
- declare that in Rust2 integers are overflow-checked, and have everybody
port their Rust1 code.  (Well, in reality, I would expect that most
existing code would just continue to work, but some testing will be needed).
Both sound somewhat painful.

I still think it would be a good idea to provide explicitly wrapping int
types (which aren't the default ones) from get-go and point out in
documentation that those are the ones to be used when writing hashes, etc.
Even if they are just aliases of the default int types.  If nothing else,
this will allow people to document in their code whether the ints were
supposed to wrap around or not.

Vadim

On Sun, Jun 22, 2014 at 4:12 PM, Patrick Walton 
wrote:

> On 6/22/14 2:12 PM, Cameron Zwarich wrote:
>
>> For some applications, Rust’s bounds checks and the inability of rustc
>> to eliminate them in nontrivial cases will already be too much of a
>> performance sacrifice. What do we say to those people? Is it just that
>> memory safety is important because of its security implications, and
>> other forms of program correctness are not?
>>
>> I am wary of circling around on this topic again, but I feel that the
>> biggest mistake in this discussion js that checked overflow in a
>> language requires a potential trap on every single integer operation.
>> Languages like Ada (and Swift, to a lesser extent), allow for slightly
>> imprecise exceptions in the case of integer overflow.
>>
>
> I believe that it is possible that the overhead of integer overflow will
> be negligible in the future, and that paper is exciting to me too! But I
> feel that:
>
> 1. Integer overflow is primarily a security concern when it compromises
> memory safety. Quoting OWASP [1], emphasis mine:
>
> "An integer overflow condition exists when an integer, which has not been
> properly sanity checked, is used in the *determination of an offset or size
> for memory allocation, copying, concatenation, or similarly*."
>
> Other features in Rust, such as bounds checks, unsigned indexing, and the
> borrow check, defend against the memory safety problems. So the benefits of
> defending against memory safety are likely to be quite different in Rust
> from in C. It's a question of costs versus benefits, as always.
>
> 2. Signed integer overflow is not undefined behavior in Rust as it is in
> C. This mitigates some of the more frightening risks associated with it.
>
> 3. The As-If-Infinitely-Ranged paper is research. Like all research, the
> risk of adopting integer overflow checks is somewhat high; it might still
> not work out to be acceptable in practice when we've exhausted all
> potential compiler optimizations that it allows. That risk has to be
> compared against the potential reward, which is likely to be lesser in Rust
> than in C because of the reasons outlined in (1) and (2).
>
> 4. We have a pretty tight shipping schedule at this point: 1.0, which is
> the backwards compatible release, is on track to be released this year. We
> are making excellent progress on backwards incompatible language changes
> (closed 10 out of 45 issues over the past 2 weeks!), but we must be
> consciously fighting scope creep in order to release a product.
>
> 5. It's not clear to me that integer overflow cannot be added backwards
> compatibly: we can lexically scope checked arithmetic in much the same way
> C# lexically scopes unchecked arithmetic. It's not in line with Rust's
> philosophy of safe-by-default, but saying that we might introduce a safer
> opt-in version of Rust in the future strikes me as a fairly pragmatic
> compromise, and one that is not without successful precedent: for example,
> C, has, for all intents and purposes, successfully shed the baggage of its
> "variables without an annotated type default to int" design mistake via a
> bog-standard compiler warning.
>
> For these reasons, I'm pretty comfortable with not making any changes for
> 1.0, and deferring this issue to later.
>
> Patrick
>
> [1]: https://www.owasp.org/index.php/Integer_overflow
>
>
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Integer overflow, round -2147483648

[Gábor Lehel]

On Sun, Jun 22, 2014 at 11:05 PM, Daniel Micay 
wrote:

> On 22/06/14 09:31 AM, Gábor Lehel wrote:
> >
> > The prospect of future architectures with cheaper (free) overflow
> > checking isn't my primary motivation, though if we also end up better
> > prepared for them as a side effect, that's icing on the cake.
>
> It's never going to be free or even cheap. Replacing very common pure
> operations with impure ones in most code will destroy performance. Rust
> relies heavily on compiler optimizations to even come close to C level
> performance. Anyway, no one has offered an explanation of how they're
> planning on integrating this into LLVM and how they propose turning a
> trapping operation into unwinding across various platforms.
>

> > My primary motivation is that, outside of a couple of specialized cases
> > like hashing and checksums, wraparound semantics on overflow is
> > **wrong**. It may be well-defined behavior, and it may be fast, but it's
> > **wrong**. What's the value of a well-defined, performant semantics
> > which does the wrong thing?
>
> Few functions check all of the necessary preconditions. For example, a
> binary search implementation doesn't check to see if the array is
> sorted.  It's not incorrect to require a precondition from the caller
> and overflow is only one of countless cases of this.
>

Checking preconditions is a good thing. The fact that we can't have the
good thing in some instances is not an argument for not having it in other
instances.


>
> Choosing to enforce invariants in the type system or checking them at
> runtime is always a compromise, and Rust eschews runtime checks not
> strictly required for memory safety.
>
> In some cases, the type system has been leveraged to enforce invariants
> at compile-time (Ord vs. PartialOrd) but even though that's quite easy
> to sidestep, it's not without drawbacks.
>
> > I also agree that performance is non-negotiable in this case, however.
> > The only good thing about always wrong is that it's not that hard to do
> > better.
> >
> > Given the circumstances, I think the least bad outcome we could achieve,
> > and which we *should* aim to achieve, would be this:
> >
> >  * Where performance is known to not be a requirement, Rust code in the
> > wild uses either overflow-checked arithmetic or unbounded integer types,
> > with the choice between them depending on ergonomic and semantic
> > considerations.
> >
> >  * When the performance requirement can't be ruled out, Rust code in the
> > wild uses arithmetic for which overflow checking can be turned on or off
> > with a compiler flag. For testing and debugging, it is turned on. For
> > production and benchmarks, it is turned off.
>
> The Rust developers have been consistently opposed to introducing
> dialects of the language via compiler switches.


We already have `debug_assert`. I'm essentially proposing to turn overflow
checks into `debug_assert`s. `debug_assert`s are something we already have,
and not a new dialect.


> I brought up the issue
> of macros and syntax extensions but you've chosen to ignore that.
>

The assumption of bad faith is appreciated. I must have missed it
accidentally. If there's an issue, I'd be interested to know about it.

Could you point out where you brought it up? A Ctrl-F over the thread finds
the above comment as the only occurrence of the words "macro" and
"extension".



>
> >  * For code where wraparound semantics is desired, the appropriate
> > facilities are also available.
> >
> > Given the discussion so far, the design I'd be leaning toward to
> > accomplish the above might be something like this:
> >
> >  * Two sets of fixed-sized integer types are available in the `prelude`.
> >
> >  * `u8`..`u64`, `i8`..`i64`, `int`, and `uint` have unspecified results
> > on overflow (**not** undefined behavior). A compiler flag turns overflow
> > checks on or off. Essentially, the checks are `debug_assert`s, though
> > whether they should be controlled by the same flag is open to debate.
> >
> >  * `uc8`..`uc64`, `ic8`..`ic64`, `intc`, and `uintc` are *always*
> > checked for overflow, regardless of flags. (Names are of course open to
> > bikeshedding.)
> >
> >  * Given that these are not really different semantically, automatic
> > coercions between corresponding types can be considered. (Even then, for
> > `a + b` where `a: int` and `b: intc`, explicit disambiguation would
> > presumably still be required.)
> >
> >  * Unbounded integer types using owned memory allocation are available
> > in the `prelude`. I might prefer to call them `Integer` and `Natural`
> > instead of `BigInt` and `BigUint`.
> >
> >  * Types and/or operations which wrap around on overflow are available
> > in the standard library. Given how specialized the use cases for these
> > seem to be, perhaps they could even go directly in the `hash` module.
> > It's not clear to me yet whether a separate set of types (`uw8`..`uw64`,
> > `iw8`..`iw64`) or just a separate set of operation