Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Andrew Haley]

On 08/20/2018 08:25 PM, Jason Greene wrote:
> 
>> On Aug 20, 2018, at 12:23 PM, Andrew Haley  wrote:
>>
>> On 08/20/2018 04:14 PM, Jason Greene wrote:
>>
>>> IMO departing from C semantics (non-zero = TRUE, zero = false)
>>> offers little gain and will likely just lead to hard to catch
>>> bugs. Even if the JNI developer knows the rules, it will be quite
>>> easy for surprises to show up. For example, a developer might assume
>>> a library call always returns 1, and wire it straight to a
>>> boolean. If this assumption is broken (a condition not taken into
>>> account, or future behavioral differences), then it could quickly
>>> turn into a lot of wasted time and effort.
>>
>> It's tricky, though: a C implementation could silently truncate a
>> value of 0xff00_ to (jboolean) 0x00, and there's not a damn thing
>> that a JVM can do about it: there's literally no way to know.  It's
>> pretty much unfixable from our end.
> 
> OK but in that case the C developer can see its an unsigned char,

It's a jboolean.

> and their compiler is likely to throw an overflow warning.

Mmm, but GCC desn't.

> Granted, it’s also true that int is a more common return value than
> unsigned char.

The function in question is *defined* as returning a jboolean: there's
no doubt about that. The question is whether a C implementation
truncates return values, and you can only see that by reading the
system ABI specification. Not every system documents it: the only way
until recently to find out what GCC on x86 did was to look at the
generated code.

The system ABI is different on different systems: on AArch64 a
jboolean return value is silently truncated, on x86 it isn't. The only
solution is to do what's suggested: make sure that you return
JNI_TRUE/JNI_FALSE (or 1/0).

It's all too horrible even to think about.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Jason Greene]

> On Aug 20, 2018, at 12:23 PM, Andrew Haley  wrote:
> 
> On 08/20/2018 04:14 PM, Jason Greene wrote:
> 
>> IMO departing from C semantics (non-zero = TRUE, zero = false)
>> offers little gain and will likely just lead to hard to catch
>> bugs. Even if the JNI developer knows the rules, it will be quite
>> easy for surprises to show up. For example, a developer might assume
>> a library call always returns 1, and wire it straight to a
>> boolean. If this assumption is broken (a condition not taken into
>> account, or future behavioral differences), then it could quickly
>> turn into a lot of wasted time and effort.
> 
> It's tricky, though: a C implementation could silently truncate a
> value of 0xff00_ to (jboolean) 0x00, and there's not a damn thing
> that a JVM can do about it: there's literally no way to know.  It's
> pretty much unfixable from our end.

OK but in that case the C developer can see its an unsigned char, and their
compiler is likely to throw an overflow warning. Granted, it’s also true that 
int is a more common return value than unsigned char.

-Jason

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Krystal Mok]

Thanks Volker! I hadn't been following new development much and this is
great!
I just checked the latest HS sources [1] and my old code is still there.
I'll prepare a webrev to remove that workaround code.

Thanks,
Kris

[1]:
http://hg.openjdk.java.net/jdk/jdk/file/8dfed4387312/src/hotspot/share/c1/c1_Runtime1.cpp#l1380

On Mon, Aug 20, 2018 at 9:28 AM, Volker Simonis 
wrote:

> On Mon, Aug 20, 2018 at 6:09 PM, Krystal Mok 
> wrote:
> > Hi guys,
> >
> > Haha this is fun. I actually hit this issue the hard way and had to
> tweak a
> > bit of my code to accommodate that: I had to return a jint from a
> function
> > that I wanted to return a jbool at first:
> > http://hg.openjdk.java.net/hsx/hsx25/hotspot/diff/
> 8f37087fc13f/src/share/vm/c1/c1_Runtime1.cpp
> >
>
> Interesting, but I think that mis-behavior should have been fixed
> meanwhile by the infamous "Better byte behavior" change [1] (which was
> a security fix, so you won't find too much information about it).
>
> So maybe you can remove your workaround and comment now :)
>
> [1] http://hg.openjdk.java.net/jdk/jdk/rev/291ee208fb72
>
> > - Kris
> >
> > On Mon, Aug 20, 2018 at 8:38 AM Volker Simonis  >
> > wrote:
> >>
> >> On Mon, Aug 20, 2018 at 4:55 PM, Aleksey Shipilev 
> >> wrote:
> >> > On 08/20/2018 12:22 PM, Volker Simonis wrote:
> >> >> So to summarize, my current view on this topic is:
> >> >>  - JNI functions returning a jboolean are only allowed to return
> >> >> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI
> spcification.
> >> >
> >> > Now *I* am having trouble seeing where exactly the JNI spec says the
> >> > domain of jboolean is
> >> > (JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following
> >> > definition is provided for
> >> > convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the
> >> > domain, because those are
> >> > "convenience" defines. And "Description" in the table says jboolean is
> >> > "unsigned 8 bits", which
> >> > seems to invite interpretation that all 8 bits are usable.
> >> >
> >> > John says [2]:
> >> >
> >> > "The JNI documents specify that, at least for returning values from
> >> > native methods, a Java boolean
> >> > (T_BOOLEAN) value is converted to the value-set 0..1 by first
> truncating
> >> > to a byte (0..255 or maybe
> >> > -128..127) and then testing against zero."
> >> >
> >> > ...which is what I am looking for, but I cannot find the "JNI
> document"
> >> > that actually says that. I
> >> > can see the idea of that in JVMS [3], but that seems to only apply to
> >> > on-heap booleans, does that
> >> > also extend to jboolean's? Maybe John can point out the JNI document
> >> > where it is said explicitly?
> >> >
> >>
> >> Yes, you're right - there's no exact documentation for neither of the
> >> two possible interpretations. A colleague just pointed me to the
> >> definition of invokestatic in the JVMS [4] which has the following
> >> sentence:
> >>
> >> "If the native method returns a value, the return value of the
> >> platform-dependent code is converted in an implementation-dependent
> >> way to the return type of the native method and pushed onto the
> >> operand stack."
> >>
> >> But then again, it has this unfortunate "implementation-dependent"
> >> which can be interpreted either way :(
> >>
> >> [4]
> >> https://docs.oracle.com/javase/specs/jvms/se10/html/
> jvms-6.html#jvms-6.5.invokestatic
> >>
> >> >
> >> >>  - to code in Java_java_io_Console_echo() should be fixed (as
> >> >> confirmed by Sherman later in this thread)
> >> >
> >> > Yes, that's a bug waiting to happen anyway.
> >> >
> >> >
> >> >>  - normalization of native, off-heap 8-bit values to Java booleans as
> >> >> currently implemented in the HotSpot (and fixed by JDK-8161720) is
> (1)
> >> >> only for convenience to simply access to off-heap data in Unsafe, (2)
> >> >> to implement better Java/Native integration in projects like Panama
> >> >> and (3) to fix legacy JNI code which was developed under the
> >> >> assumption that the advice in the "JNI Programmer's Guide &
> >> >> Specification" book is specification relevant.
> >> >
> >> > Yes, the intent seems to be what you describe. But see above about the
> >> > spec.
> >> >
> >> >
> >> > -Aleksey
> >> >
> >> > [1]
> >> > https://docs.oracle.com/javase/8/docs/technotes/
> guides/jni/spec/types.html#primitive_types
> >> > [2]
> >> > http://mail.openjdk.java.net/pipermail/hotspot-compiler-
> dev/2016-August/024263.html
> >> > [3]
> >> > https://docs.oracle.com/javase/specs/jvms/se10/html/
> jvms-2.html#jvms-2.3.4
> >> >
>

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Andrew Haley]

On 08/20/2018 04:14 PM, Jason Greene wrote:

> IMO departing from C semantics (non-zero = TRUE, zero = false)
> offers little gain and will likely just lead to hard to catch
> bugs. Even if the JNI developer knows the rules, it will be quite
> easy for surprises to show up. For example, a developer might assume
> a library call always returns 1, and wire it straight to a
> boolean. If this assumption is broken (a condition not taken into
> account, or future behavioral differences), then it could quickly
> turn into a lot of wasted time and effort.

It's tricky, though: a C implementation could silently truncate a
value of 0xff00_ to (jboolean) 0x00, and there's not a damn thing
that a JVM can do about it: there's literally no way to know.  It's
pretty much unfixable from our end.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Volker Simonis]

On Mon, Aug 20, 2018 at 6:09 PM, Krystal Mok  wrote:
> Hi guys,
>
> Haha this is fun. I actually hit this issue the hard way and had to tweak a
> bit of my code to accommodate that: I had to return a jint from a function
> that I wanted to return a jbool at first:
> http://hg.openjdk.java.net/hsx/hsx25/hotspot/diff/8f37087fc13f/src/share/vm/c1/c1_Runtime1.cpp
>

Interesting, but I think that mis-behavior should have been fixed
meanwhile by the infamous "Better byte behavior" change [1] (which was
a security fix, so you won't find too much information about it).

So maybe you can remove your workaround and comment now :)

[1] http://hg.openjdk.java.net/jdk/jdk/rev/291ee208fb72

> - Kris
>
> On Mon, Aug 20, 2018 at 8:38 AM Volker Simonis 
> wrote:
>>
>> On Mon, Aug 20, 2018 at 4:55 PM, Aleksey Shipilev 
>> wrote:
>> > On 08/20/2018 12:22 PM, Volker Simonis wrote:
>> >> So to summarize, my current view on this topic is:
>> >>  - JNI functions returning a jboolean are only allowed to return
>> >> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.
>> >
>> > Now *I* am having trouble seeing where exactly the JNI spec says the
>> > domain of jboolean is
>> > (JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following
>> > definition is provided for
>> > convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the
>> > domain, because those are
>> > "convenience" defines. And "Description" in the table says jboolean is
>> > "unsigned 8 bits", which
>> > seems to invite interpretation that all 8 bits are usable.
>> >
>> > John says [2]:
>> >
>> > "The JNI documents specify that, at least for returning values from
>> > native methods, a Java boolean
>> > (T_BOOLEAN) value is converted to the value-set 0..1 by first truncating
>> > to a byte (0..255 or maybe
>> > -128..127) and then testing against zero."
>> >
>> > ...which is what I am looking for, but I cannot find the "JNI document"
>> > that actually says that. I
>> > can see the idea of that in JVMS [3], but that seems to only apply to
>> > on-heap booleans, does that
>> > also extend to jboolean's? Maybe John can point out the JNI document
>> > where it is said explicitly?
>> >
>>
>> Yes, you're right - there's no exact documentation for neither of the
>> two possible interpretations. A colleague just pointed me to the
>> definition of invokestatic in the JVMS [4] which has the following
>> sentence:
>>
>> "If the native method returns a value, the return value of the
>> platform-dependent code is converted in an implementation-dependent
>> way to the return type of the native method and pushed onto the
>> operand stack."
>>
>> But then again, it has this unfortunate "implementation-dependent"
>> which can be interpreted either way :(
>>
>> [4]
>> https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-6.html#jvms-6.5.invokestatic
>>
>> >
>> >>  - to code in Java_java_io_Console_echo() should be fixed (as
>> >> confirmed by Sherman later in this thread)
>> >
>> > Yes, that's a bug waiting to happen anyway.
>> >
>> >
>> >>  - normalization of native, off-heap 8-bit values to Java booleans as
>> >> currently implemented in the HotSpot (and fixed by JDK-8161720) is (1)
>> >> only for convenience to simply access to off-heap data in Unsafe, (2)
>> >> to implement better Java/Native integration in projects like Panama
>> >> and (3) to fix legacy JNI code which was developed under the
>> >> assumption that the advice in the "JNI Programmer's Guide &
>> >> Specification" book is specification relevant.
>> >
>> > Yes, the intent seems to be what you describe. But see above about the
>> > spec.
>> >
>> >
>> > -Aleksey
>> >
>> > [1]
>> > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#primitive_types
>> > [2]
>> > http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-August/024263.html
>> > [3]
>> > https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-2.html#jvms-2.3.4
>> >

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Krystal Mok]

Hi guys,

Haha this is fun. I actually hit this issue the hard way and had to tweak a
bit of my code to accommodate that: I had to return a jint from a function
that I wanted to return a jbool at first:
http://hg.openjdk.java.net/hsx/hsx25/hotspot/diff/8f37087fc13f/src/share/vm/c1/c1_Runtime1.cpp

- Kris

On Mon, Aug 20, 2018 at 8:38 AM Volker Simonis 
wrote:

> On Mon, Aug 20, 2018 at 4:55 PM, Aleksey Shipilev 
> wrote:
> > On 08/20/2018 12:22 PM, Volker Simonis wrote:
> >> So to summarize, my current view on this topic is:
> >>  - JNI functions returning a jboolean are only allowed to return
> >> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.
> >
> > Now *I* am having trouble seeing where exactly the JNI spec says the
> domain of jboolean is
> > (JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following
> definition is provided for
> > convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the
> domain, because those are
> > "convenience" defines. And "Description" in the table says jboolean is
> "unsigned 8 bits", which
> > seems to invite interpretation that all 8 bits are usable.
> >
> > John says [2]:
> >
> > "The JNI documents specify that, at least for returning values from
> native methods, a Java boolean
> > (T_BOOLEAN) value is converted to the value-set 0..1 by first truncating
> to a byte (0..255 or maybe
> > -128..127) and then testing against zero."
> >
> > ...which is what I am looking for, but I cannot find the "JNI document"
> that actually says that. I
> > can see the idea of that in JVMS [3], but that seems to only apply to
> on-heap booleans, does that
> > also extend to jboolean's? Maybe John can point out the JNI document
> where it is said explicitly?
> >
>
> Yes, you're right - there's no exact documentation for neither of the
> two possible interpretations. A colleague just pointed me to the
> definition of invokestatic in the JVMS [4] which has the following
> sentence:
>
> "If the native method returns a value, the return value of the
> platform-dependent code is converted in an implementation-dependent
> way to the return type of the native method and pushed onto the
> operand stack."
>
> But then again, it has this unfortunate "implementation-dependent"
> which can be interpreted either way :(
>
> [4]
> https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-6.html#jvms-6.5.invokestatic
>
> >
> >>  - to code in Java_java_io_Console_echo() should be fixed (as
> >> confirmed by Sherman later in this thread)
> >
> > Yes, that's a bug waiting to happen anyway.
> >
> >
> >>  - normalization of native, off-heap 8-bit values to Java booleans as
> >> currently implemented in the HotSpot (and fixed by JDK-8161720) is (1)
> >> only for convenience to simply access to off-heap data in Unsafe, (2)
> >> to implement better Java/Native integration in projects like Panama
> >> and (3) to fix legacy JNI code which was developed under the
> >> assumption that the advice in the "JNI Programmer's Guide &
> >> Specification" book is specification relevant.
> >
> > Yes, the intent seems to be what you describe. But see above about the
> spec.
> >
> >
> > -Aleksey
> >
> > [1]
> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#primitive_types
> > [2]
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-August/024263.html
> > [3]
> https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-2.html#jvms-2.3.4
> >
>

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Volker Simonis]

On Mon, Aug 20, 2018 at 4:55 PM, Aleksey Shipilev  wrote:
> On 08/20/2018 12:22 PM, Volker Simonis wrote:
>> So to summarize, my current view on this topic is:
>>  - JNI functions returning a jboolean are only allowed to return
>> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.
>
> Now *I* am having trouble seeing where exactly the JNI spec says the domain 
> of jboolean is
> (JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following 
> definition is provided for
> convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the domain, 
> because those are
> "convenience" defines. And "Description" in the table says jboolean is 
> "unsigned 8 bits", which
> seems to invite interpretation that all 8 bits are usable.
>
> John says [2]:
>
> "The JNI documents specify that, at least for returning values from native 
> methods, a Java boolean
> (T_BOOLEAN) value is converted to the value-set 0..1 by first truncating to a 
> byte (0..255 or maybe
> -128..127) and then testing against zero."
>
> ...which is what I am looking for, but I cannot find the "JNI document" that 
> actually says that. I
> can see the idea of that in JVMS [3], but that seems to only apply to on-heap 
> booleans, does that
> also extend to jboolean's? Maybe John can point out the JNI document where it 
> is said explicitly?
>

Yes, you're right - there's no exact documentation for neither of the
two possible interpretations. A colleague just pointed me to the
definition of invokestatic in the JVMS [4] which has the following
sentence:

"If the native method returns a value, the return value of the
platform-dependent code is converted in an implementation-dependent
way to the return type of the native method and pushed onto the
operand stack."

But then again, it has this unfortunate "implementation-dependent"
which can be interpreted either way :(

[4] 
https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-6.html#jvms-6.5.invokestatic

>
>>  - to code in Java_java_io_Console_echo() should be fixed (as
>> confirmed by Sherman later in this thread)
>
> Yes, that's a bug waiting to happen anyway.
>
>
>>  - normalization of native, off-heap 8-bit values to Java booleans as
>> currently implemented in the HotSpot (and fixed by JDK-8161720) is (1)
>> only for convenience to simply access to off-heap data in Unsafe, (2)
>> to implement better Java/Native integration in projects like Panama
>> and (3) to fix legacy JNI code which was developed under the
>> assumption that the advice in the "JNI Programmer's Guide &
>> Specification" book is specification relevant.
>
> Yes, the intent seems to be what you describe. But see above about the spec.
>
>
> -Aleksey
>
> [1] 
> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#primitive_types
> [2] 
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-August/024263.html
> [3] https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-2.html#jvms-2.3.4
>

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Jason Greene]

> On Aug 20, 2018, at 9:55 AM, Aleksey Shipilev  wrote:
> 
> On 08/20/2018 12:22 PM, Volker Simonis wrote:
>> So to summarize, my current view on this topic is:
>> - JNI functions returning a jboolean are only allowed to return
>> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.
> 
> Now *I* am having trouble seeing where exactly the JNI spec says the domain 
> of jboolean is
> (JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following 
> definition is provided for
> convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the domain, 
> because those are
> "convenience" defines. And "Description" in the table says jboolean is 
> "unsigned 8 bits", which
> seems to invite interpretation that all 8 bits are usable.
> 
> John says [2]:
> 
> "The JNI documents specify that, at least for returning values from native 
> methods, a Java boolean
> (T_BOOLEAN) value is converted to the value-set 0..1 by first truncating to a 
> byte (0..255 or maybe
> -128..127) and then testing against zero."
> 
> ...which is what I am looking for, but I cannot find the "JNI document" that 
> actually says that. I
> can see the idea of that in JVMS [3], but that seems to only apply to on-heap 
> booleans, does that
> also extend to jboolean's? Maybe John can point out the JNI document where it 
> is said explicitly?



IMO departing from C semantics (non-zero = TRUE, zero = false) offers little 
gain and will likely just lead to hard to catch bugs. Even if the JNI developer 
knows the rules, it will be quite easy for surprises to show up. For example, a 
developer might assume a library call always returns 1, and wire it straight to 
a boolean. If this assumption is broken (a condition not taken into account, or 
future behavioral differences), then it could quickly turn into a lot of wasted 
time and effort.  The only way to deal with it is to safeguard every call with 
a conversion macro/function, and it’s easy to envision that being forgotten.

-Jason 

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Aleksey Shipilev]

On 08/20/2018 12:22 PM, Volker Simonis wrote:
> So to summarize, my current view on this topic is:
>  - JNI functions returning a jboolean are only allowed to return
> JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.

Now *I* am having trouble seeing where exactly the JNI spec says the domain of 
jboolean is
(JNI_FALSE, JNI_TRUE). In "Primitive Types" [1] it says "The following 
definition is provided for
convenience: JNI_FALSE, JNI_TRUE", but that does not restrict the domain, 
because those are
"convenience" defines. And "Description" in the table says jboolean is 
"unsigned 8 bits", which
seems to invite interpretation that all 8 bits are usable.

John says [2]:

"The JNI documents specify that, at least for returning values from native 
methods, a Java boolean
(T_BOOLEAN) value is converted to the value-set 0..1 by first truncating to a 
byte (0..255 or maybe
-128..127) and then testing against zero."

...which is what I am looking for, but I cannot find the "JNI document" that 
actually says that. I
can see the idea of that in JVMS [3], but that seems to only apply to on-heap 
booleans, does that
also extend to jboolean's? Maybe John can point out the JNI document where it 
is said explicitly?


>  - to code in Java_java_io_Console_echo() should be fixed (as
> confirmed by Sherman later in this thread)

Yes, that's a bug waiting to happen anyway.


>  - normalization of native, off-heap 8-bit values to Java booleans as
> currently implemented in the HotSpot (and fixed by JDK-8161720) is (1)
> only for convenience to simply access to off-heap data in Unsafe, (2)
> to implement better Java/Native integration in projects like Panama
> and (3) to fix legacy JNI code which was developed under the
> assumption that the advice in the "JNI Programmer's Guide &
> Specification" book is specification relevant.

Yes, the intent seems to be what you describe. But see above about the spec.


-Aleksey

[1] 
https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#primitive_types
[2] 
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-August/024263.html
[3] https://docs.oracle.com/javase/specs/jvms/se10/html/jvms-2.html#jvms-2.3.4

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Volker Simonis]

On Fri, Aug 17, 2018 at 7:25 PM, Aleksey Shipilev  wrote:
> On 08/17/2018 05:12 PM, Volker Simonis wrote:
>> The offending code in Console_md.c looks as follows:
>>
>> #define ECHO8
>>
>> JNIEXPORT jboolean JNICALL
>> Java_java_io_Console_echo(...) {
>>
>> jboolean old;
>> ...
>> old = (tio.c_lflag & ECHO);
>> ...
>> return old;
>> }
>>
>> The intention of this code is to return "true" if the ECHO flag was
>> set but it really returns the value of ECHO (which is defined as '8'
>> in a system header).
>>
>> The question now is, if a Java SE compatible VM guarantees that any
>> arbitrary, non-zero valued jboolean will be interpreted as "JNI_TRUE"
>> and only a zero valued jboolean will be interpreted as "JNI_FALSE"?
>>
>> Or, the other way round, is the normalization performed by the HotSpot
>> result handlers necessary (i.e. enforced by the specification) or just
>> a convenience to fix broken code like the above from Console.echo()?
>
> I think this is intentional aftermath of boolean value normalization:
>   https://bugs.openjdk.java.net/browse/JDK-8161720
>

OMG - I'm getting old :)

Thanks for the link to JDK-8161720! Funny enough I was one of the
reviewers of JDK-8161720 and almost exactly two years ago I already
asked the exactly same question [1]:

"And I have a question about JNI: is it only a convention that JNI
functions should return JNI_TRUE/JNI_FALSE or is this required by the
specification?"

Zoltan answered to that question by citing an excerpt from the JNI
Programmer's Guide & Specification [2]:

"A |jboolean| is an 8-bit unsigned C type that can store values from 0
to 255. The value 0 corresponds to the constant |JNI_FALSE|, and the
values from 1 to 255 correspond to |JNI_TRUE|."

But the "JNI Programmer's Guide & Specification" is from 1999 and it
only contains the cited sentence in its "Traps and Pitfalls" section
and not in the "Specification" part. The latest JNI Specification [3]
doesn't seem to contain such a rule.

So to summarize, my current view on this topic is:
 - JNI functions returning a jboolean are only allowed to return
JNI_TRUE/JNI_FALSE (or 1/0) according to the current JNI spcification.
 - to code in Java_java_io_Console_echo() should be fixed (as
confirmed by Sherman later in this thread)
 - normalization of native, off-heap 8-bit values to Java booleans as
currently implemented in the HotSpot (and fixed by JDK-8161720) is (1)
only for convenience to simply access to off-heap data in Unsafe, (2)
to implement better Java/Native integration in projects like Panama
and (3) to fix legacy JNI code which was developed under the
assumption that the advice in the "JNI Programmer's Guide &
Specification" book is specification relevant.

Do you agree?

Thank you and best regards,
Volker

[1] 
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-August/024192.html
[2] 
https://web.archive.org/web/20120626012047/http://java.sun.com/docs/books/jni/html/pitfalls.html#30066
[3] https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/jniTOC.html

> So, Java_java_io_Console_echo looks broken and needs to be fixed, by e.g.:
>
> -   old = (tio.c_lflag & ECHO);
> +   old = (tio.c_lflag & ECHO) != 0;
>
> Thanks,
> -Aleksey
>

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Xueming Shen]

On 08/17/2018 10:25 AM, Aleksey Shipilev wrote:

On 08/17/2018 05:12 PM, Volker Simonis wrote:

The offending code in Console_md.c looks as follows:

#define ECHO8

JNIEXPORT jboolean JNICALL
Java_java_io_Console_echo(...) {

 jboolean old;
 ...
 old = (tio.c_lflag&  ECHO);
 ...
 return old;
}

The intention of this code is to return "true" if the ECHO flag was
set but it really returns the value of ECHO (which is defined as '8'
in a system header).

The question now is, if a Java SE compatible VM guarantees that any
arbitrary, non-zero valued jboolean will be interpreted as "JNI_TRUE"
and only a zero valued jboolean will be interpreted as "JNI_FALSE"?

Or, the other way round, is the normalization performed by the HotSpot
result handlers necessary (i.e. enforced by the specification) or just
a convenience to fix broken code like the above from Console.echo()?

I think this is intentional aftermath of boolean value normalization:
   https://bugs.openjdk.java.net/browse/JDK-8161720

So, Java_java_io_Console_echo looks broken and needs to be fixed, by e.g.:

-   old = (tio.c_lflag&  ECHO);
+   old = (tio.c_lflag&  ECHO) != 0;

Thanks,
-Aleksey



Yes, the code in Console_md.c need/will be updated to fix this issue. We planed 
and
tried in early circle of jdk11 when working on 8194750, at least for the newly 
added
code.  But change had been backed out for other reason. Will fix this specific 
one in
12. But the existing code is about decade old, I would assume you probably 
still need
to fix the s390x part as a "regression".

Thanks!
Sherman

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Aleksey Shipilev]

On 08/17/2018 05:12 PM, Volker Simonis wrote:
> The offending code in Console_md.c looks as follows:
> 
> #define ECHO8
> 
> JNIEXPORT jboolean JNICALL
> Java_java_io_Console_echo(...) {
> 
> jboolean old;
> ...
> old = (tio.c_lflag & ECHO);
> ...
> return old;
> }
> 
> The intention of this code is to return "true" if the ECHO flag was
> set but it really returns the value of ECHO (which is defined as '8'
> in a system header).
> 
> The question now is, if a Java SE compatible VM guarantees that any
> arbitrary, non-zero valued jboolean will be interpreted as "JNI_TRUE"
> and only a zero valued jboolean will be interpreted as "JNI_FALSE"?
> 
> Or, the other way round, is the normalization performed by the HotSpot
> result handlers necessary (i.e. enforced by the specification) or just
> a convenience to fix broken code like the above from Console.echo()?

I think this is intentional aftermath of boolean value normalization:
  https://bugs.openjdk.java.net/browse/JDK-8161720

So, Java_java_io_Console_echo looks broken and needs to be fixed, by e.g.:

-   old = (tio.c_lflag & ECHO);
+   old = (tio.c_lflag & ECHO) != 0;

Thanks,
-Aleksey

Re: Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Roger Riggs]

Hi Volker,

This may be more of a hotspot question than core-libs.

This seems like a practical feature of the implementation and a convenience
to match the conventions of C/C++ with JNI.

It might be worth adding a test to verify the assumptions between Java 
and Native.


$.02, Roger


On 8/17/18 11:12 AM, Volker Simonis wrote:

Hi,

we've recently discovered a problem with Console.readPassword() which
failed to restore the correct echo mode on Linux/s390x [1]. We could
finally track this down to a "bug" [2] in the s390x template
interpreter which failed to call the "result handler" after a native
call. For jboolean return types, the "result handler" merges all
values > 0 into 1 (i.e. JNI_TRUE). Not calling the result handler
after returning from native calls resulted in cases where non-zero
jboolean values returned from native functions where interpreted as
zero (i.e. JNI_FALSE).

The offending code in Console_md.c looks as follows:

#define ECHO8

JNIEXPORT jboolean JNICALL
Java_java_io_Console_echo(...) {

 jboolean old;
 ...
 old = (tio.c_lflag & ECHO);
 ...
 return old;
}

The intention of this code is to return "true" if the ECHO flag was
set but it really returns the value of ECHO (which is defined as '8'
in a system header).

The question now is, if a Java SE compatible VM guarantees that any
arbitrary, non-zero valued jboolean will be interpreted as "JNI_TRUE"
and only a zero valued jboolean will be interpreted as "JNI_FALSE"?

Or, the other way round, is the normalization performed by the HotSpot
result handlers necessary (i.e. enforced by the specification) or just
a convenience to fix broken code like the above from Console.echo()?

All I could find from the Specification so far is the following, which
seems to suggest that only JNI_TRUE/JNI_FASLE are valid values for a
jboolean, but I'm not completely sure...

From: JNI Types and Data Structures [3]

Java "boolean" == JNI "jboolean" == unsigned 8 bits

The following definition is provided for convenience.

#define JNI_FALSE  0
#define JNI_TRUE   1

From: JVMLS, § 2.3.4. The boolean Type [4]

Although the Java Virtual Machine defines a boolean type, it only
provides very limited support for it. There are no Java Virtual
Machine instructions solely dedicated to operations on boolean values.
Instead, expressions in the Java programming language that operate on
boolean values are compiled to use values of the Java Virtual Machine
int data type.

The Java Virtual Machine encodes boolean array components using 1 to
represent true and 0 to represent false. Where Java programming
language boolean values are mapped by compilers to values of Java
Virtual Machine type int, the compilers must use the same encoding.

Thank you and best regards,
Volker

[1] 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/thread.html#54479
[2] https://bugs.openjdk.java.net/browse/JDK-8209637
[3] https://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/types.html
[4] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.3.4

Is returning a value != '0' or '1' as jboolean from a JNI function legal?

[Volker Simonis]

Hi,

we've recently discovered a problem with Console.readPassword() which
failed to restore the correct echo mode on Linux/s390x [1]. We could
finally track this down to a "bug" [2] in the s390x template
interpreter which failed to call the "result handler" after a native
call. For jboolean return types, the "result handler" merges all
values > 0 into 1 (i.e. JNI_TRUE). Not calling the result handler
after returning from native calls resulted in cases where non-zero
jboolean values returned from native functions where interpreted as
zero (i.e. JNI_FALSE).

The offending code in Console_md.c looks as follows:

#define ECHO8

JNIEXPORT jboolean JNICALL
Java_java_io_Console_echo(...) {

jboolean old;
...
old = (tio.c_lflag & ECHO);
...
return old;
}

The intention of this code is to return "true" if the ECHO flag was
set but it really returns the value of ECHO (which is defined as '8'
in a system header).

The question now is, if a Java SE compatible VM guarantees that any
arbitrary, non-zero valued jboolean will be interpreted as "JNI_TRUE"
and only a zero valued jboolean will be interpreted as "JNI_FALSE"?

Or, the other way round, is the normalization performed by the HotSpot
result handlers necessary (i.e. enforced by the specification) or just
a convenience to fix broken code like the above from Console.echo()?

All I could find from the Specification so far is the following, which
seems to suggest that only JNI_TRUE/JNI_FASLE are valid values for a
jboolean, but I'm not completely sure...

From: JNI Types and Data Structures [3]

Java "boolean" == JNI "jboolean" == unsigned 8 bits

The following definition is provided for convenience.

#define JNI_FALSE  0
#define JNI_TRUE   1

From: JVMLS, § 2.3.4. The boolean Type [4]

Although the Java Virtual Machine defines a boolean type, it only
provides very limited support for it. There are no Java Virtual
Machine instructions solely dedicated to operations on boolean values.
Instead, expressions in the Java programming language that operate on
boolean values are compiled to use values of the Java Virtual Machine
int data type.

The Java Virtual Machine encodes boolean array components using 1 to
represent true and 0 to represent false. Where Java programming
language boolean values are mapped by compilers to values of Java
Virtual Machine type int, the compilers must use the same encoding.

Thank you and best regards,
Volker

[1] 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/thread.html#54479
[2] https://bugs.openjdk.java.net/browse/JDK-8209637
[3] https://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/types.html
[4] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.3.4