Re: RFR: 8254231: Implementation of Foreign Linker API (Incubator) [v14]
On Mon, 9 Nov 2020 03:56:38 GMT, David Holmes wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Fix typo in upcall helper for aarch64 > > src/java.base/share/classes/java/lang/System.java line 2086: > >> 2084: break; >> 2085: case "allow": >> 2086: allowSecurityManager = MAYBE; > > Why is this removed? I don't see the connection. Seems to be a problem from a merge gone wrong. Not related as you say. Will remove - PR: https://git.openjdk.java.net/jdk/pull/634
Re: RFR: 8254231: Implementation of Foreign Linker API (Incubator) [v14]
On Fri, 30 Oct 2020 12:16:02 GMT, Maurizio Cimadamore wrote: >> This patch contains the changes associated with the first incubation round >> of the foreign linker access API incubation >> (see JEP 389 [1]). This work is meant to sit on top of the foreign memory >> access support (see JEP 393 [2] and associated pull request [3]). >> >> The main goal of this API is to provide a way to call native functions from >> Java code without the need of intermediate JNI glue code. In order to do >> this, native calls are modeled through the MethodHandle API. I suggest >> reading the writeup [4] I put together few weeks ago, which illustrates what >> the foreign linker support is, and how it should be used by clients. >> >> Disclaimer: the pull request mechanism isn't great at managing *dependent* >> reviews. For this reasons, I'm attaching a webrev which contains only the >> differences between this PR and the memory access PR. I will be periodically >> uploading new webrevs, as new iterations come out, to try and make the life >> of reviewers as simple as possible. >> >> A big thank to Jorn Vernee and Vladimir Ivanov - they are the main >> architects of all the hotspot changes you see here, and without their help, >> the foreign linker support wouldn't be what it is today. As usual, a big >> thank to Paul Sandoz, who provided many insights (often by trying the bits >> first hand). >> >> Thanks >> Maurizio >> >> Webrev: >> http://cr.openjdk.java.net/~mcimadamore/8254231_v1/webrev >> >> Javadoc: >> >> http://cr.openjdk.java.net/~mcimadamore/8254231_v1/javadoc/jdk/incubator/foreign/package-summary.html >> >> Specdiff (relative to [3]): >> >> http://cr.openjdk.java.net/~mcimadamore/8254231_v1/specdiff_delta/overview-summary.html >> >> CSR: >> >> https://bugs.openjdk.java.net/browse/JDK-8254232 >> >> >> >> ### API Changes >> >> The API changes are actually rather slim: >> >> * `LibraryLookup` >> * This class allows clients to lookup symbols in native libraries; the >> interface is fairly simple; you can load a library by name, or absolute >> path, and then lookup symbols on that library. >> * `FunctionDescriptor` >> * This is an abstraction that is very similar, in spirit, to `MethodType`; >> it is, at its core, an aggregate of memory layouts for the function >> arguments/return type. A function descriptor is used to describe the >> signature of a native function. >> * `CLinker` >> * This is the real star of the show. A `CLinker` has two main methods: >> `downcallHandle` and `upcallStub`; the first takes a native symbol (as >> obtained from `LibraryLookup`), a `MethodType` and a `FunctionDescriptor` >> and returns a `MethodHandle` instance which can be used to call the target >> native symbol. The second takes an existing method handle, and a >> `FunctionDescriptor` and returns a new `MemorySegment` corresponding to a >> code stub allocated by the VM which acts as a trampoline from native code to >> the user-provided method handle. This is very useful for implementing >> upcalls. >>* This class also contains the various layout constants that should be >> used by clients when describing native signatures (e.g. `C_LONG` and >> friends); these layouts contain additional ABI classfication information (in >> the form of layout attributes) which is used by the runtime to *infer* how >> Java arguments should be shuffled for the native call to take place. >> * Finally, this class provides some helper functions e.g. so that clients >> can convert Java strings into C strings and back. >> * `NativeScope` >> * This is an helper class which allows clients to group together logically >> related allocations; that is, rather than allocating separate memory >> segments using separate *try-with-resource* constructs, a `NativeScope` >> allows clients to use a _single_ block, and allocate all the required >> segments there. This is not only an usability boost, but also a performance >> boost, since not all allocation requests will be turned into `malloc` calls. >> * `MemorySegment` >> * Only one method added here - namely `handoff(NativeScope)` which allows >> a segment to be transferred onto an existing native scope. >> >> ### Safety >> >> The foreign linker API is intrinsically unsafe; many things can go wrong >> when requesting a native method handle. For instance, the description of the >> native signature might be wrong (e.g. have too many arguments) - and the >> runtime has, in the general case, no way to detect such mismatches. For >> these reasons, obtaining a `CLinker` instance is a *restricted* operation, >> which can be enabled by specifying the usual JDK property >> `-Dforeign.restricted=permit` (as it's the case for other restricted method >> in the foreign memory API). >> >> ### Implementation changes >> >> The Java changes associated with `LibraryLookup` are relative >> straightforward; the only interesting thing to note here is that library
Re: RFR: 8254231: Implementation of Foreign Linker API (Incubator) [v14]
> This patch contains the changes associated with the first incubation round of > the foreign linker access API incubation > (see JEP 389 [1]). This work is meant to sit on top of the foreign memory > access support (see JEP 393 [2] and associated pull request [3]). > > The main goal of this API is to provide a way to call native functions from > Java code without the need of intermediate JNI glue code. In order to do > this, native calls are modeled through the MethodHandle API. I suggest > reading the writeup [4] I put together few weeks ago, which illustrates what > the foreign linker support is, and how it should be used by clients. > > Disclaimer: the pull request mechanism isn't great at managing *dependent* > reviews. For this reasons, I'm attaching a webrev which contains only the > differences between this PR and the memory access PR. I will be periodically > uploading new webrevs, as new iterations come out, to try and make the life > of reviewers as simple as possible. > > A big thank to Jorn Vernee and Vladimir Ivanov - they are the main architects > of all the hotspot changes you see here, and without their help, the foreign > linker support wouldn't be what it is today. As usual, a big thank to Paul > Sandoz, who provided many insights (often by trying the bits first hand). > > Thanks > Maurizio > > Webrev: > http://cr.openjdk.java.net/~mcimadamore/8254231_v1/webrev > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254231_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff (relative to [3]): > > http://cr.openjdk.java.net/~mcimadamore/8254231_v1/specdiff_delta/overview-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254232 > > > > ### API Changes > > The API changes are actually rather slim: > > * `LibraryLookup` > * This class allows clients to lookup symbols in native libraries; the > interface is fairly simple; you can load a library by name, or absolute path, > and then lookup symbols on that library. > * `FunctionDescriptor` > * This is an abstraction that is very similar, in spirit, to `MethodType`; > it is, at its core, an aggregate of memory layouts for the function > arguments/return type. A function descriptor is used to describe the > signature of a native function. > * `CLinker` > * This is the real star of the show. A `CLinker` has two main methods: > `downcallHandle` and `upcallStub`; the first takes a native symbol (as > obtained from `LibraryLookup`), a `MethodType` and a `FunctionDescriptor` and > returns a `MethodHandle` instance which can be used to call the target native > symbol. The second takes an existing method handle, and a > `FunctionDescriptor` and returns a new `MemorySegment` corresponding to a > code stub allocated by the VM which acts as a trampoline from native code to > the user-provided method handle. This is very useful for implementing upcalls. >* This class also contains the various layout constants that should be > used by clients when describing native signatures (e.g. `C_LONG` and > friends); these layouts contain additional ABI classfication information (in > the form of layout attributes) which is used by the runtime to *infer* how > Java arguments should be shuffled for the native call to take place. > * Finally, this class provides some helper functions e.g. so that clients > can convert Java strings into C strings and back. > * `NativeScope` > * This is an helper class which allows clients to group together logically > related allocations; that is, rather than allocating separate memory segments > using separate *try-with-resource* constructs, a `NativeScope` allows clients > to use a _single_ block, and allocate all the required segments there. This > is not only an usability boost, but also a performance boost, since not all > allocation requests will be turned into `malloc` calls. > * `MemorySegment` > * Only one method added here - namely `handoff(NativeScope)` which allows a > segment to be transferred onto an existing native scope. > > ### Safety > > The foreign linker API is intrinsically unsafe; many things can go wrong when > requesting a native method handle. For instance, the description of the > native signature might be wrong (e.g. have too many arguments) - and the > runtime has, in the general case, no way to detect such mismatches. For these > reasons, obtaining a `CLinker` instance is a *restricted* operation, which > can be enabled by specifying the usual JDK property > `-Dforeign.restricted=permit` (as it's the case for other restricted method > in the foreign memory API). > > ### Implementation changes > > The Java changes associated with `LibraryLookup` are relative > straightforward; the only interesting thing to note here is that library > loading does _not_ depend on class loaders, so `LibraryLookup` is not subject > to the same restrictions which apply to JNI library loading (e.g. same > library
