Re: On Java 9+ support, Cleaners, modules and the death of reflection
For those following, I have a PR up at https://github.com/apache/spark/pull/22993 The implication is that ignoring MaxDirectMemorySize doesn't work out of the box in Java 9+ now. However, you can make it work by setting JVM flags to allow access to the new Cleaner class. Or set MaxDirectMemorySize if it's an issue. Or do nothing if you don't actually run up against the MaxDirectMemorySize limit, which seems to default to equal the size of the JVM heap. On Thu, Nov 8, 2018 at 12:46 PM Sean Owen wrote: > > I think this is a key thread, perhaps one of the only big problems, > for Java 9+ support: > > https://issues.apache.org/jira/browse/SPARK-24421?focusedCommentId=16680169=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16680169 > > We basically can't access a certain method (Cleaner.clean()) anymore > without the user adding JVM flags to allow it. As far as I can tell. > I'm working out what the alternatives or implications are. > > Thoughts welcome. - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
On Java 9+ support, Cleaners, modules and the death of reflection
I think this is a key thread, perhaps one of the only big problems, for Java 9+ support: https://issues.apache.org/jira/browse/SPARK-24421?focusedCommentId=16680169=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16680169 We basically can't access a certain method (Cleaner.clean()) anymore without the user adding JVM flags to allow it. As far as I can tell. I'm working out what the alternatives or implications are. Thoughts welcome. - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: Java 9
Hi, Let me fill you in on the Quality Outreach. I send out email every 2-3 weeks depending on contents of the builds, example attached. I try to highlight significant changes in the builds, allowing you to decide if you might want to test with a particular build. We don't expect you to test every build, it's entirely up to you. If you would like us to list your project(s) on the Quality Outreach wiki [1] , can you provide a contact name , mailing list , the current status of you testing against JDK 8 & JDK 9, and a CI if possible ? Rgds, Rory [1] https://wiki.openjdk.java.net/display/quality/Quality+Outreach On 09/02/2017 10:10, Felix Yang wrote: Excuse me, more detail of JDK 9 can be found here . http://openjdk.java.net/projects/jdk9/ The changes mentioned below were tracked in http://openjdk.java.net/jeps/260 If you want something digested, technical sessions in recent JavaOne would be quite helpful. There is a Quality Outreach program. Welcome to join and feedback. Quality Outreach: https://wiki.openjdk.java.net/display/quality/Quality+Outreach Thanks, Felix On 2017/2/8 13:09, kant kodali wrote: Well and the module system! On Tue, Feb 7, 2017 at 4:03 AM, Timur Shenkao <t...@timshenkao.su <mailto:t...@timshenkao.su>> wrote: If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9.* * This class is till used by several libraries & frameworks. http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ <http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/> On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins <robbin...@gmail.com <mailto:robbin...@gmail.com>> wrote: Yes, I agree but it may be worthwhile starting to look at this. I was just trying a build and it trips over some of the now defunct/inaccessible sun.misc classes. I was just interested in hearing if anyone has already gone through this to save me duplicating effort. Cheers, On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com <mailto:so...@cloudera.com>> wrote: I don't think anyone's tried it. I think we'd first have to agree to drop Java 7 support before that could be seriously considered. The 8-9 difference is a bit more of a breaking change. On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com <mailto:robbin...@gmail.com>> wrote: Is anyone working on support for running Spark on Java 9? Is this in a roadmap anywhere? Cheers, -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland --- Begin Message --- JDK 9 Early Access b151 is available on java.net There have been a number of fixes to bugs reported by Open Source projects since the last availability email : JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner JDK-8075793 : Source incompatibility for inference using -source 7 JDK-8087303 : LSSerializer pretty print does not work anymore JDK-8167143 :CLDR timezone parsing does not work for all locales Other changes that maybe of interest: JDK-8066474 : Remove the lib/$ARCH directory from Linux and Solaris images JDK-8170428 : Move src.zip to JDK/lib/src.zip JEPs intergrated: JEP 295: Ahead-of-Time Compilation has been integrated in b150. Schedule - Milestones since last availability email Feature Extension Complete 22nd of December 2016 Rampdown Started 5th of January 2017 Phases in which increasing levels of scrutiny are applied to incoming changes. In phase 1, only P1-P3 bugs can be fixed. In phase 2 only showstopper bugs can be fixed. Rgds,Rory -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland --- End Message --- - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: Java 9
tl;dr:
The critical internal APIs proposed to remain accessible in JDK 9 are:
sun.misc.{Signal,SignalHandler}
sun.misc.Unsafe (The functionality of many of the methods in this class is
now available via variable handles (JEP 193).)
sun.reflect.Reflection::getCallerClass(int) (The functionality of this
method may be provided in a standard form via JEP 259.)
sun.reflect.ReflectionFactory.newConstructorForSerialization
On Thu, Feb 9, 2017 at 11:10 AM, Felix Yang <felix.y...@oracle.com> wrote:
> Excuse me,
>
>more detail of JDK 9 can be found here .
>
> http://openjdk.java.net/projects/jdk9/
>
> The changes mentioned below were tracked in http://openjdk.java.net/jeps/
> 260
>
> If you want something digested, technical sessions in recent JavaOne would
> be quite helpful.
>
>
> There is a Quality Outreach program. Welcome to join and feedback.
>
> Quality Outreach: https://wiki.openjdk.java.net/display/quality/Quality+
> Outreach
> Thanks,
> Felix
>
> On 2017/2/8 13:09, kant kodali wrote:
>
> Well and the module system!
>
> On Tue, Feb 7, 2017 at 4:03 AM, Timur Shenkao <t...@timshenkao.su> wrote:
>
>> If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9.
>>
>> This class is till used by several libraries & frameworks.
>>
>> http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/
>>
>> On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins <robbin...@gmail.com>
>> wrote:
>>
>>> Yes, I agree but it may be worthwhile starting to look at this. I was
>>> just trying a build and it trips over some of the now defunct/inaccessible
>>> sun.misc classes.
>>>
>>> I was just interested in hearing if anyone has already gone through this
>>> to save me duplicating effort.
>>>
>>> Cheers,
>>>
>>> On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com> wrote:
>>>
>>>> I don't think anyone's tried it. I think we'd first have to agree to
>>>> drop Java 7 support before that could be seriously considered. The 8-9
>>>> difference is a bit more of a breaking change.
>>>>
>>>> On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com>
>>>> wrote:
>>>>
>>>> Is anyone working on support for running Spark on Java 9? Is this in a
>>>> roadmap anywhere?
>>>>
>>>>
>>>> Cheers,
>>>>
>>>>
>>
>
>
Re: Java 9
Excuse me, more detail of JDK 9 can be found here . http://openjdk.java.net/projects/jdk9/ The changes mentioned below were tracked in http://openjdk.java.net/jeps/260 If you want something digested, technical sessions in recent JavaOne would be quite helpful. There is a Quality Outreach program. Welcome to join and feedback. Quality Outreach: https://wiki.openjdk.java.net/display/quality/Quality+Outreach Thanks, Felix On 2017/2/8 13:09, kant kodali wrote: Well and the module system! On Tue, Feb 7, 2017 at 4:03 AM, Timur Shenkao <t...@timshenkao.su <mailto:t...@timshenkao.su>> wrote: If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9.* * This class is till used by several libraries & frameworks. http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ <http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/> On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins <robbin...@gmail.com <mailto:robbin...@gmail.com>> wrote: Yes, I agree but it may be worthwhile starting to look at this. I was just trying a build and it trips over some of the now defunct/inaccessible sun.misc classes. I was just interested in hearing if anyone has already gone through this to save me duplicating effort. Cheers, On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com <mailto:so...@cloudera.com>> wrote: I don't think anyone's tried it. I think we'd first have to agree to drop Java 7 support before that could be seriously considered. The 8-9 difference is a bit more of a breaking change. On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com <mailto:robbin...@gmail.com>> wrote: Is anyone working on support for running Spark on Java 9? Is this in a roadmap anywhere? Cheers,
Re: Java 9
Well and the module system! On Tue, Feb 7, 2017 at 4:03 AM, Timur Shenkao <t...@timshenkao.su> wrote: > If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9. > > This class is till used by several libraries & frameworks. > > http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ > > On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins <robbin...@gmail.com> wrote: > >> Yes, I agree but it may be worthwhile starting to look at this. I was >> just trying a build and it trips over some of the now defunct/inaccessible >> sun.misc classes. >> >> I was just interested in hearing if anyone has already gone through this >> to save me duplicating effort. >> >> Cheers, >> >> On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com> wrote: >> >>> I don't think anyone's tried it. I think we'd first have to agree to >>> drop Java 7 support before that could be seriously considered. The 8-9 >>> difference is a bit more of a breaking change. >>> >>> On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com> >>> wrote: >>> >>> Is anyone working on support for running Spark on Java 9? Is this in a >>> roadmap anywhere? >>> >>> >>> Cheers, >>> >>> >
Re: Java 9
If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9. This class is till used by several libraries & frameworks. http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins <robbin...@gmail.com> wrote: > Yes, I agree but it may be worthwhile starting to look at this. I was just > trying a build and it trips over some of the now defunct/inaccessible > sun.misc classes. > > I was just interested in hearing if anyone has already gone through this > to save me duplicating effort. > > Cheers, > > On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com> wrote: > >> I don't think anyone's tried it. I think we'd first have to agree to drop >> Java 7 support before that could be seriously considered. The 8-9 >> difference is a bit more of a breaking change. >> >> On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com> wrote: >> >> Is anyone working on support for running Spark on Java 9? Is this in a >> roadmap anywhere? >> >> >> Cheers, >> >>
Re: Java 9
Yes, I agree but it may be worthwhile starting to look at this. I was just trying a build and it trips over some of the now defunct/inaccessible sun.misc classes. I was just interested in hearing if anyone has already gone through this to save me duplicating effort. Cheers, On Tue, 7 Feb 2017 at 11:46 Sean Owen <so...@cloudera.com> wrote: > I don't think anyone's tried it. I think we'd first have to agree to drop > Java 7 support before that could be seriously considered. The 8-9 > difference is a bit more of a breaking change. > > On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com> wrote: > > Is anyone working on support for running Spark on Java 9? Is this in a > roadmap anywhere? > > > Cheers, > >
Re: Java 9
I don't think anyone's tried it. I think we'd first have to agree to drop Java 7 support before that could be seriously considered. The 8-9 difference is a bit more of a breaking change. On Tue, Feb 7, 2017 at 11:44 AM Pete Robbins <robbin...@gmail.com> wrote: > Is anyone working on support for running Spark on Java 9? Is this in a > roadmap anywhere? > > > Cheers, >
Java 9
Is anyone working on support for running Spark on Java 9? Is this in a roadmap anywhere? Cheers,
