"+ *setting class-loading strategy to parent-first *could" ... otherwise the classes will be always loaded from the jar provided via the REST API
On Tue, Dec 21, 2021 at 5:46 PM Lior Liviev <[email protected]> wrote: > So again, after putting the jar in that folder I don’t need to configure > anything else? > > Get Outlook for iOS <https://aka.ms/o0ukef> > ------------------------------ > *From:* David Morávek <[email protected]> > *Sent:* Tuesday, December 21, 2021 6:39:10 PM > *To:* Lior Liviev <[email protected]> > *Cc:* user <[email protected]> > *Subject:* Re: Avoiding Dynamic Classloading for User Code > > > *CAUTION*: external source > hmm, with this approach I can only think about not really nice > solutions... I guess putting a jar into `/lib` folder + setting > class-loading strategy to parent-first could do the trick (load everything > in the "main" class loader), but then using this endpoint / deployment path > for submission kind of seems to lack purpose. > > on the other hand, I don't really think we provide a nice way to achieve > this with session cluster right now :( > > D. > > On Tue, Dec 21, 2021 at 5:27 PM Lior Liviev <[email protected]> > wrote: > > Yes, I’m using "/jars/:jarid/run" > > Get Outlook for iOS > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IRLFsUVDrvQctIL1KVrKgr%2FFBnySt28d5OH70EyFBHc%3D&reserved=0> > ------------------------------ > *From:* David Morávek <[email protected]> > *Sent:* Tuesday, December 21, 2021 6:08:51 PM > *To:* Lior Liviev <[email protected]>; user <[email protected]> > *Subject:* Re: Avoiding Dynamic Classloading for User Code > > > *CAUTION*: external source > Please always include the ML in the reply-list, so other can participate > in the discussion / learn from the findings > > we are aware of multiple issues when web-submission can result in > classloader / thread local leaks, which could potentially result in the > behavior you're describing. We're working on addressing them. > > FLINK-25022 [1]: The most critical one leaking thread locals. > FLINK-25027 [2]: Is only a memory improvement for a particular situation (a > lot of small batch jobs) and could be fixed by accounting for when setting > Metaspace size. > FLINK-25023 [3]: Can leak the classloader of the first job submitted via > rest API. (constant overhead for Metaspace) > > In general, web-submission is different from a normal submission in way, > that the "main method" of the uploaded jar is executed on JobManager and > it's really hard to isolate it's execution from possible side effects. > > Could you by any chance try to submit jobs with the Flink CLI instead? > That should be more robust when it comes to the class loading issues. > > Which endpoint are you using for submitting the job? "/jars/:jarid/run"? > > [1] https://issues.apache.org/jira/browse/FLINK-25022 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-25022&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PQ2lCSaoNSfo8pamCOffd3HisVKZ2WRo8NFKCKptnGo%3D&reserved=0> > [2] https://issues.apache.org/jira/browse/FLINK-25027 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-25027&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vyUyAVKsQHtSYtEChwe%2BhtGrYdH3A5K3iGup1PSfBC0%3D&reserved=0> > [3] https://issues.apache.org/jira/browse/FLINK-25023 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-25023&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9me9rhr6gN%2F1Gv%2BdMHyfbhyYTaRfT1ZHtFJNjavEgQE%3D&reserved=0> > > On Tue, Dec 21, 2021 at 4:49 PM Lior Liviev <[email protected]> > wrote: > > Yes, I use the REST API. I'm running into OOM Metaspace, and I think it's > a class-loading problem, so that's why I'm thinking of putting the jar in > flink/lib > ------------------------------ > *From:* David Morávek <[email protected]> > *Sent:* Tuesday, December 21, 2021 5:43 PM > *To:* Lior Liviev <[email protected]> > *Cc:* [email protected] <[email protected]> > *Subject:* Re: Avoiding Dynamic Classloading for User Code > > > *CAUTION*: external source > Hi Lior, > > can you please provide details about the steps (I'm not sure what load jar > / execute with the API means)? are you submitting the job using the REST > API or Flink CLI? I assume you're using a session cluster. > > also what is the concern here? do you run into any class-loading related > issues? > > D. > > On Tue, Dec 21, 2021 at 3:48 PM Lior Liviev <[email protected]> > wrote: > > Hello, I have existing fixed cluster (*not* a new one with every job > execution) and a single Jar +multiple executions with different params. > > Currently my procedure is: 1. Download Jar 2. Load Jar with API 3. > Execute with API. > I plan to avoid dynamic class loading by applying method described in: > https://nightlies.apache.org/flink/flink-docs-master/docs/ops/debugging/debugging_classloading/#avoiding-dynamic-classloading-for-user-code > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnightlies.apache.org%2Fflink%2Fflink-docs-master%2Fdocs%2Fops%2Fdebugging%2Fdebugging_classloading%2F%23avoiding-dynamic-classloading-for-user-code&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86yWNUSPJQDyRdHzrVEMxnBFGbz3srJMXAPvDG2TPpg%3D&reserved=0> > Debugging Classloading | Apache Flink > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnightlies.apache.org%2Fflink%2Fflink-docs-master%2Fdocs%2Fops%2Fdebugging%2Fdebugging_classloading%2F%23avoiding-dynamic-classloading-for-user-code&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86yWNUSPJQDyRdHzrVEMxnBFGbz3srJMXAPvDG2TPpg%3D&reserved=0> > Debugging Classloading # Overview of Classloading in Flink # When running > Flink applications, the JVM will load various classes over time. These > classes can be divided into three groups based on their origin: The Java > Classpath: This is Java’s common classpath, and it includes the JDK > libraries, and all code in Flink’s /lib folder (the classes of Apache Flink > and some dependencies). > nightlies.apache.org > <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fnightlies.apache.org%2F&data=04%7C01%7CLior.Liviev%40earnix.com%7C0e45bb0786ae4c8de65f08d9c4a0724c%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637757015663353183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XFoF%2F%2BOVYszMRZmXbtVOVgP39I2ga3KyNvoZeAWIjaA%3D&reserved=0> > My question is: > > After putting the Jar in $FLINK/lib, do I need to load Jar and execute it > the old way, or what? > > Do not click on links or open attachments unless you recognize the sender. > Please use the report button if you believe this email is suspicious. > > Do not click on links or open attachments unless you recognize the sender. > Please use the report button if you believe this email is suspicious. > > Do not click on links or open attachments unless you recognize the sender. > Please use the report button if you believe this email is suspicious. >
