Re: Missing module spark-hadoop-cloud in Maven central

Hi, Stephen and Steve. Apache Spark community starts to publish it as a snapshot and Apache Spark 3.2.0 will be the first release has it. - https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-hadoop-cloud_2.12/3.2.0-SNAPSHOT/ Please check the snapshot artifacts and

Re: CVEs

Ok, that sounds like a plan. I will gather what I found and either reach out on the security channel and/or try and upgrade with a pull request. Thanks for pointing me in the right direction. On Mon, Jun 21, 2021 at 4:52 PM Sean Owen wrote: > Yeah if it were clearly exploitable right now we'd

Re: CVEs

Yeah if it were clearly exploitable right now we'd handle it via private@ instead of JIRA; depends on what you think the importance is. If in doubt reply to priv...@spark.apache.org On Mon, Jun 21, 2021 at 6:50 PM Holden Karau wrote: > If you get to a point where you find something you think is

Re: CVEs

You could comment on https://issues.apache.org/jira/browse/SPARK-35550 which covered the updated to Jackson 2.12.3. If there's a decent case for backporting and it doesn't have major compatibility issues, we can do it. Then if you have time, try back-porting the patch to branch-3.1 and run tests.

Re: CVEs

If you get to a point where you find something you think is highly likely a valid vulnerability the best path forward is likely reaching out to private@ to figure out how to do a security release. On Mon, Jun 21, 2021 at 4:42 PM Eric Richardson wrote: > Thanks for the quick reply. Yes, since it

Re: CVEs

Thanks for the quick reply. Yes, since it is included in the jars then it is unclear whether it is used internally at least to me. I can substitute the jar in the distro to avoid the scanner from finding it but then it is unclear whether I could be breaking something or not. Given that 3.1.2 is

Re: CVEs

Whether it matters really depends on whether the CVE affects Spark. Sometimes it clearly could and so we'd try to back-port dependency updates to active branches. Sometimes it clearly doesn't and hey sometimes the dependency is updated anyway for good measure (mostly to keep this off static

CVEs

Hi, I am working with Spark 3.1.2 and getting several vulnerabilities popping up. I am wondering if the Spark distros are scanned etc. and how people resolve these. For example. I am finding - https://nvd.nist.gov/vuln/detail/CVE-2020-25649 This looks like it is fixed in 2.11.0 -

Re: Long schedule delay time of one spark task

Hi, Have you already confirmed SPARK-30458? https://issues.apache.org/jira/browse/SPARK-30458 The problem you met seems related to that issue. Kousuke Hi all, I have a spark streaming job. One of its tasks shows abnormal long running time compared to others. When I check the event timeline

how Spark achieves memory fairness between tasks?

Dear Spark community I was watching this presentation that is about spark memory management. He talks about how they achieve fairness between different tasks in one executor (12:00). And he presents the idea of dynamic

Unsubscribe

-- Thanks & Regards Manikant Goutam Solution and Development 8982243462 [cid:ed15908c-8610-45f4-84e5-949b4053bbb0] About me - Manikant Goutam Disclaimer: This e-mail and any documents, files, or previous e-mail messages appended or attached to it may contain