Re: [DISCUSS] Spark 4.0.0 release

Is there some point of contact that can provide me needed context and >> permissions? >> I'd also love to see why the costs are high and see how we can reduce >> them... >> >> Thanks, >> Nimrod >> >> On Wed, May 8, 2024 at 8:26 AM Holden Karau >

Re: [DISCUSS] Spark 4.0.0 release

> will be automated and the only thing which will be manual is to sign the > release for security reasons that would be reasonable. > > Thanks, > Nimrod > > > בתאריך יום ד׳, 8 במאי 2024, 00:54, מאת Holden Karau ‏< > holden.ka...@gmail.com>: > >> Indeed. We could concei

Re: [DISCUSS] Spark 4.0.0 release

ore, my pgp >> key is lost, etc.). I'll start the RC process at my tomorrow. Thanks for >> your patience! >> >> Wenchen >> >> On Fri, May 3, 2024 at 7:47 AM yangjie01 wrote: >> >>> +1 >>> >>> >>> >>> *发件人**: *Jun

Re: ASF board report draft for May

gt;>>> >>>> In addition, Apache Spark PMC received an official notice from ASF >>>> Infra team. >>>> >>>> https://lists.apache.org/thread/rgy1cg17tkd3yox7qfq87ht12sqclkbg >>>> > [NOTICE] Apache Spark's GitHub Actions us

Re: ASF board report draft for May

possible, we >> opened a blocker-level JIRA issue and have been working on it. >> - https://infra.apache.org/github-actions-policy.html >> >> Please include a sentence that Apache Spark PMC is working on under the >> following umbrella JIRA issue. >> >

Re: ASF board report draft for May

Do we want to include that we’re planning on having a preview release of Spark 4 so folks can see the APIs “soon”? Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams:

[LincolnTalk] Summer Rental in Maine

Please email me directly if you are interested in this rental. Sarah *SOLACE BY THE SEA* SUMMER RENTAL *S. HARPSWELL, MAINE (near Brunswick)* Cottage that sleeps 4 adults and 2 children ( + smaller cottage that sleeps 2). June – 1 week: $1907.50 ( + $763.00 for 2nd cottage) July –

[jira] [Updated] (SPARK-48101) When using INSERT OVERWRITE with Spark CTEs they may not be fully resolved

[ https://issues.apache.org/jira/browse/SPARK-48101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-48101: - Priority: Minor (was: Major) > When using INSERT OVERWRITE with Spark CTEs they

[jira] [Created] (SPARK-48101) When using INSERT OVERWRITE with Spark CTEs they may not be fully resolved

Holden Karau created SPARK-48101: Summary: When using INSERT OVERWRITE with Spark CTEs they may not be fully resolved Key: SPARK-48101 URL: https://issues.apache.org/jira/browse/SPARK-48101 Project

Re: [DISCUSS] Spark 4.0.0 release

+1 :) yay previews On Wed, May 1, 2024 at 5:36 PM Chao Sun wrote: > +1 > > On Wed, May 1, 2024 at 5:23 PM Xiao Li wrote: > >> +1 for next Monday. >> >> We can do more previews when the other features are ready for preview. >> >> Tathagata Das 于2024年5月1日周三 08:46写道： >> >>> Next week sounds

Re: [LincolnTalk] Car mechanic

And another plug for Marconi’s!! Sarah Cannon HoldenOn Apr 29, 2024, at 9:14 AM, Margo Fisher-Martin wrote:I also recommend Marconi’s. Frankie from T is also still there some days.All good!MargoOn Mon, Apr 29, 2024 at 9:06 AM Rob Haslinger wrote:Second plug for

Re: [VOTE] SPARK-46122: Set spark.sql.legacy.createHiveTableByDefault to false

+1 Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Fri, Apr 26, 2024 at 12:06 PM L. C. Hsieh wrote: > +1 > > On Fri, Apr 26, 2024

Re: [FYI] SPARK-47993: Drop Python 3.8

+1 Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Thu, Apr 25, 2024 at 11:18 AM Maciej wrote: > +1 > > Best regards, > Maciej

[Bug 2018504] Re: cups-browsed is using an excessive amount of CPU

+1 also running into this If I restart cups the issue goes away for "awhile" though (interestingly printing does not seem to impact cups meaning it's probably behavior that is unrelated to the printing). -- You received this bug notification because you are a member of Ubuntu Bugs, which is

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

+1 -- even if it's not perfect now is the time to change default values On Sat, Apr 13, 2024 at 4:11 PM Hyukjin Kwon wrote: > +1 > > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: > >> +1. >> >> This feature is very helpful for guarding against correctness issues, >> such as null results due

Re: Introducing Apache Gluten(incubating), a middle layer to offload Spark to native engine

On Wed, Apr 10, 2024 at 9:54 PM Binwei Yang wrote: > > Gluten currently already support Velox backend and Clickhouse backend. > data fusion support is also proposed but no one worked on it. > > Gluten isn't a POC. It's under actively developing but some companies > already used it. > > > On

Re: SPIP: Enhancing the Flexibility of Spark's Physical Plan to Enable Execution on Various Native Engines

I like the idea of improving flexibility of Sparks physical plans and really anything that might reduce code duplication among the ~4 or so different accelerators. Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9

Re: Apache Spark 3.4.3 (?)

Sounds good to me :) Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Sat, Apr 6, 2024 at 2:51 PM Dongjoon Hyun wrote: > Hi, All.

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

+1 Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Mon, Apr 1, 2024 at 5:44 PM Xinrong Meng wrote: > +1 > > Thank you @Hyukjin

[jira] [Created] (SPARK-47672) Avoid double evaluation of non-trivial projected elements from filter pushdown

Holden Karau created SPARK-47672: Summary: Avoid double evaluation of non-trivial projected elements from filter pushdown Key: SPARK-47672 URL: https://issues.apache.org/jira/browse/SPARK-47672

Re: [RBW] Re: How do I know when a saddle fits?

+1 for the Rivet Pearl. I've got one on a Soma Double Cross, and the version with the cutout is indeed an excellent fit for several positions fore and aft! On Sat, Mar 30, 2024, 3:25 PM ascpgh wrote: > Emily, I have dealt with roughly your sort of physiological dimensioning > my whole riding

Re: [RBW] I have questions

Oh, man. Those wheels are gonna look ACE!! I'll chime in anecdotally that the difference between 42-48 isn't super noticeable as long as you get your PSI where it feels comfy for you. Have an amazing time on that 2-day ride. As for front rack security, I don't use a strap, but I also check

[RBW] Re: How do I know when a saddle fits?

Pain is an obvious sign that something is wrong. And I think that's one reason why it's easy to tell when a saddle ISN'T a fit. But what feels right can be so subjective. Obviously Riv and Grant are big proponents of Brooks. I've had several Brooks saddles, and not every one feels the same.

[RBW] Re: anyone else tried Ritchey Beacon Bars?

Dave!! They look amazing on that Jones. I have an SWB that I've wanted to put drop bars on... these might be a candidate. Where do you mount your shifter? On Friday, March 22, 2024 at 12:39:14 PM UTC-7 DavidP wrote: > Great that the Roadini is working out so well for your son (and wife)! > > I

[RBW] Re: Roadini on F1 Track ride report

That sounds like a total blast! I think it speaks well of an event when you can see riders of all styles, abilities, and ages coming together to have a good time. Love that Roadini. The Safety Pizza and snazzy bar tape complement it so well. On Wednesday, March 13, 2024 at 1:37:05 PM UTC-7

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

+1 Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Mon, Mar 11, 2024 at 7:44 PM Reynold Xin wrote: > +1 > > > On Mon, Mar 11 2024

[jira] [Created] (SPARK-47220) log4j race condition during shutdown

Holden Karau created SPARK-47220: Summary: log4j race condition during shutdown Key: SPARK-47220 URL: https://issues.apache.org/jira/browse/SPARK-47220 Project: Spark Issue Type: Improvement

[weewx-user] Re: wxt5x0 v0.7 driver issue

>> No info on whether you have a 'pip' or dpkg or setup installation. >> >> We can't read minds. Need more info to even try to help. >> >> On Tuesday, February 27, 2024 at 9:53:23 AM UTC-8 Stephen Holden wrote: >> >>> I just updated my WXT5x0 driver to t

[weewx-user] wxt5x0 v0.7 driver issue

I just updated my WXT5x0 driver to the latest version as per the README on Matthew's github page, but now the wxt5x0 driver is not listed as an option when I re-configure. Tried installing it again, but no luck. And the weewx.conf is unchanged so points to the (older) wxt5x0 driver, which it

Re: Generating config docs automatically

I think this is a good idea. I like having everything in one source of truth rather than two (so option 1 sounds like a good idea); but that’s just my opinion. I'd be happy to help with reviews though. On Wed, Feb 21, 2024 at 6:37 AM Nicholas Chammas wrote: > I know config documentation is not

Re: Spark 4.0 Query Analyzer Bug Report

Do you mean Spark 3.4? 4.0 is very much not released yet. Also it would help if you could share your query & more of the logs leading up to the error. On Tue, Feb 20, 2024 at 3:07 PM Sharma, Anup wrote: > Hi Spark team, > > > > We ran into a dataframe issue after upgrading from spark 3.1 to 4.

[jira] [Resolved] (SPARK-47077) sbt build is broken due to selenium change

[ https://issues.apache.org/jira/browse/SPARK-47077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-47077. -- Resolution: Cannot Reproduce After blowing away my maven + ivy cache it works fine – should

[jira] [Created] (SPARK-47077) sbt build is broken due to selenium change

Holden Karau created SPARK-47077: Summary: sbt build is broken due to selenium change Key: SPARK-47077 URL: https://issues.apache.org/jira/browse/SPARK-47077 Project: Spark Issue Type

[jira] [Updated] (SPARK-47001) Pushdown Verification in Optimizer.scala should support changed data types

[ https://issues.apache.org/jira/browse/SPARK-47001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-47001: - Description: When pushing a filter down in a union the data type may not match exactly

Re: Dynamically Support Spark Native Engine in Iceberg

This is great work! Very excited to see this. Cell : 425-233-8271 On Tue, Feb 13, 2024 at 4:38 PM huaxin gao wrote: > Hello Iceberg community, > > As you may already know, Project Comet > , a plugin to > accelerate Spark query execution via

Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow

This looks really cool :) Out of interest what are the differences in the approach between this and Glutten? On Tue, Feb 13, 2024 at 12:42 PM Chao Sun wrote: > Hi all, > > We are very happy to announce that Project Comet, a plugin to > accelerate Spark query execution via leveraging DataFusion

Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow

This looks really cool :) Out of interest what are the differences in the approach between this and Glutten? On Tue, Feb 13, 2024 at 12:42 PM Chao Sun wrote: > Hi all, > > We are very happy to announce that Project Comet, a plugin to > accelerate Spark query execution via leveraging DataFusion

[jira] [Created] (SPARK-47031) Union of with non-determinstic expression should be non-deterministic

Holden Karau created SPARK-47031: Summary: Union of with non-determinstic expression should be non-deterministic Key: SPARK-47031 URL: https://issues.apache.org/jira/browse/SPARK-47031 Project: Spark

Question about OpenCL VLOOKUP implementation

I have a question about the OpenCL implementation of VLOOKUP as given in sc/source/core/opencl/op_spreadsheet.cxx. Right now, both the unsorted and sorted vlookup both do a linear search to find the right value. I am wondering if it would be better to do a binary search for sorted vlookup. I could

Re: [Spark-Core] Improving Reliability of spark when Executors OOM

Oh interesting solution, a co-worker was suggesting something similar using resource profiles to increase memory -- but your approach avoids a lot of complexity I like it (and we could extend it out to support resource profile growth too). I think an SPIP sounds like a great next step. On Tue,

Re: [BVARC] Solar effect

e way to go for Ham radio ops during solar storms.Thank you for bringing up this topic!73,Stephen (W2WF)On Jan 5, 2024, at 9:03 AM, David Holden via BVARC <bvarc@bvarc.org> wrote:A strong solar storm can cause a complete blackout of HF communication including the higher frequency bands. I was in a

Re: [BVARC] Solar effect

A strong solar storm can cause a complete blackout of HF communication including the higher frequency bands. I was in a QSO a year or so ago and it just dropped as a solar storm hit. The noise floor dropped to zero as not even noise could propagate through the highly energized atmosphere.

[LincolnTalk] Stucco

Good Morning, Perhaps I do not understand the archives but I don't see how I can search for a STUCCO person who can repair some damaged outside walls on my house. I know there have been suggestions in the past. So... if anyone has a suggestion for a Stucco repair person and/or can tell me how I

(spark) branch master updated: [SPARK-45807][SQL] Add createOrReplaceView(..) / replaceView(..) to ViewCatalog

This is an automated email from the ASF dual-hosted git repository. holden pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 89673da0beb2 [SPARK-45807][SQL] Add

Re: Spark-Connect: Param `--packages` does not take effect for executors.

So I think this sounds like a bug to me, in the help options for both regular spark-submit and ./sbin/start-connect-server.sh we say: " --packages Comma-separated list of maven coordinates of jars to include on the driver and executor classpaths.

Re: [LincolnTalk] Possible to vote early today?

When you check in this am you will be given two ballots. One is for Community Center ~ Article 2. The other is for Housing Choice Act ~ Article 4. Voting will take place for each article after the presentation and discussions of each article. There is no early voting. Sarah Cannon Holden Town

[LincolnTalk] Fwd: Your request for Reasonable Accommodation at Special Town Meeting Re: Voting on Saturday

Jane Marie , bethany.h.br...@mass.govCc: "Pereira, Dan" , "Hutchinson, Jim" , "Cannon Holden, Sarah" , "Pereira, Dan" Subject: RE: Your request for Reasonable Accommodation at Special Town Meeting Re: [LincolnTalk] Voting on Saturday Hello everyone –   We h

Help need to recover my account

Now, my OpenOffice account is gone. How can I restore it? (I got my > computer back from repair and I lost a lot of my programs ... like Open > Office. HELP

Re: Classpath isolation per SparkSession without Spark Connect

So I don’t think we make any particular guarantees around class path isolation there, so even if it does work it’s something you’d need to pay attention to on upgrades. Class path isolation is tricky to get right. On Mon, Nov 27, 2023 at 2:58 PM Faiz Halde wrote: > Hello, > > We are using spark

Re: [VOTE] SPIP: An Official Kubernetes Operator for Apache Spark

+1 On Tue, Nov 14, 2023 at 10:21 AM DB Tsai wrote: > +1 > > DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > > On Nov 14, 2023, at 10:14 AM, Vakaris Baškirov < > vakaris.bashki...@gmail.com> wrote: > > +1 (non-binding) > > > On Tue, Nov 14, 2023 at 8:03 PM Chao Sun wrote: > >> +1

[Python-Dev] Re: Retiring this mailing list ?

On Mon, 13 Nov 2023 at 10:18, Marc-Andre Lemburg wrote: > [...] > > Question: Should we retire and archive this mailing list ? > (I'm asking as one of the maintainers of the ML) > > [...] Hi Marc-Andre, Maybe just require senders to be members of the python.org domain, and retain the release

Re: [DISCUSSION] SPIP: An Official Kubernetes Operator for Apache Spark

To be clear: I am generally supportive of the idea (+1) but have some follow-up questions: Have we taken the time to learn from the other operators? Do we have a compatible CRD/API or not (and if so why?) The API seems to assume that everything is packaged in the container in advance, but I

Re: [LincolnTalk] Transfer Station

Thank you everyone! It's open. On Sat, Nov 11, 2023 at 10:05 AM sarah cannon holden < sarahcannonhol...@gmail.com> wrote: > Is it open today? > -- The LincolnTalk mailing list. To post, send mail to Lincoln@lincolntalk.org. Browse the archives at https://pairlist9.pair.net/mai

[LincolnTalk] Transfer Station

Is it open today? -- The LincolnTalk mailing list. To post, send mail to Lincoln@lincolntalk.org. Browse the archives at https://pairlist9.pair.net/mailman/private/lincoln/. Change your subscription settings at https://pairlist9.pair.net/mailman/listinfo/lincoln.

Re: Apache Spark 3.4.2 (?)

+1 On Mon, Nov 6, 2023 at 4:30 PM yangjie01 wrote: > +1 > > > > *发件人**: *Yuming Wang > *日期**: *2023年11月7日 星期二 07:00 > *收件人**: *Santosh Pingale > *抄送**: *Dongjoon Hyun , dev > > *主题**: *Re: Apache Spark 3.4.2 (?) > > > > +1 > > > > On Tue, Nov 7, 2023 at 3:55 AM Santosh Pingale > wrote: > >

[LincolnTalk] Photo Printers

Does anyone have experience with either an Epson700 Sure Color or a Canon Pro 300 printer? Or if you want to suggest other printers, please do. I want to print 9x12 photo prints as well as note cards - that's my starting point. Thank you, Sarah -- The LincolnTalk mailing list. To post, send

[jira] [Created] (SPARK-45712) Provide a command line flag to override the log4j properties file

Holden Karau created SPARK-45712: Summary: Provide a command line flag to override the log4j properties file Key: SPARK-45712 URL: https://issues.apache.org/jira/browse/SPARK-45712 Project: Spark

[spark] branch master updated: [SPARK-44735][SQL] Add warning msg when inserting columns with the same name by row that don't match up

This is an automated email from the ASF dual-hosted git repository. holden pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ae100267c28b [SPARK-44735][SQL] Add warning msg

[LincolnTalk] Roof Repair

Does anyone have a suggestion for roof repairs? We have a couple of missing shingles and a leak. Sarah -- The LincolnTalk mailing list. To post, send mail to Lincoln@lincolntalk.org. Browse the archives at https://pairlist9.pair.net/mailman/private/lincoln/. Change your subscription settings at

[jira] [Updated] (SPARK-45563) Spark history files backend currently depend on polling for loading into the history server

[ https://issues.apache.org/jira/browse/SPARK-45563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-45563: - Affects Version/s: 4.0.0 (was: 3.3.0

[jira] [Updated] (SPARK-45563) Spark history files backend currently depend on polling for loading into the history server

[ https://issues.apache.org/jira/browse/SPARK-45563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-45563: - Issue Type: Improvement (was: Bug) > Spark history files backend currently depend on poll

[jira] [Updated] (SPARK-45563) Spark history files backend currently depend on polling for loading into the history server

[ https://issues.apache.org/jira/browse/SPARK-45563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-45563: - Description: The spark history server FS  currently depends on polling for loading history

[jira] [Updated] (SPARK-45563) Spark history files backend currently depend on polling for loading into the history server

[ https://issues.apache.org/jira/browse/SPARK-45563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-45563: - Summary: Spark history files backend currently depend on polling for loading into the history

[jira] [Created] (SPARK-45563) Spark rolling history files currently depend on polling for loading into the history server

Holden Karau created SPARK-45563: Summary: Spark rolling history files currently depend on polling for loading into the history server Key: SPARK-45563 URL: https://issues.apache.org/jira/browse/SPARK-45563

[jira] [Resolved] (SPARK-44735) Log a warning when inserting columns with the same name by row that don't match up

[ https://issues.apache.org/jira/browse/SPARK-44735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-44735. -- Fix Version/s: 4.0.0 Resolution: Fixed > Log a warning when inserting colu

[jira] [Assigned] (SPARK-44735) Log a warning when inserting columns with the same name by row that don't match up

[ https://issues.apache.org/jira/browse/SPARK-44735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reassigned SPARK-44735: Assignee: Jia Fan > Log a warning when inserting columns with the same name by

[Python-Dev] Re: detecting statements which result is not stored

Sounds like you might want the "Python Help" group on https;// discuss.python.org - the dev conversation migrated there quite a while ago now, so thighs channel is more or less announcements only. Good luck with your project! Kind regards, Steve On Thu, 5 Oct 2023 at 16:07, Guenther Sohler

Bug#1052991: apt: Missing Release File

Package: apt Version: 2.2.4 Severity: important X-Debbugs-Cc: stu...@gmail.com Dear Maintainer, *** Reporter, please consider answering these questions, where appropriate *** My release Debian 11.7 sudo apt update - gives The repository 'http://archive.raspbian.org/raspbian stretch

[LincolnTalk] Power at 1:59 am

Sarah Cannon Holden -- The LincolnTalk mailing list. To post, send mail to Lincoln@lincolntalk.org. Browse the archives at https://pairlist9.pair.net/mailman/private/lincoln/. Change your subscription settings at https://pairlist9.pair.net/mailman/listinfo/lincoln.

[LincolnTalk] Power outage

I’m on Weston Rd. without power. What about others? Sarah Cannon Holden -- The LincolnTalk mailing list. To post, send mail to Lincoln@lincolntalk.org. Browse the archives at https://pairlist9.pair.net/mailman/private/lincoln/. Change your subscription settings at https://pairlist9.pair.net

Re: Write Spark Connection client application in Go

That’s so cool! Great work y’all :) On Tue, Sep 12, 2023 at 8:14 PM bo yang wrote: > Hi Spark Friends, > > Anyone interested in using Golang to write Spark application? We created a > Spark > Connect Go Client library . > Would love to hear

Re: Write Spark Connection client application in Go

That’s so cool! Great work y’all :) On Tue, Sep 12, 2023 at 8:14 PM bo yang wrote: > Hi Spark Friends, > > Anyone interested in using Golang to write Spark application? We created a > Spark > Connect Go Client library . > Would love to hear

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

+1 pip installing seems to function :) On Thu, Sep 7, 2023 at 7:22 PM Yuming Wang wrote: > +1. > > On Thu, Sep 7, 2023 at 10:33 PM yangjie01 > wrote: > >> +1 >> >> >> >> *发件人**: *Gengliang Wang >> *日期**: *2023年9月7日 星期四 12:53 >> *收件人**: *Yuanjian Li >> *抄送**: *Xiao Li ,

Re: [VOTE] Release Apache Spark 3.5.0 (RC3)

Can we delay the next RC cut until after Labor Day? On Sat, Sep 2, 2023 at 9:59 PM Yuanjian Li wrote: > Thank you for all the reports! > The vote has failed. I plan to cut RC4 in two days. > > @Dipayan Dev I quickly skimmed through the > corresponding ticket, and it doesn't seem to be a

[Spectacle] [Bug 473931] New: Consistent Spectacle Crashes after disconnecting secondary monitor

https://bugs.kde.org/show_bug.cgi?id=473931 Bug ID: 473931 Summary: Consistent Spectacle Crashes after disconnecting secondary monitor Classification: Applications Product: Spectacle Version: 23.04.3 Platform: openSUSE

[jira] [Created] (SPARK-44992) Add support for rack information from an environment variable

Holden Karau created SPARK-44992: Summary: Add support for rack information from an environment variable Key: SPARK-44992 URL: https://issues.apache.org/jira/browse/SPARK-44992 Project: Spark

Re: Elasticsearch support for Spark 3.x

What’s the version of the ES connector you are using? On Sat, Aug 26, 2023 at 10:17 AM Dipayan Dev wrote: > Hi All, > > We're using Spark 2.4.x to write dataframe into the Elasticsearch index. > As we're upgrading to Spark 3.3.0, it throwing out error > Caused by:

[jira] [Created] (SPARK-44970) Spark History File Uploads Can Fail on S3

Holden Karau created SPARK-44970: Summary: Spark History File Uploads Can Fail on S3 Key: SPARK-44970 URL: https://issues.apache.org/jira/browse/SPARK-44970 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-44955) Add the option for dynamically marking containers for preemption based data

Holden Karau created SPARK-44955: Summary: Add the option for dynamically marking containers for preemption based data Key: SPARK-44955 URL: https://issues.apache.org/jira/browse/SPARK-44955 Project

[jira] [Created] (SPARK-44954) Make DEA algorithms pluggable

Holden Karau created SPARK-44954: Summary: Make DEA algorithms pluggable Key: SPARK-44954 URL: https://issues.apache.org/jira/browse/SPARK-44954 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44953) Log a warning (or automatically disable) when shuffle tracking is enabled along side another DA supported mechanism

[ https://issues.apache.org/jira/browse/SPARK-44953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-44953: - Parent: SPARK-44951 Issue Type: Sub-task (was: Improvement) > Log a warn

[jira] [Created] (SPARK-44953) Log a warning (or automatically disable) when shuffle tracking is enabled along side another DA supported mechanism

Holden Karau created SPARK-44953: Summary: Log a warning (or automatically disable) when shuffle tracking is enabled along side another DA supported mechanism Key: SPARK-44953 URL: https://issues.apache.org/jira

[jira] [Created] (SPARK-44951) Improve Spark Dynamic Allocation

Holden Karau created SPARK-44951: Summary: Improve Spark Dynamic Allocation Key: SPARK-44951 URL: https://issues.apache.org/jira/browse/SPARK-44951 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-44950) Improve Spark Driver Launch Time

Holden Karau created SPARK-44950: Summary: Improve Spark Driver Launch Time Key: SPARK-44950 URL: https://issues.apache.org/jira/browse/SPARK-44950 Project: Spark Issue Type: Improvement

Re: [Internet]Re: Improving Dynamic Allocation Logic for Spark 4+

d >>> London >>> United Kingdom >>> >>> >>>view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>>

[jira] [Created] (SPARK-44769) Add SQL statement to create an empty array with a type

Holden Karau created SPARK-44769: Summary: Add SQL statement to create an empty array with a type Key: SPARK-44769 URL: https://issues.apache.org/jira/browse/SPARK-44769 Project: Spark Issue

[jira] [Updated] (SPARK-42035) Add a config flag to force exit on JDK major version mismatch

[ https://issues.apache.org/jira/browse/SPARK-42035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-42035: - Target Version/s: 4.0.0 > Add a config flag to force exit on JDK major version misma

[jira] [Updated] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-42261: - Target Version/s: 4.0.0 > K8s will not allocate more execs if there are any pending execs un

[jira] [Updated] (SPARK-44511) Allow insertInto to succeed with partion columns specified when they match those on the target table

[ https://issues.apache.org/jira/browse/SPARK-44511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-44511: - Target Version/s: 4.0.0 > Allow insertInto to succeed with partion columns specified w

[jira] [Updated] (SPARK-42361) Add an option to use external storage to distribute JAR set in cluster mode on Kube

[ https://issues.apache.org/jira/browse/SPARK-42361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-42361: - Target Version/s: 4.0.0 > Add an option to use external storage to distribute JAR

[jira] [Updated] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

[ https://issues.apache.org/jira/browse/SPARK-42260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-42260: - Target Version/s: 4.0.0 > Log when the K8s Exec Pods Allocator Sta

[jira] [Commented] (SPARK-44727) Improve the error message for dynamic allocation conditions

[ https://issues.apache.org/jira/browse/SPARK-44727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752496#comment-17752496 ] Holden Karau commented on SPARK-44727: -- Do you have more context [~chengpan] ? > Improve the er

[jira] [Updated] (SPARK-42035) Add a config flag to force exit on JDK major version mismatch

[ https://issues.apache.org/jira/browse/SPARK-42035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-42035: - Description: JRE version mismatches can cause errors which are difficult to debug (potentially

[jira] [Updated] (SPARK-34337) Reject disk blocks when out of disk space

[ https://issues.apache.org/jira/browse/SPARK-34337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-34337: - Target Version/s: 4.0.0 > Reject disk blocks when out of disk sp

[jira] [Created] (SPARK-44735) Log a warning when inserting columns with the same name by row that don't match up

Holden Karau created SPARK-44735: Summary: Log a warning when inserting columns with the same name by row that don't match up Key: SPARK-44735 URL: https://issues.apache.org/jira/browse/SPARK-44735

Re: [Internet]Re: Improving Dynamic Allocation Logic for Spark 4+

ical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Tue, 8 Aug 2023 at 15:49, kalyan wrote: > >> +1 to enhancements in DEA. Long time due! >> >> There

Re: ASF board report draft for August 2023

Maybe add a link to the 4.0 JIRA where we are tracking the current plans for 4.0? On Tue, Aug 8, 2023 at 9:33 AM Dongjoon Hyun wrote: > Thank you, Matei. > > It looks good to me. > > Dongjoon > > On Mon, Aug 7, 2023 at 22:54 Matei Zaharia > wrote: > >> It’s time to send our quarterly report to

Re: Dynamic allocation does not deallocate executors

ny > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage

Re: Dynamic resource allocation for structured streaming [SPARK-24815]

Oooh fascinating. I’m going on call this week so it will take me awhile but I do want to review this :) On Mon, Aug 7, 2023 at 5:30 PM Pavan Kotikalapudi wrote: > Hi Spark Dev, > > I have extended traditional DRA to work for structured streaming > use-case. > > Here is an initial Implementation

Re: Improving Dynamic Allocation Logic for Spark 4+

Oh great point On Mon, Aug 7, 2023 at 2:23 PM bo yang wrote: > Thanks Holden for bringing this up! > > Maybe another thing to think about is how to make dynamic allocation more > friendly with Kubernetes and disaggregated shuffle storage? > > > > On Mon, Aug 7, 2023

  1   2   3   4   5   6   7   8   9   10   >