Re: [FYI] SPARK-47993: Drop Python 3.8

+1 (non-binding) On Thu, Apr 25, 2024 at 19:26 Xinrong Meng wrote: > +1 > > On Thu, Apr 25, 2024 at 2:08 PM Holden Karau > wrote: > >> +1 >> >> Twitter: https://twitter.com/holdenkarau >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

+1 (non-binding) On Sat, Apr 13, 2024 at 7:49 PM huaxin gao wrote: > +1 > > On Sat, Apr 13, 2024 at 4:36 PM L. C. Hsieh wrote: > >> +1 >> >> On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon >> wrote: >> > >> > +1 >> > >> > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: >> >> >> >> +1. >> >> >>

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

+1 (non-binding) On Mon, Apr 1, 2024 at 9:24 AM Hussein Awala wrote: > +1(non-binding) I add to the difference will it make that it will also > simplify package maintenance and easily release a bug fix/new feature > without needing to wait for Pyspark to release. > > On Mon, Apr 1, 2024 at

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

+1 (non-binding) On Sun, Mar 10, 2024 at 23:36 Gengliang Wang wrote: > Hi all, > > I'd like to start the vote for SPIP: Structured Logging Framework for > Apache Spark > > References: > >- JIRA ticket >- SPIP doc > >

Re: [VOTE] Move Arrow DataFusion Subproject to new Top Level Apache Project

+1 (non-binding) On Fri, Mar 1, 2024 at 18:58 kazuyuki tanimura wrote: > +1 (non-binding) > > Kazu > > > On Mar 1, 2024, at 5:44 PM, L. C. Hsieh wrote: > > > > +1 (binding) > > > > On Fri, Mar 1, 2024 at 1:25 PM Joris Van den Bossche > > wrote: > >> > >> +1 (binding) > >> > >> On Fri, 1 Mar

Re: [VOTE] Accept donation of Comet Spark native engine

+1 (non-binding) RAD stack for fast queries, what's not to love! On Sat, Jan 27, 2024 at 9:24 PM Jacob Wujciak-Jens wrote: > +1 (non-binding) > > Jorge Cardoso Leitão schrieb am So., 28. Jan. > 2024, 05:17: > > > +1 > > > > On Sun, 28 Jan 2024, 00:00 Wes McKinney, wrote: > > > > > +1

Re: [ANNOUNCE] New Arrow PMC chair: Andy Grove

Congratulations Andy and thanks Andrew! On Mon, Nov 27, 2023 at 5:45 PM Ian Joiner wrote: > Congrats Andy and thanks Andrew! > > > On Monday, November 27, 2023, Yibo Cai wrote: > > > Congrats Andy! > > > > On 11/28/23 06:03, L. C. Hsieh wrote: > > > >> Congrats Andy! > >> > >> Thanks Andrew

Re: [VOTE] Updating documentation hosted for EOL and maintenance releases

+1 On Tue, Sep 26, 2023 at 10:52 Maciej wrote: > +1 > > Best regards, > Maciej Szymkiewicz > > Web: https://zero323.net > PGP: A30CEF0C31A501EC > > On 9/26/23 17:12, Michel Miotto Barbosa wrote: > > +1 > > A disposição | At your disposal > > Michel Miotto Barbosa >

Re: First Time contribution.

Hi Ram, We have some good guidance at https://spark.apache.org/contributing.html HTH! Denny On Sun, Sep 17, 2023 at 17:18 ram manickam wrote: > > > > Hello All, > Recently, joined this community and would like to contribute. Is there a > guideline or recommendation on tasks that can be

Re: First Time contribution.

Hi Ram, We have some good guidance at https://spark.apache.org/contributing.html HTH! Denny On Sun, Sep 17, 2023 at 17:18 ram manickam wrote: > > > > Hello All, > Recently, joined this community and would like to contribute. Is there a > guideline or recommendation on tasks that can be

Re: [VOTE][SPIP] Python Data Source API

+1 (non-binding) On Fri, Jul 7, 2023 at 00:50 Maciej wrote: > +0 > > Best regards, > Maciej Szymkiewicz > > Web: https://zero323.net > PGP: A30CEF0C31A501EC > > On 7/6/23 17:41, Xiao Li wrote: > > +1 > > Xiao > > Hyukjin Kwon 于2023年7月5日周三 17:28写道： > >> +1. >> >> See

Re: [DISCUSS] SPIP: Python Data Source API

Slightly biased, but per my conversations - this would be awesome to have! On Mon, Jun 19, 2023 at 09:43 Abdeali Kothari wrote: > I would definitely use it - is it's available :) > > On Mon, 19 Jun 2023, 21:56 Jacek Laskowski, wrote: > >> Hi Allison and devs, >> >> Although I was against this

Re: JDK version support policy?

+1 on dropping Java 8 in Spark 4.0, saying this as a fan of the fast-paced (positive) updates to Arrow, eh?! On Tue, Jun 6, 2023 at 4:02 PM Sean Owen wrote: > I haven't followed this discussion closely, but I think we could/should > drop Java 8 in Spark 4.0, which is up next after 3.5? > > On

Re: [CONNECT] New Clients for Go and Rust

+1 on separate repo allowing different APIs to run at different speeds and ensuring they get community support. On Wed, May 24, 2023 at 00:37 Hyukjin Kwon wrote: > I think we can just start this with a separate repo. > I am fine with the second option too but in this case we would have to >

Re: Slack for Spark Community: Merging various threads

important? > 2. I'm wondering what ASF misses here if Apache Spark PMC invites all > remaining subscribers of `user@spark` and `dev@spark` mailing lists. > > Thanks, > Dongjoon. > > [image: invitation.png] > [image: invited.png] > > On Wed, Apr 5, 2023 at 7:23 PM Denny L

Slack for Spark Community: Merging various threads

There have been a number of threads discussing creating a Slack for the Spark community that I'd like to try to help reconcile. Topic: Slack for Spark Why: Allows for the community to have the option to communicate with each other using Slack; a pretty popular async communication. Discussion

Re: Slack for PySpark users

and affordable. Alternatives have been >>>>>>> suggested as well so those who like investigative search can agree and >>>>>>> come >>>>>>> up with a freebie one. >>>>>>> I am inclined to agree with Bjorn that th

Re: Slack for PySpark users

and affordable. Alternatives have been >>>>>>> suggested as well so those who like investigative search can agree and >>>>>>> come >>>>>>> up with a freebie one. >>>>>>> I am inclined to agree with Bjorn that th

Re: Slack for PySpark users

;>>>>>> +1 >>>>>>> >>>>>>> + @d...@spark.apache.org >>>>>>> >>>>>>> This is a good idea. The other Apache projects (e.g., Pinot, Druid, >>>>>>> Flink) have created their

Re: Slack for PySpark users

t;>>>>>> +1 >>>>>>> >>>>>>> + @dev@spark.apache.org >>>>>>> >>>>>>> This is a good idea. The other Apache projects (e.g., Pinot, Druid, >>>>>>> Flink) have created their

Re: Slack for PySpark users

+1 I think this is a great idea! On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon wrote: > Yeah, actually I think we should better have a slack channel so we can > easily discuss with users and developers. > > On Tue, 28 Mar 2023 at 03:08, keen wrote: > >> Hi all, >> I really like *Slack *as

Re: Topics for Spark online classes & webinars

. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> >> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali >> wrote: >> >> Hello Mich, >> >> My apologies ... but I am

Re: Topics for Spark online classes & webinars

. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> >> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali >> wrote: >> >> Hello Mich, >> >> My apologies ... but I am

Re: Topics for Spark online classes & webinars

Thanks Mich for tackling this! I encourage everyone to add to the list so we can have a comprehensive list of topics, eh?! On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh wrote: > Hi all, > > Thanks to @Denny Lee to give access to > > https://www.linkedin.com/comp

Re: Topics for Spark online classes & webinars

Thanks Mich for tackling this! I encourage everyone to add to the list so we can have a comprehensive list of topics, eh?! On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh wrote: > Hi all, > > Thanks to @Denny Lee to give access to > > https://www.linkedin.com/comp

Re: Topics for Spark online classes & webinars

equest to leverage the original Spark confluence page <https://cwiki.apache.org/confluence/display/SPARK>.WDYT? On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh wrote: > Well that needs to be created first for this purpose. The appropriate name > etc. to be decided. Maybe @Denny Lee

Re: Topics for Spark online classes & webinars

equest to leverage the original Spark confluence page <https://cwiki.apache.org/confluence/display/SPARK>.WDYT? On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh wrote: > Well that needs to be created first for this purpose. The appropriate name > etc. to be decided. Maybe @Denny Lee

Re: Online classes for spark topics

Looks like we have some good topics here - I'm glad to help with setting up the infrastructure to broadcast if it helps? On Thu, Mar 9, 2023 at 6:19 AM neeraj bhadani wrote: > I am happy to be a part of this discussion as well. > > Regards, > Neeraj > > On Wed, 8 Mar 2023 at 22:41, Winston Lai

Re: Online classes for spark topics

We used to run Spark webinars on the Apache Spark LinkedIn group but honestly the turnout was pretty low. We had dove into various features. If there are particular topics that. you would like to discuss during a live session,

Re: Spark on Kube (virtua) coffee/tea/pop times

Woohoo, Holden! I’m in (hopefully my schedule allows me) - PST. On Tue, Feb 7, 2023 at 15:34 Andrew Melo wrote: > I'm Central US time (AKA UTC -6:00) > > On Tue, Feb 7, 2023 at 5:32 PM Holden Karau wrote: > > > > Awesome, I guess I should have asked folks for timezones that they’re in. > > >

Re: Prometheus with spark

Hi Raja, A little atypical way to respond to your question - please check out the most recent Spark AMA where we discuss this: https://www.linkedin.com/posts/apachespark_apachespark-ama-committers-activity-6989052811397279744-jpWH?utm_source=share_medium=member_ios HTH! Denny On Tue, Oct 25,

Re: [VOTE] SPIP: Support Docker Official Image for Spark

+1 (non-binding) On Wed, Sep 21, 2022 at 10:33 PM Ankit Gupta wrote: > +1 > > Regards, > > Ankit Prakash Gupta > > On Thu, Sep 22, 2022 at 10:38 AM Yang,Jie(INF) > wrote: > >> +1 (non-binding) >> >> >> >> Regards, >> >> Yang Jie >> >> >> >> *发件人**: *Gengliang Wang >> *日期**: *2022年9月22日 星期四

Re: [DISCUSS] SPIP: Support Docker Official Image for Spark

+1 (non-binding). This is a great idea and we should definitely do this. Count me in to help as well, eh?! :) On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo wrote: > +1 (non-binding) > > This will bring the good experience to customers. So excited about this. > ;-) > > Yuming Wang 于2022年9月19日周一

Re: Databricks notebook - cluster taking a long time to get created, often timing out

Hi Karan, You may want to ping Databricks Help or Forums as this is a Databricks specific question. I'm a little surprised that a Databricks cluster would take a long time to create so it may be best to utilize these

Re: Append to an existing Delta Lake using structured streaming

Including the Delta Lake Users and Developers DL to help out. Saying this, could you clarify how data is not being added? By any chance do you have any code samples to recreate this? Sent via Superhuman On Wed, Jul 21, 2021 at 2:49 AM, wrote: >

Re: [VOTE] SPIP: Support pandas API layer on PySpark

+1 (non-binding) On Sun, Mar 28, 2021 at 9:06 PM 郑瑞峰 wrote: > +1 (non-binding) > > > -- 原始邮件 -- > *发件人:* "Maxim Gekk" ; > *发送时间:* 2021年3月29日(星期一) 凌晨2:08 > *收件人:* "Matei Zaharia"; > *抄送:* "Gengliang Wang";"Mridul Muralidharan"< > mri...@gmail.com>;"Xiao Li";"Spark

Re: I'm going to be out starting Nov 5th

Best wishes Holden! :) On Sat, Oct 31, 2020 at 11:00 Dongjoon Hyun wrote: > Take care, Holden! I believe everything goes well. > > Bests, > Dongjoon. > > On Sat, Oct 31, 2020 at 10:24 AM Reynold Xin wrote: > >> Take care Holden and best of luck with everything! >> >> >> On Sat, Oct 31 2020 at

Re: [vote] Apache Spark 3.0 RC3

+1 (non-binding) On Sun, Jun 7, 2020 at 3:21 PM Jungtaek Lim wrote: > I'm seeing the effort of including the correctness issue SPARK-28067 [1] > to 3.0.0 via SPARK-31894 [2]. That doesn't seem to be a regression so > technically doesn't block the release, so while it'd be good to weigh its >

Re: How to unsubscribe

Hi Fred, To unsubscribe, could you please email: user-unsubscr...@spark.apache.org (for more information, please refer to https://spark.apache.org/community.html). Thanks! Denny On Wed, May 6, 2020 at 10:12 AM Fred Liu wrote: > Hi guys > > > >

Re: can we all help use our expertise to create an IT solution for Covid-19

There are a number of really good datasets already available including (but not limited to): - South Korea COVID-19 Dataset - 2019 Novel Coronavirus COVID-19 (2019-nCoV) Data Repository by Johns Hopkins CSSE

Re: [VOTE] Amend Spark's Semantic Versioning Policy

+1 (non-binding) On Mon, Mar 9, 2020 at 1:59 AM Hyukjin Kwon wrote: > The proposal itself seems good as the factors to consider, Thanks Michael. > > Several concerns mentioned look good points, in particular: > > > ... assuming that this is for public stable APIs, not APIs that are > marked as

Re: Block a user from spark-website who repeatedly open the invalid same PR

+1 On Sun, Jan 26, 2020 at 09:59 Nicholas Chammas wrote: > +1 > > I think y'all have shown this person more patience than is merited by > their behavior. > > On Sun, Jan 26, 2020 at 5:16 AM Takeshi Yamamuro > wrote: > >> +1 >> >> Bests, >> Takeshi >> >> On Sun, Jan 26, 2020 at 3:05 PM Hyukjin

Re: Should python-2 be supported in Spark 3.0?

+1 On Fri, May 31, 2019 at 17:58 Holden Karau wrote: > +1 > > On Fri, May 31, 2019 at 5:41 PM Bryan Cutler wrote: > >> +1 and the draft sounds good >> >> On Thu, May 30, 2019, 11:32 AM Xiangrui Meng wrote: >> >>> Here is the draft announcement: >>> >>> === >>> Plan for dropping Python 2

Re: Should python-2 be supported in Spark 3.0?

+1 On Fri, May 31, 2019 at 17:58 Holden Karau wrote: > +1 > > On Fri, May 31, 2019 at 5:41 PM Bryan Cutler wrote: > >> +1 and the draft sounds good >> >> On Thu, May 30, 2019, 11:32 AM Xiangrui Meng wrote: >> >>> Here is the draft announcement: >>> >>> === >>> Plan for dropping Python 2

Re: [VOTE] [SPARK-25994] SPIP: DataFrame-based Property Graphs, Cypher Queries, and Algorithms

+1 yay - let's do it! On Tue, Jan 29, 2019 at 6:28 AM Xiangrui Meng wrote: > Hi all, > > I want to call for a vote of SPARK-25994 > . It introduces a new > DataFrame-based component to Spark, which supports property graph > construction,

Re: [VOTE] SPARK 2.2.3 (RC1)

+1 On Wed, Jan 9, 2019 at 4:30 AM Dongjoon Hyun wrote: > +1 > > Bests, > Dongjoon. > > On Tue, Jan 8, 2019 at 6:30 PM Wenchen Fan wrote: > >> +1 >> >> On Wed, Jan 9, 2019 at 3:37 AM DB Tsai wrote: >> >>> +1 >>> >>> Sincerely, >>> >>> DB Tsai >>>

Re: [VOTE] SPARK 2.4.0 (RC5)

+1 On Wed, Oct 31, 2018 at 12:54 PM Chitral Verma wrote: > +1 > > On Wed, 31 Oct 2018 at 11:56, Reynold Xin wrote: > >> +1 >> >> Look forward to the release! >> >> >> >> On Mon, Oct 29, 2018 at 3:22 AM Wenchen Fan wrote: >> >>> Please vote on releasing the following candidate as Apache Spark

Re: welcome a new batch of committers

Congratulations! On Wed, Oct 3, 2018 at 05:26 Dongjin Lee wrote: > Congratulations to ALL!! > > - Dongjin > > On Wed, Oct 3, 2018 at 7:48 PM Jack Kolokasis > wrote: > >> Congratulations to all !! >> >> -Iacovos >> >> On 03/10/2018 12:54 μμ, Ted Yu wrote: >> >> Congratulations to all ! >> >>

Re: [VOTE] SPARK 2.4.0 (RC2)

+1 (non-binding) On Sat, Sep 29, 2018 at 10:24 AM Stavros Kontopoulos < stavros.kontopou...@lightbend.com> wrote: > +1 > > Stavros > > On Sat, Sep 29, 2018 at 5:59 AM, Sean Owen wrote: > >> +1, with comments: >> >> There are 5 critical issues for 2.4, and no blockers: >> SPARK-25378

Re: [VOTE] SPARK 2.3.2 (RC6)

+1 On Thu, Sep 20, 2018 at 9:55 AM Xiao Li wrote: > +1 > > > John Zhuge 于2018年9月19日周三 下午1:17写道： > >> +1 (non-binding) >> >> Built on Ubuntu 16.04 with Maven flags: -Phadoop-2.7 -Pmesos -Pyarn >> -Phive-thriftserver -Psparkr -Pkinesis-asl -Phadoop-provided >> >> java version "1.8.0_181" >>

Re: [VOTE] SPARK 2.3.2 (RC3)

+1 (non-binding) On Tue, Jul 17, 2018 at 23:04 John Zhuge wrote: > +1 (non-binding) > > On Mon, Jul 16, 2018 at 8:04 PM Saisai Shao > wrote: > >> I will put my +1 on this RC. >> >> For the test failure fix, I will include it if there's another RC. >> >> Sean Owen 于2018年7月16日周一 下午10:47写道： >> >

Re: [VOTE] Spark 2.3.1 (RC4)

+1 On Sat, Jun 2, 2018 at 4:53 PM Nicholas Chammas wrote: > I'll give that a try, but I'll still have to figure out what to do if none > of the release builds work with hadoop-aws, since Flintrock deploys Spark > release builds to set up a cluster. Building Spark is slow, so we only do > it if

Re: Revisiting Online serving of Spark models?

I most likely will not be able to join SF next week but definitely up for a session after Summit in Seattle to dive further into this, eh?! On Wed, May 30, 2018 at 9:32 AM Felix Cheung wrote: > Hi! > > Thank you! Let’s meet then > > June 6 4pm > > Moscone West Convention Center > 800 Howard

Re: Welcome Zhenhua Wang as a Spark committer

Awesome - congrats Zhenhua! On Sun, Apr 1, 2018 at 10:33 PM 叶先进 wrote: > Big congs. > > > On Apr 2, 2018, at 1:28 PM, Wenchen Fan wrote: > > > > Hi all, > > > > The Spark PMC recently added Zhenhua Wang as a committer on the project. > Zhenhua is the

Re: [VOTE] Spark 2.3.0 (RC5)

+1 (non-binding) On Fri, Feb 23, 2018 at 07:08 Josh Goldsborough < joshgoldsboroughs...@gmail.com> wrote: > New to testing out Spark RCs for the community but I was able to run some > of the basic unit tests without error so for what it's worth, I'm a +1. > > On Thu, Feb 22, 2018 at 4:23 PM,

Re: Does Pyspark Support Graphx?

com> wrote: > Hi Denny, > The pyspark script uses the --packages option to load graphframe library, > what about the SparkLauncher class? > > > > ------ Original -- > *From:* Denny Lee <denny.g@gmail.com> > *Date:* Sun,Feb 18,2018 1

Re: [VOTE] Spark 2.3.0 (RC4)

+1 (non-binding) Built and tested on macOS and Ubuntu. On Sun, Feb 18, 2018 at 3:19 PM Ricardo Almeida < ricardo.alme...@actnowib.com> wrote: > +1 (non-binding) > > Built and tested on macOS 10.12.6 Java 8 (build 1.8.0_111). No > regressions detected so far. > > > On 18 February 2018 at 16:12,

Re: Does Pyspark Support Graphx?

. GraphFrames? On Sat, Feb 17, 2018 at 8:26 PM xiaobo <guxiaobo1...@qq.com> wrote: > Thanks Denny, will it be supported in the near future? > > > > -- Original ------ > *From:* Denny Lee <denny.g@gmail.com> > *Date:* Sun,Feb

Re: Does Pyspark Support Graphx?

That’s correct - you can use GraphFrames though as it does support PySpark. On Sat, Feb 17, 2018 at 17:36 94035420 wrote: > I can not find anything for graphx module in the python API document, does > it mean it is not supported yet? >

[jira] [Updated] (SPARK-21866) SPIP: Image support in Spark

[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denny Lee updated SPARK-21866: -- Description: h2. Background and motivation As Apache Spark is being used more and more

Re: [VOTE] Spark 2.1.2 (RC4)

+1 (non-binding) On Wed, Oct 4, 2017 at 11:08 PM Holden Karau wrote: > Awesome, thanks for digging into the packaging on the R side in more > detail. I'll look into how to update the keys file as well. > > On Wed, Oct 4, 2017 at 10:46 PM Felix Cheung

Re: [VOTE] Spark 2.1.2 (RC2)

+1 (non-binding) On Wed, Sep 27, 2017 at 6:54 AM Sean Owen wrote: > +1 > > I tested the source release. > Hashes and signature (your signature) check out, project builds and tests > pass with -Phadoop-2.7 -Pyarn -Phive -Pmesos on Debian 9. > List of issues look good and

Re: [VOTE][SPIP] SPARK-21866 Image support in Apache Spark

+1 On Thu, Sep 21, 2017 at 11:15 Sean Owen wrote: > Am I right that this doesn't mean other packages would use this > representation, but that they could? > > The representation looked fine to me w.r.t. what DL frameworks need. > > My previous comment was that this is

Re: [VOTE] Spark 2.1.2 (RC1)

+1 (non-binding) On Thu, Sep 14, 2017 at 10:57 PM Felix Cheung wrote: > +1 tested SparkR package on Windows, r-hub, Ubuntu. > > _ > From: Sean Owen > Sent: Thursday, September 14, 2017 3:12 PM > Subject: Re: [VOTE]

[TYPO3-german] Re: Powermail speichert Flexform nicht vollständig RTE Fehler

Datenbank utf8_general_ci Tabelle tt_content: InnoDB utf8_general_ci ___ TYPO3-german mailing list TYPO3-german@lists.typo3.org http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-german

[TYPO3-german] Powermail speichert Flexform nicht vollständig RTE Fehler

Hallo, vorab einmal vielen Dank für das super Forum was mir schon so einige Male gut weitergeholfen hat. Leider stoße ich aktuell bei einem mir nicht erklärbaren Fehler an meine Grenzen. Problembeschreibung: Beim Bearbeiten des Plugins erscheint die unten aufgeführte Fehlermeldung (Das

Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers

Congrats!! On Mon, Aug 7, 2017 at 17:39 Yanbo Liang wrote: > Great. > Congratulations, Hyukjin and Sameer! > > On Tue, Aug 8, 2017 at 7:53 AM, Holden Karau wrote: > >> Congrats! >> >> On Mon, Aug 7, 2017 at 3:54 PM Bryan Cutler

Re: With 2.2.0 PySpark is now available for pip install from PyPI :)

This is amazingly awesome! :) On Wed, Jul 12, 2017 at 13:23 lucas.g...@gmail.com wrote: > That's great! > > > > On 12 July 2017 at 12:41, Felix Cheung wrote: > >> Awesome! Congrats!! >> >> -- >> *From:*

Re: [VOTE] Apache Spark 2.2.0 (RC6)

+1 (non-binding) On Mon, Jul 3, 2017 at 6:45 PM Liang-Chi Hsieh wrote: > +1 > > > Sameer Agarwal wrote > > +1 > > > > On Mon, Jul 3, 2017 at 6:08 AM, Wenchen Fan > > > cloud0fan@ > > > wrote: > > > >> +1 > >> > >> On 3 Jul 2017, at 8:22 PM, Nick Pentreath > > >

Re: [VOTE] Apache Spark 2.2.0 (RC4)

+1 non-binding Tested on macOS Sierra, Ubuntu 16.04 test suite includes various test cases including Spark SQL, ML, GraphFrames, Structured Streaming On Wed, Jun 7, 2017 at 9:40 PM vaquar khan wrote: > +1 non-binding > > Regards, > vaquar khan > > On Jun 7, 2017 4:32

Re: Spark Shell issue on HDInsight

com> wrote: > Works for me tooyou are a life-saver :) > > But the question: should/how we report this to Azure team? > > On Fri, May 12, 2017 at 10:32 AM, Denny Lee <denny.g@gmail.com> wrote: > >> I was able to repro your issue when I had downloaded the ja

Re: Spark Shell issue on HDInsight

SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185) > at > org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210) > at org.apache.spark.deploy.SparkSubmi

Re: Spark Shell issue on HDInsight

This appears to be an issue with the Spark to DocumentDB connector, specifically version 0.0.1. Could you run the 0.0.3 version of the jar and see if you're still getting the same error? i.e. spark-shell --master yarn --jars

Re: [VOTE] Apache Spark 2.1.1 (RC4)

+1 On Fri, Apr 28, 2017 at 9:17 AM Kazuaki Ishizaki wrote: > +1 (non-binding) > > I tested it on Ubuntu 16.04 and OpenJDK8 on ppc64le. All of the tests for > core have passed.. > > $ java -version > openjdk version "1.8.0_111" > OpenJDK Runtime Environment (build >

Re: Azure Event Hub with Pyspark

As well, perhaps another option could be to use the Spark Connector to DocumentDB (https://github.com/Azure/azure-documentdb-spark) if sticking with Scala? On Thu, Apr 20, 2017 at 21:46 Nan Zhu wrote: > DocDB does have a java client? Anything prevent you using that? > >

Re: [VOTE] Apache Spark 2.1.1 (RC3)

+1 (non-binding) On Wed, Apr 19, 2017 at 9:23 PM Dong Joon Hyun wrote: > +1 > > I tested RC3 on CentOS 7.3.1611/OpenJDK 1.8.0_121/R 3.3.3 > with `-Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver > –Psparkr` > > At the end of R test, I saw `Had CRAN check

Support Stored By Clause

Per SPARK-19630, wondering if there are plans to support "STORED BY" clause for Spark 2.x? Thanks!

Re: welcoming Burak and Holden as committers

Awesome! Congrats Burak & Holden!! On Tue, Jan 24, 2017 at 10:39 Joseph Bradley wrote: > Congratulations Burak & Holden! > > On Tue, Jan 24, 2017 at 10:33 AM, Dongjoon Hyun > wrote: > > Great! Congratulations, Burak and Holden. > > Bests, > Dongjoon.

Re: unsubscribe

Please unsubscribe by sending an email to user-unsubscr...@spark.apache.org HTH! On Mon, Jan 9, 2017 4:40 PM, william tellme williamtellme...@gmail.com wrote:

Re: UNSUBSCRIBE

Please unsubscribe by sending an email to user-unsubscr...@spark.apache.org HTH! On Mon, Jan 9, 2017 4:41 PM, Chris Murphy - ChrisSMurphy.com cont...@chrissmurphy.com wrote: PLEASE!!

Re: [VOTE] Apache Spark 2.1.0 (RC5)

+1 (non-binding) On Sat, Dec 17, 2016 at 11:45 PM Liwei Lin wrote: > +1 > > Cheers, > Liwei > > On Sat, Dec 17, 2016 at 10:29 AM, Yuming Wang wrote: > > I hope https://github.com/apache/spark/pull/16252 can be fixed until > release 2.1.0. It's a fix for

Re: Spark app write too many small parquet files

Generally, yes - you should try to have larger data sizes due to the overhead of opening up files. Typical guidance is between 64MB-1GB; personally I usually stick with 128MB-512MB with the default of snappy codec compression with parquet. A good reference is Vida Ha's presentation Data Storage

[jira] [Created] (SPARK-18426) Python Documentation Fix for Structured Streaming Programming Guide

Denny Lee created SPARK-18426: - Summary: Python Documentation Fix for Structured Streaming Programming Guide Key: SPARK-18426 URL: https://issues.apache.org/jira/browse/SPARK-18426 Project: Spark

Re: Handling questions in the mailing lists

lan to have SO as a main resource. > > I know I originally suggested SO, however, if we do not have contributors > with the required privileges and the willingness to help manage everything > then I am not sure this is a good fit. > > Assaf. > > > > *From:* Denny Lee [vi

Re: [VOTE] Release Apache Spark 2.0.2 (RC3)

+1 (non binding) On Tue, Nov 8, 2016 at 10:14 PM vaquar khan wrote: > *+1 (non binding)* > > On Tue, Nov 8, 2016 at 10:21 PM, Weiqing Yang > wrote: > > +1 (non binding) > > > Environment: CentOS Linux release 7.0.1406 (Core) / openjdk version

Re: Handling questions in the mailing lists

Here here! :) Completely agree with you - here's the latest updates to Proposed Community Mailing Lists / StackOverflow Changes . Keep them coming though at this point, I'd like to limit new verbiage to prevent

Re: Handling questions in the mailing lists

treaming tag etc.). The main issue I see and the one we can’t seem > to get around is dividing between simple questions that the community > should answer and hard questions which only advanced users can answer. > > > > Maybe SO isn’t the correct platform for that but even within

Re: Handling questions in the mailing lists

To help track and get the verbiage for the Spark community page and welcome email jump started, here's a working document for us to work with: https://docs.google.com/document/d/1N0pKatcM15cqBPqFWCqIy6jdgNzIoacZlYDCjufBh2s/edit# Hope this will help us collaborate on this stuff a little faster.

Re: hope someone can recommend some books for me,a spark beginner

There are a number of great resources to learn Apache Spark - a good starting point is the Apache Spark Documentation at: http://spark.apache.org/documentation.html The two books that immediately come to mind are - Learning Spark: http://shop.oreilly.com/product/mobile/0636920028512.do (there's

Re: Newbie question - Best way to bootstrap with Spark

The one you're looking for is the Data Sciences and Engineering with Apache Spark at https://www.edx.org/xseries/data-science-engineering-apacher-sparktm. Note, a great quick start is the Getting Started with Apache Spark on Databricks at https://databricks.com/product/getting-started-guide HTH!

Re: How do I convert a data frame to broadcast variable?

If you're able to read the data in as a DataFrame, perhaps you can use a BroadcastHashJoin so that way you can join to that table presuming its small enough to distributed? Here's a handy guide on a BroadcastHashJoin:

Re: GraphFrame BFS

You should be able to GraphX or GraphFrames subgraph to build up your subgraph. A good example for GraphFrames can be found at: http://graphframes.github.io/user-guide.html#subgraphs. HTH! On Mon, Oct 10, 2016 at 9:32 PM cashinpj wrote: > Hello, > > I have a set of data

[jira] [Created] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

Denny Lee created SPARK-18200: - Summary: GraphX Invalid initial capacity when running triangleCount Key: SPARK-18200 URL: https://issues.apache.org/jira/browse/SPARK-18200 Project: Spark Issue

Re: [VOTE] Release Apache Spark 2.0.2 (RC1)

Oh, I forgot to note that when downloading and running against the Spark 2.0.2 without Hadoop binaries, I got a JNI error due to an exception with org / slf4j / logger (i.e. org.slf4j.Logger class is not found). On Mon, Oct 31, 2016 at 4:35 PM Reynold Xin wrote: > OK I

Re: [VOTE] Release Apache Spark 2.0.2 (RC1)

+1 (non-binding) On Thu, Oct 27, 2016 at 3:36 PM Ricardo Almeida < ricardo.alme...@actnowib.com> wrote: > +1 (non-binding) > > built and tested without regressions from 2.0.1. > > > > On 27 October 2016 at 19:07, vaquar khan wrote: > > +1 > > > > On Thu, Oct 27, 2016 at

Re: welcoming Xiao Li as a committer

Congrats, Xiao! On Tue, Oct 4, 2016 at 00:00 Takeshi Yamamuro wrote: > congrats, xiao! > > On Tue, Oct 4, 2016 at 3:59 PM, Hyukjin Kwon wrote: > > Congratulations! > > 2016-10-04 15:51 GMT+09:00 Dilip Biswal : > > Hi Xiao, > >

Re: [VOTE] Release Apache Spark 2.0.1 (RC4)

+1 (non-binding) On Thu, Sep 29, 2016 at 9:43 PM Jeff Zhang wrote: > +1 > > On Fri, Sep 30, 2016 at 9:27 AM, Burak Yavuz wrote: > >> +1 >> >> On Sep 29, 2016 4:33 PM, "Kyle Kelley" wrote: >> >>> +1 >>> >>> On Thu, Sep 29, 2016 at 4:27 PM,

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

lt;sam...@databricks.com>: >>> >>>> +1 (non-binding) >>>> >>>> On Mon, Sep 26, 2016 at 9:54 AM, Davies Liu <dav...@databricks.com> >>>> wrote: >>>> >>>>> +1 (non-binding) >>>>> >>>>

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) On Sun, Sep 25, 2016 at 23:20 Jeff Zhang wrote: > +1 > > On Mon, Sep 26, 2016 at 2:03 PM, Shixiong(Ryan) Zhu < > shixi...@databricks.com> wrote: > >> ＋1 >> >> On Sun, Sep 25, 2016 at 10:43 PM, Pete Lee wrote: >> >>> +1 >>> >>> >>> On

Re: Welcoming Felix Cheung as a committer

Awesome - congrats Felix! On Mon, Aug 8, 2016 at 9:44 PM Felix Cheung wrote: > Thank you! > Looking forward to work with you all! > > > > > > On Mon, Aug 8, 2016 at 7:41 PM -0700, "Yanbo Liang" > wrote: > > Congrats Felix! > > 2016-08-08 18:21

  1   2   3   >