Re: Slack for PySpark users

I'm reading through the page "Briefing: The Apache Way", and in the section of "Open Communications", restriction of communication inside ASF INFRA (mailing list) is more about code and decision-making. https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define It's

Re: Creating InMemory relations with data in ColumnarBatches

Is this purely for performance consideration? Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies Limited view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it

Re: Slack for PySpark users

Good discussions and proposals.all around. I have used slack in anger on a customer site before. For small and medium size groups it is good and affordable. Alternatives have been suggested as well so those who like investigative search can agree and come up with a freebie one. I am inclined to

Re: Slack for PySpark users

+1. To Shani’s point, there are multiple OSS projects that use the free Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub, MLflow, etc. On Thu, Mar 30, 2023 at 14:15 wrote: > Hey everyone, > > I think we should remain on a free program in slack. > > In my option the free

Re: Slack for PySpark users

Hey everyone,I think we should remain on a free program in slack.In my option the free program is more then enough, the only down side is we could only see the last 90 days messages.From what I know the Airflow community (which has strong active community in slack) also use the free program (You

Re: Slack for PySpark users

Thanks for flagging the concern Dongjoon, I was not aware of the discussion - but I can understand the concern. Would be great if you or Matei could update the thread on the result of deliberations, once it reaches a logical consensus: before we set up official policy around it. Regards, Mridul

unsubscribe

unsubscribe

Re: Slack for PySpark users

I like the idea of having a talk channel. It can make it easier for everyone to say hello. Or to dare to ask about small or big matters that you would not have dared to ask about before on mailing lists. But then there is the price and what is the best for an open source project. The price for

Creating InMemory relations with data in ColumnarBatches

Hi, I have been trying to implement InMemoryRelation based on spark ColumnarBatches, so far I have not been able to store the vectorised columnarbatch into the relation. Is there a way to achieve this without going with an intermediary representation like Arrow, so as to enable spark to do fast

Re: Slack for PySpark users

Hi Dongjoon to your points if I may - Do you have any reference from other official ASF-related Slack channels? No, I don't have any reference from other official ASF-related Slack channels because I don't think that matters. However, I stand corrected - To be clear, I intentionally didn't

Re: Slack for PySpark users

To Mich. - Do you have any reference from other official ASF-related Slack channels? - To be clear, I intentionally didn't refer to any specific mailing list because we didn't set up any rule here yet. To Xiao. I understand what you mean. That's the reason why I added Matei from your side. > I

Re: Slack for PySpark users

Hi, Dongjoon, The other communities (e.g., Pinot, Druid, Flink) created their own Slack workspaces last year. I did not see an objection from the ASF board. At the same time, Slack workspaces are very popular and useful in most non-ASF open source communities. TBH, we are kind of late. I think we

Re: Slack for PySpark users

Hi Dongjoon, Thanks for your point. I gather you are referring to archive as below https://lists.apache.org/list.html?user@spark.apache.org Otherwise, correct me. Thanks Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies Limited view my Linkedin profile

Re: Slack for PySpark users

Hi, Xiao and all. (cc Matei) Please hold on the vote. There is a concern expressed by ASF board because recent Slack activities created an isolated silo outside of ASF mailing list archive. We need to establish a way to embrace it back to ASF archive before starting anything official. Bests,

[ANNOUNCE] Apache Celeborn(incubating) 0.2.1 available

Hi all, Apache Celeborn(Incubating) community is glad to announce the new release of Apache Celeborn(Incubating) 0.2.1 Celeborn is dedicated to improving the efficiency and elasticity of different map-reduce engines and provides an elastic, high-efficient service for intermediate data including

Re: Slack for PySpark users

The ownership of slack belongs to spark community Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies Limited view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use

Re: Slack for PySpark users

We already have it general - Apache Spark Community - Slack Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies Limited view my Linkedin profile

Re: Slack for PySpark users

Hey there,I agree, If Apache Spark PMC can maintain the spark community workspace, that would be great!Instead of creating a new one, they can also become the owner of the current one .Best regards,ShaniOn 30 Mar 2023, at 9:32, Xiao Li wrote:+1 + @d...@spark.apache.org This is a good idea. The

Re: Slack for PySpark users

+1 + @d...@spark.apache.org This is a good idea. The other Apache projects (e.g., Pinot, Druid, Flink) have created their own dedicated Slack workspaces for faster communication. We can do the same in Apache Spark. The Slack workspace will be maintained by the Apache Spark PMC. I propose to