Re: Use /usr/bin/env python3 in scripts?

For me merge script worked for python 2.7, but I got some trouble with the encoding issue (probably from contributor's name) so now I use the merge script with virtualenv & python 3.7.7. "python3" would be OK for me as well as it doesn't break virtualenv with python 3. On Sat, Jul 18, 2020 at 6:1

Re: Use /usr/bin/env python3 in scripts?

+1 I'm in favor of using python3 Cheers, Fokko Op vr 17 jul. 2020 om 19:49 schreef Sean Owen : > Yeah I figured it's a best practice, so I'll raise a PR unless > somebody tells me not to. This is about build scripts, not Pyspark > itself, and half the scripts already specify python3. > > On Fri,

Re: R installation broken on ubuntu workers, impacts K8s PRB builds

this is done, except for amp-jenkins-staging-worker-02 which is refusing to allow me to reinstall R... i marked that worker offline and will beat on it later today. On Fri, Jul 17, 2020 at 11:36 AM shane knapp ☠ wrote: > starting now... pausing jenkins so no new builds are launched. > > On Thu

Re: R installation broken on ubuntu workers, impacts K8s PRB builds

starting now... pausing jenkins so no new builds are launched. On Thu, Jul 16, 2020 at 3:09 PM Holden Karau wrote: > Sounds good, thanks. No rush :) > > On Thu, Jul 16, 2020 at 3:03 PM shane knapp ☠ wrote: > >> i'll get to this tomorrow afternoon, and there will be a short downtime. >> more de

Re: Use /usr/bin/env python3 in scripts?

Yeah I figured it's a best practice, so I'll raise a PR unless somebody tells me not to. This is about build scripts, not Pyspark itself, and half the scripts already specify python3. On Fri, Jul 17, 2020 at 12:36 PM Oli McCormack wrote: > > [Warning: not spark+python specific information] > > It

Re: Use /usr/bin/env python3 in scripts?

[Warning: not spark+python specific information] It's recommended that you should explicitly call out python3 in a case like this (see PEP-0394 , and SO

Use /usr/bin/env python3 in scripts?

So, we are on Python 3 entirely now right? It might be just my local Mac env, but "/usr/bin/env python" uses Python 2 on my mac. Some scripts write "/usr/bin/env python3" now. Should that be the case in all scripts? Right now the merge script doesn't work for me b/c it was just updated to be Python

Re: Catalog API for Partition

Hi, wenchen. Thanks for your attention and reply. Firstly. These Partition Catalog APIs are not specially used for hive, they can be used with LakeHouse or myql or other source support partitions. Secondly. These Partition Catalog APIs are only designed for better data management, not for speed up

Re: Catalog API for Partition

In Hive, partition does two things: 1. Act as an index to speed up data scan 2. Act as a way to manage the data. People can add/drop partitions. How do you unify these 2 things in your API design? On Fri, Jul 17, 2020 at 12:03 AM JackyLee wrote: > Hi devs, > > In order to support Partition Comm

Re: Welcoming some new Apache Spark committers

You earned it Dilip. Congrats again! On Fri, 17 Jul 2020, 14:13 Dilip Biswal, wrote: > Thank you all for your kind words. A special "thank you" to *Xiao Li *for > his help and mentorship over the years that helped me immensely. I would > also like to mention *Wenchen Fan*, *Takeshi Yamamuro,* *S