I use jupyterlab on k8s with minio as s3 storage.
https://github.com/bjornjorgensen/jlpyk8s
With this code to start it all :)
from pyspark import pandas as ps
import re
import numpy as np
import pandas as pd
from pyspark.sql import SparkSession
from pyspark.sql.functions import concat,
99490615]**][signature_299490615_signature_299490615]
> > >
> > > | | |
> > > ----------
think you will find an IDE that does it. You will have to customize it.
>>> (correct me if wrong, of course).
>>>
>>> Have a good day.
>>>
>>> *[image: signature_299490615]* <https://www.neuroo.ai/>
>>>
>>>
>>>
>>>
od day.
>>
>> *[image: signature_299490615]* <https://www.neuroo.ai/>
>>
>>
>>
>> [image: Banner] <http://www.redlab.io/>
>>
>>
>>
>> *Florian CASTELAIN *
>> *Ingénieur Logiciel*
>>
>> 72 Rue de la République, 761
> With IntelliJ you are OK with Spark & Scala.
also intelliJ as a nice python plugin that turns it into pycharm.
On Thu Sep 30, 2021 at 1:57 PM CEST, Jeff Zhang wrote:
> IIRC, you want an IDE for pyspark on yarn ?
>
> Mich Talebzadeh 于2021年9月30日周四
> 下午7:00写道：
>
> > Hi,
> >
> > This may look
gt;
> [image: Banner] <http://www.redlab.io/>
>
>
>
> *Florian CASTELAIN *
> *Ingénieur Logiciel*
>
> 72 Rue de la République, 76140 Le Petit-Quevilly
> m: +33 616 530 226
> e: florian.castel...@redlab.io w: www.redlab.io
>
> ------
&g
_
De : Jeff Zhang
Envoyé : jeudi 30 septembre 2021 13:57
À : Mich Talebzadeh
Cc : user @spark
Objet : Re: Choice of IDE for Spark
IIRC, you want an IDE for pyspark on yarn ?
Mich Talebzadeh mailto:mich.talebza...@gmail.com>>
于2021年9月30日周四 下午7:00写道：
Hi,
This may look li
IIRC, you want an IDE for pyspark on yarn ?
Mich Talebzadeh 于2021年9月30日周四 下午7:00写道：
> Hi,
>
> This may look like a redundant question but it comes about because of the
> advent of Cloud workstation usage like Amazon workspaces and others.
>
> With IntelliJ you are OK with Spark & Scala. With
Hi,
This may look like a redundant question but it comes about because of the
advent of Cloud workstation usage like Amazon workspaces and others.
With IntelliJ you are OK with Spark & Scala. With PyCharm you are fine with
PySpark and the virtual environment. Mind you as far as I know PyCharm
