Hello.
Any "evolved" code editor allows you to create tasks (or builds, or whatever
they are called in the IDE you chose). If you do not find anything that
packages by default all you need, you could just create your own tasks.
For yarn, one needs to open a terminal and submit from there.
You can create task(s) that launch your yarn commands.
With VSC, you get stuff for working with json files but I am not sure with a
plugin for Python
In your json task configuration, you can launch whatever you want: python,
shell. I bet you could launch your favorite video game (just make a task called
"let's have a break" 😉)
Just to say, if you want everything exactly the way you want, I do not think
you will find an IDE that does it. You will have to customize it. (correct me
if wrong, of course).
Have a good day.
[signature_299490615]<https://www.neuroo.ai/>
[Banner]<http://www.redlab.io/>
Florian CASTELAIN
Ingénieur Logiciel
72 Rue de la République, 76140 Le Petit-Quevilly
m: +33 616 530 226
e: florian.castel...@redlab.io<mailto:florian.castel...@redlab.io> w:
www.redlab.io<http://www.redlab.io/>
De : Jeff Zhang
Envoyé : jeudi 30 septembre 2021 13:57
À : Mich Talebzadeh
Cc : user @spark
Objet : Re: Choice of IDE for Spark
IIRC, you want an IDE for pyspark on yarn ?
Mich Talebzadeh mailto:mich.talebza...@gmail.com>>
于2021年9月30日周四 下午7:00写道：
Hi,
This may look like a redundant question but it comes about because of the
advent of Cloud workstation usage like Amazon workspaces and others.
With IntelliJ you are OK with Spark & Scala. With PyCharm you are fine with
PySpark and the virtual environment. Mind you as far as I know PyCharm only
executes spark-submit in local mode. For yarn, one needs to open a terminal and
submit from there.
However, in Amazon workstation, you get Visual Studio
Code<https://code.visualstudio.com/> (VSC, an MS product) and openoffice
installed. With VSC, you get stuff for working with json files but I am not
sure with a plugin for Python etc, will it be as good as PyCharm? Has anyone
used VSC in anger for Spark and if so what is the experience?
Thanks
[https://docs.google.com/uc?export=download&id=1-q7RFGRfLMObPuQPWSd9sl_H1UPNFaIZ&revid=0B1BiUVX33unjMWtVUWpINWFCd0ZQTlhTRHpGckh4Wlg4RG80PQ]
view my Linkedin
profile<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
Disclaimer: Use it at your own risk. Any and all responsibility for any loss,
damage or destruction of data or any other property which may arise from
relying on this email's technical content is explicitly disclaimed. The author
will in no case be liable for any monetary damages arising from such loss,
damage or destruction.
--
Best Regards
Jeff Zhang
