Christophe, On Wed, Dec 6, 2017 at 10:56 AM, Christophe Meessen < christophe.mees...@gmail.com> wrote:
> Hello, > > I'm a computer scientist in charge of developing an image processing > pipeline for telescope images. > It will also have a web server and DB connection. > > The project is going through reviews by external experts, and the problem > I'm facing is that my proposal to use Go is about to be rejected. > > The main opposing arguments are > - everybody uses python in astrophysics > - it is very easy to find someone who knows python > - risk that I, sole Go programmer, might become unavailable > > I would have the same arguments if I was project leader and unfamiliar > with Go. > > The counter arguments I found so far are that > - Go is simpler and safer than Python > - I learned Go in a week-end > > The problem is that they don't convince people who don't know Go, are not > experienced software developers, and don't want to do the due diligence. > It's the usual inertia to change. > > What other arguments could I use ? > > Do you know other significant scientific experiments that have adopted Go > ? > in physics, I know 3 experiments that have parts of their software written in Go: - SoLiD (they wrote their data acquisition system in Go, as well as some environmental sensors monitoring (on raspi3)). The DAQ code isn't public but there is an IEEE presentation about it (Nick Ryder) The monitoring code is there: https://github.com/sbinet-solid/solid-mon-rpi - LSST: one of the subcomponents testing system has been written in Go (slow control): https://github.com/go-lsst/fcs-lpc-motor-ctl - EIC: there is some R&D code for the Electron-Ion collider that is being written in Go (and can generate code for C++, Python and Java) https://github.com/decibelCooper/eicio In my lab, there is also a team working with transfering HEP technologies to the medical environment, that is using Go for their DAQ+monitoring. (and babysteps for the analysis too) There's an IEEE publication that has just been accepted (Busato et al.), not sure it's public yet. An early version of the code is there: https://github.com/ ebusato/analysis-go You may also want to talk to Fabio Hernandez (from LSST/CC-IN2P3) about his experience with Go. He may have some interesting "reviewers worthy" insights. There are also a few interesting research papers about Go: https://github.com/golang/go/wiki/ResearchPapers many are CS oriented, but a few Gonum friends have also a few papers *using* Go: - http://arc.aiaa.org/doi/abs/10.2514/6.2015-1287 - https://arxiv.org/abs/1706.02419 you also probably know about my work: https://go-hep.org https://github.com/astrogo but there is also a great library for biology (some of the "students" I had 2 weeks ago at a Go tutorial in Marseille, knew about it): - https://github.com/biogo/biogo - http://dx.doi.org/10.21105/joss.00167 - http://biorxiv.org/content/early/2014/05/12/005033 You could also perhaps mention SpaceX (which is -apparently- using Go for its rocket monitoring system.) and I am in the middle of writing an aXiv preprint wrt why I think HEP should migrate en masse to Go. it's only in my head right now, but the seeds for that article are contained in my ACAT-2017 presentation: - http://talks.godoc.org/github.com/go-hep/talks/2017/2017-08-24-go-hep-acat/talk.slide and the (not yet approved) accompanying paper: - https://github.com/go-hep/talks/tree/master/2017/2017-08-24-go-hep-acat finally, here is what I got with a Master-1 Physics student who knew a bit of Python (and had no previous exposure to Go), in a 3 months internship: - https://github.com/lsst-lpc/hubble-fit a little cosmo analysis replicating results from a previous python-based analysis involving reading a few FITS files containing a few 2220x2220 matrices that are inverted and such, and then a minimization is applied to extract Hubble-related parameters. it was nice to see the student learning Go in a few weeks and producing a version that was OK, w/o much help from my end. the final version is faster than the python (with numpy and C++ under the hood) one, in single-threaded mode. and ~2 faster with 2 cores. ok, I don't know whether this is significant enough. >From a "close to the ground" angle, typical arguments for a Go .vs. Python are: - static type system so: refactoring is less brittle, smaller amount of errors discovered at runtime, better runtime performances - even if the language is compiled and statically typed, the development speed isn't hindered too much thanks to the type inference system and the fast compiler (so edit-compile-run dev cycles are *comparable* to python) - concurrency programming. no need for the "multiprocessing" and "threading" python modules. concurrency is builtin and the language was designed with it from the start. - the language is simpler than python (I always show the "for i in range(x): print(i)" snippet which involves knowing about exceptions, coroutines and special double-under methods). python looks deceptively simple, but isn't. Go is more WYSIWYG. - deployment: just a single binary to ship, that can be easily cross-compiled, without the headache of Python2/Python3 (the astro community has mostly migrated to Py3, now, though). - better usage resources (CPU, VMem) - refactoring (again), especially when it's automated, thanks to the great Go tooling. this is especially important for scientific experiments that need to operate over many years and thus that need to be maintained (and groomed) for as long a period. For a more holistic angle, I usually refer to the Splash article: https://talks.golang.org/2012/splash.article (and its associated slides https://talks.golang.org/2012/splash.slide#1). That said, it's still a tough sell, because not nearly half of the libraries that available for physics analyses are available from Go. Of course, we have Gonum that is already quite competitive wrt numpy/scipy. But astropy has way more (battletested) libraries than what I am currently providing in astrogo. I don't know where you envisionned to use Go (analysis? daq? slow-control? control-framework? everything?) but that could play a huge role in your strategy to best show the advantages of Go wrt a numpy/scipy/astropy stack (not everything can be rewritten into a nice numpy-vectorized loop) also, it may not be an either/or situation: one could imagine having the user-facing, end-user part written in CPython and still have the heavy lifting (with concurrency) in Go, neatly exposed as CPython extension modules. it all depends on where it makes the most sense (and is cost effective) to put Go. > > > I have found this github project. https://github.com/indigo-astr > onomy/indigo > INDI is a well known Python Observatory Control System. > INDIGO is its translation into Go. > is it ? looks like C to me :) hth, -s PS: you could also perhaps get more "sciency" feedback on the gonum-dev mailing list. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
