I agree on focusing on binary support for now. There is only one "annoyance" here: I noticed the dockerskeleton always expects a zip file including an "exec". I would like to be able to support also binaries pushed straight, and I was trying to do that using magic file detection (there is a Go library for that).
To be honest, as I told already on Slack, I am a bit skeptic on source support tot court as it is implemented for example in the Swift image. It is noticeably slow. Go is a bit faster but the problem with Go is downloading dependencies (that can be a lot...). You will have a long startup time when you scale, and having a long startup time when you need quickly more instances is definitely a killer problem for scaling support. As I mentioned, for binaries like swift and go, and also for Java, I was thinking we should have something similar to what Kubernetes allows to do: an image for building, producing an executable, to be cached somewhere, to be reused when you launch more images. Just my 2cents of ideas.... -- Michele Sciabarra mich...@sciabarra.com On Mon, Feb 26, 2018, at 1:24 PM, Rodric Rabbah wrote: > Supporting go as source is nice for parity but you can also stage the > changes focusing on the binary case first, get the /run details nailed, > and working through logs and full experience first. Go cross-compilation > works well and if you’re writing in go, beyond the convenience of > quickly getting started with a source snippet, I would speculate the > common use case is for the binary path. > > -r > > > On Feb 25, 2018, at 10:39 PM, Carlos Santana <csantan...@gmail.com> wrote: > > > > Great progress Michele +1 > > > > I agree with the order of your steps. > > > > Step #1 > > Let's nock step 1 first. > > I will summarize it as having first class support for Go at the same level > > of JavaScript, Python, Java, Swift, PHP > > Meaning from end user support from CLI, docs, runtime image. > > wsk action create goaction goaction.go > > wsk action create goaction goexec --kind go:1.10 > > wsk action create goaction goaction.zip --kind go:1.10 > > extend the runtime image with anything you want to add to the image > > From example: > > FROM openwhisk/action-go-v1.10 > > apt-get install -y imagemagick > > docker build . -t csantanapr/action-go-v1.10 > > wsk action create goaction goaction.go --docker csantanapr/action-go-v1.10 > > I guess will have a FDK (Functions Developer Kit) :-), making it super easy > > to import it and just write a simple go main method/function the FDK > > handles the json input, output, etc.. > > > > > > I can help by requesting a new runtime repo > > "apache/incubator-openwhisk-runtime-go" > > I can setup a boilerplate repo content with some structure for gradle, > > build, travis, scala tests > > Then you can submit PR to the repo with go specifics. > > I guess since your go guru you can handle the few changes to the wsk cli > > Do you have a repo with some code already I can take a look? > > > > Step #2 > > For swift, I can help since I'm working on swift4, it looks like we can > > have a new proxy instead of using the python one. > > We do some pre-compilation/magic today with buildandrecord.py [1] to > > compile as much as possible up front and creating a shell script that has > > the exact lines for compiling and linking. > > Do you have some code on swift consuming from pipe instead of ReadLine I > > can take a look? > > > > Step #3 > > After doing go and swift, we can see what are the common parts we can > > extract to be common in the docker runtime repo. > > And work on backwards compatible single image, and handle the new pipe with > > some annotation, or maybe a new image, we can discuss more deeply once we > > get there. > > > > > > [1] > > https://github.com/apache/incubator-openwhisk-runtime-swift/blob/master/core/swift41Action/buildandrecord.py > > > > > > > > > > > > > > On Sat, Feb 24, 2018 at 1:58 PM Michele Sciabarra <openwh...@sciabarra.com> > > wrote: > > > >> Hello, whiskers > >> > >> I am at the risk of expanding too much the scope of what I am trying to > >> do, so I want to try to focus and work step by step. > >> > >> So I am writing down my plan for review. > >> > >> # Step one: implement a good support for GoLang, with piped standard input > >> and output, and log in stderr. > >> > >> Because there is not a past support for Go, except the generic docker > >> support, I think I have some freedom here. > >> > >> The support will then work this way (please review carefully): > >> > >> ## /init > >> > >> The runtime expects posted in /init the usual > >> > >> { > >> "binary": ... > >> "code": ... > >> "main": .. > >> } > >> > >> where "code" is either a zip file (with binary: true) or a source code > >> (with binary: false or absent). > >> > >> If it source code, it expects to be a go source. > >> It places in /action/main/exec.go and compiles it with go build -o > >> /action/exec /action/main/exec.go, getting a /action/exec to execute. > >> > >> If is is a zip file, it will unzip it in /action. If there is a > >> /action/exec file in the zip it will run it. > >> > >> If there is not, it will look if there is a "main" parameter, getting the > >> <main-value>. > >> > >> It will compile a /action/main/<main-value>.go. If the <main-value> is > >> missing, it will default to /action/main/exec.go. > >> > >> Note the zip can include other source files that will be stored in > >> /action/<folders>. So I will have to use GOPATH=/action to find them. > >> > >> ## /run > >> > >> The /run behaviour is: > >> > >> At every request, if no action process is running,it will start one and > >> will pass the first line as an argument and also in standard output. > >> > >> If the executable does not terminate, it will keep feeding more input in > >> standard input, expecting output in standard output (line by line) and logs > >> in stderr. > >> > >> The server will have to collect stderr logs and write them in docker > >> stdout, and terminate the output with the markers when an action completes: > >> > >> XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX > >> XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX > >> > >> It the executable DOES terminate, it will execute again at the next /run > >> request > >> > >> ## Step 2: need to implement the same logic for swift actions (we have to > >> keep the existing compilation infrastructure) > >> > >> Issues to solve here are the compilation and the "warm" startup problem. I > >> have some ideas but... I do better to discuss them once the Go support is > >> finished. > >> > >> ## Step 3: will discuss with the list if it is worth to extend the support > >> to the generic dockerskeleton, that is based on the python proxy. > >> > >> There are compatibility issues, because the current docker support logs > >> everything in stdout and the last line is the answer, while for using the > >> pipe you need to do separate log. > >> > >> How does it sound? > >> Suggested or required changes? > >> Am I missing something? > >> Approved? > >> > >> > >> -- > >> Michele Sciabarra > >> openwh...@sciabarra.com > >>
