Hi Michele this looks very cool in deed Where you able to create a container in k8s with a main.go that took websocket input and output to a persistent kafka connection
I'm interested in this as I have a use case that I want IBM Cloud Function to ingest into kafka, and I was going to build similar thing as you did using http 1or 2 server, that then ingest into kafka as a service (Event Streams). I was going to deploy this container into kubernetes, but I wanted the container to scale down to zero using Knative. You have such main.go for your runtime with websocket at one side and kafka producer at the other? -- Carlos On Thu, Jan 17, 2019 at 1:07 PM Michele Sciabarra <mich...@sciabarra.com> wrote: > Hello whiskers! > > Sorry it is a bit long, so I split it into parts with headlines. > > TL;DR > > I implemented support for Websocket so you can deploy an Action as a > WebSocket server if you have a Kubernetes cluster (or just a Docker > server). See at the end of this post for an example of a Kubernetes > deployment descriptor. > > Here is a very simple demo using it: > > https://hellows.sciabarra.net/ > > It uses a websocket server implemented using the golang runtime and the > code of an UNCHANGED OpenWhisk action. All the magic happens at deployment > using the descriptor provided. > > I believe it is a good foundation for implementing websocket support in > OpenWhisk. The next step would be to provide support at the API level. > > After reading the rest, the question is: does the community approve this > feature? If yes, I can submit a PR for including it in the next release of > the actionloop. > > 1. Motivation: why I did this > > A few days ago I asked what was the problem in having an action that > creates a persistent connection to Kafka. I was answered that a Serverless > environment can flood Kafka with requests because more actions are spawn > when the load increase. > > The solution is to create a separate server to be deployed somewhere, for > example, Kubernetes, maybe using WebSockets to communicate with Kafka. In > short, I had the need to transform an action in a WebSocket server. > > Hence I had the idea of adding WebSocket support to Action as WebSocket > server, adding support for WebSocket to ActionLoop, so I could create a > WebSocket server in the same way as you write an action. > > 2. What I did > > I implemented WebSocket support in the action runtime. If you deploy the > action now it answers not only to `/run` but also to `/ws` (it is > configurable) as a WebSocket in continuous mode. > > You enable the WebSocket setting the environment variable OW_WEBSOCKET. > Also, for the sake of easy deployment, there is also now an autoinit > feature. If you set the environment variable to OW_AUTOINIT, it will > initialize the runtime from the file you specified in the variable. > > Ok, fine you can say, but how can I use it? > > With a Kubernetes descriptor! You can launch the runtime in Kubernetes, > provide the main action in it (you can also download it from a git repo or > store in a volume), and now your action is a web socket server answering to > your requests. > > Look to the following descriptor for an example: > > It is a bit long, this is what it does > > - it creates a configmap containing the action code > - it launches the image mounting the action code > - the image initialize the action and then listen to the WebSocket > - the image also exposes the web socket using an ingress > > > apiVersion: v1 > kind: Namespace > metadata: > name: hellows > --- > apiVersion: v1 > kind: ConfigMap > metadata: > name: hellows > namespace: hellows > data: > main.go: | > package main > import "fmt" > func Main(obj map[string]interface{}) map[string]interface{} { > name, ok := obj["name"].(string) > if !ok { > name = "world" > } > fmt.Printf("name=%s\n", name) > msg := make(map[string]interface{}) > msg["golang-main-single"] = "Hello, " + name + "!" > return msg > } > --- > apiVersion: v1 > kind: Pod > metadata: > name: hellows > namespace: hellows > labels: > app: hellows > spec: > volumes: > - name: mnt > configMap: > name: hellows > containers: > - name: hellows > image: actionloop/golang-v1.11:ws3 > ports: > - containerPort: 8080 > protocol: TCP > volumeMounts: > - name: mnt > mountPath: "/mnt" > env: > - name: OW_WEBSOCKET > value: /hello > - name: OW_AUTOINIT > value: /mnt/main.go > --- > apiVersion: v1 > kind: Service > metadata: > name: hellows > namespace: hellows > spec: > ports: > - port: 8080 > protocol: TCP > targetPort: 8080 > selector: > app: hellows > --- > apiVersion: extensions/v1beta1 > kind: Ingress > metadata: > name: hellows > namespace: hellows > spec: > rules: > - host: hellows.sciabarra.net > http: > paths: > - path: /hello > backend: > serviceName: hellows > servicePort: 8080 > > > -- > Michele Sciabarra > mich...@sciabarra.com > -- Carlos Santana <csantan...@gmail.com>
