What do you want to accomplish, Simon? Run Grid Engine as a job processing service on top of Kubernetes which also runs other service workloads in parallel? Would you do so to be able to scale the size of the Grid Engine cluster? Or to make it fault tolerant or both? And would you want to run containerized workloads then also underneath Grid Engine?
There is a solid amount of work we’ve done at Univa in this context. You may or may not be aware that we provide means to install Kubernetes clusters more easily with Navops Launch and we add sophisticated workload management policies to Kubernetes with Navops Command (see http://navops.io for more information). We also provide Docker integration to Grid Engine via Univa Grid Engine Container Edition. In those contexts we’ve done work to run Grid Engine on top of Kubernetes and it is not trivial. To do this more cleanly some changes were necessary like having a REST API for Grid Engine which we offer. If you are, on the other hand, are asking whether one can run high performance or high throughput workloads you typically would process via Grid Engine on top of Kubernetes instead then the answer leans towards a 'no'. Your mileage can always vary but Kubernetes really is geared at running persistent, microservice-based applications which can autoscale by adding or removing replicas of microservice instances. It does have a job concept but it is nothing similar to what you have in tools like Grid Engine. And that doesn’t even talk about the policies needed to manage diverse job-style workloads (sequential, parallel, arrays spread across coming from groups of users or projects, etc) or the scheduling performance required for throughput computing. So if you could specify your use case in more detail I might be able to provide a more targeted answer. Cheers, Fritz > Am 04.08.2016 um 07:01 schrieb Simon Matthews <[email protected]>: > > Has anyone constructed a cluster using Kubernetes, Docker and > Gridengine (SGE or newer variants)? Does this even make sense? Is > there a better set of tools to achieve something similar? > > Simon > _______________________________________________ > users mailing list > [email protected] > https://gridengine.org/mailman/listinfo/users Fritz Ferstl | CTO and Business Development, EMEA Univa Corporation <http://www.univa.com/> | The Data Center Optimization Company E-Mail: [email protected] | Phone: +49.941.5695.9351 | Mobile: +49.170.819.7390
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
