Hi all, Here is draft of my proposal. I will be very glad if you review it. Should it be more detailed or shorter? I stayed close with publicated ideas ( https://code.google.com/p/ganeti/wiki/SummerOfCode2016Ideas) and give my own plan to solve givin general task and sub-tasks. Any your comments are highly welcome. (direct link to gdoc: https://docs.google.com/document/d/1o5aaB-tWCK7wq-Uq9BEW9plC80FGRq6fZWGcjciXN8A/edit?usp=sharing )
Introduction. Ganeti is a cluster virtual server management software tool, so one of important task of it is to achieve as good performance for whole cluster as possible. In Ganeti this problem partially solved by balancing module HTools, which calculate series of moves for achieve more cluster optimality. There are several problems here: 1. Some of steps unacceptable amounts of time. 2. There is no any estimations for each step of balancing. 3. User can’t control this process. I wish to solve the task of optimize balancing speed and providing balancing user options for HTools. The main idea is to get make process of balancing more predictive and controllable. After that improve HTools by preventing bad moves during balancing calculation. Project goals. List of deliverables 1. Interface providing time estimation for each HTools move. 2. Implement user option(s) for limiting balancing time and avoiding long moves. 3. Implement network speed data extraction. (small ganeti demon) 4. Implement collecting activity memory usage data provided by Qemu. Implementation. Firstly, I suggest to implement general interface for HTools balancing functions (e.g. Cluster.tryBalance), in order to measure time for each move. I’m sure, that this stage also should contains at least few days for discussing details of this interface and arch of further changers. After that needed user option (--avoid-long-moves) could be implemented (Also add opt to AlgorithmParams). At first approximation we can use interfaces’ disk volumes for time estimations. Further, for accurate migration time prediction network speed data needed. The idea is to implement ganeti daemon, which will measure it by sending packets between nodes. Also it's could be reasonable to add this measured information to HTools/Node.hs In order to collect data of memory usage activity I suggest to get this from Qemu directly. So it will be necessary to add this functionality to DataCollectors module. Finally with new data it will be possible to estimate time to migration much more accurate. Here I wish to implement an algorithm based on gradient optimisation, which will remove balancing moves with unapropriate estimated time from balancing series. Timeline. Week 1: Architectural discuss. (also exams in institute) Week 2-3: Implement interface for time estimations in HTools. Week 4: Implement necessary user options. (--avoid-long-moves) Week 5: Catch-up week. Week 6-7: Implement ganeti daemon for collecting node’s network speed data. Week 8-9: Add functionality to DataCollectors for get instances’ mem usage activity. Week 10: Add scoring for moves based on time estimations and optimize balancing. Week 11: Implementing tests. (also 3-5 vacation days of volunteer work) Week 12: Catch-up week. About me. Daniil Leshchev Email: [email protected] Phone: +79105505550 My name is Daniil Leshchev, I’m first year undergraduate master at Moscow Institute of Physics and Technology, Russia. My specialization is Applied Mathematics and CS. I have variety experience in area of virtualization and close technologies. I was working in research lab of Parallels company for two years and was involved in a research project of ARM chips virtualization. this work was the basis of my bachelor thesis. I do not dwell on low-level programming, and have math skills, knowledge of statistics, machine learning and optimisation algorithms. Also I’m diving into Linux kernel and have a patch (/drivers/staging/rtl8723au), which has already merged to main tree (commit 78f73d92fc2467318a377f98fe0f74e37b0a4bc1). -- Regards, Daniil Leshchev
