Hi Daniil. I've just add several comments to the beginning of your Google doc. I am going to finish the commenting process this Evening.
On Thu, Mar 24, 2016, 00:44 Даниил Лещёв <[email protected]> wrote: > Hi all, > Here is draft of my proposal. I will be very glad if you review it. > Should it be more detailed or shorter? > I stayed close with publicated ideas ( > https://code.google.com/p/ganeti/wiki/SummerOfCode2016Ideas) and give my > own plan to solve givin general task and sub-tasks. > Any your comments are highly welcome. > (direct link to gdoc: > https://docs.google.com/document/d/1o5aaB-tWCK7wq-Uq9BEW9plC80FGRq6fZWGcjciXN8A/edit?usp=sharing > ) > > > > Introduction. > > Ganeti is a cluster virtual server management software tool, so one of > important task of it is to achieve as good performance for whole cluster as > possible. > > In Ganeti this problem partially solved by balancing module HTools, which > calculate series of moves for achieve more cluster optimality. There are > several problems here: > > 1. > > Some of steps unacceptable amounts of time. > 2. > > There is no any estimations for each step of balancing. > 3. > > User can’t control this process. > > I wish to solve the task of optimize balancing speed and providing > balancing user options for HTools. > > The main idea is to get make process of balancing more predictive and > controllable. After that improve HTools by preventing bad moves during > balancing calculation. > > > Project goals. > > List of deliverables > > 1. > > Interface providing time estimation for each HTools move. > 2. > > Implement user option(s) for limiting balancing time and avoiding long > moves. > 3. > > Implement network speed data extraction. (small ganeti demon) > 4. > > Implement collecting activity memory usage data provided by Qemu. > > > Implementation. > > Firstly, I suggest to implement general interface for HTools balancing > functions (e.g. Cluster.tryBalance), in order to measure time for each > move. I’m sure, that this stage also should contains at least few days > for discussing details of this interface and arch of further changers. > > After that needed user option (--avoid-long-moves) could be implemented > (Also add opt to AlgorithmParams). At first approximation we can use > interfaces’ disk volumes for time estimations. > > Further, for accurate migration time prediction network speed data needed. > The idea is to implement ganeti daemon, which will measure it by sending > packets between nodes. Also it's could be reasonable to add this measured > information to HTools/Node.hs > > In order to collect data of memory usage activity I suggest to get this > from Qemu directly. So it will be necessary to add this functionality to > DataCollectors module. > > Finally with new data it will be possible to estimate time to migration > much more accurate. Here I wish to implement an algorithm based on > gradient optimisation, which will remove balancing moves with unapropriate > estimated time from balancing series. > > > Timeline. > > Week 1: Architectural discuss. (also exams in institute) > > Week 2-3: Implement interface for time estimations in HTools. > > Week 4: Implement necessary user options. (--avoid-long-moves) > > Week 5: Catch-up week. > > Week 6-7: Implement ganeti daemon for collecting node’s network speed > data. > > Week 8-9: Add functionality to DataCollectors for get instances’ mem > usage activity. > > Week 10: Add scoring for moves based on time estimations and optimize > balancing. > > Week 11: Implementing tests. (also 3-5 vacation days of volunteer work) > > Week 12: Catch-up week. > > > About me. > > Daniil Leshchev > > Email: [email protected] > > Phone: +79105505550 > > My name is Daniil Leshchev, I’m first year undergraduate master at Moscow > Institute of Physics and Technology, Russia. My specialization is Applied > Mathematics and CS. > > I have variety experience in area of virtualization and close > technologies. I was working in research lab of Parallels company for two > years and was involved in a research project of ARM chips virtualization. > this work was the basis of my bachelor thesis. I do not dwell on low-level > programming, and have math skills, knowledge of statistics, machine > learning and optimisation algorithms. > > Also I’m diving into Linux kernel and have a patch > (/drivers/staging/rtl8723au), which has already merged to main tree (commit > 78f73d92fc2467318a377f98fe0f74e37b0a4bc1). > > > > > > -- > > Regards, > > Daniil Leshchev >
