[GSoC] Summery of pg performance farm
Hello monters and hackers, This is a late summary of pg performance farm in gsoc. Although it is not yet perfect, but it has began to take shape. 1. The implementation of the basic test results upload interface to ensure that the upload operation for the atomic level (including different client numbers in different modes (rw, ro) corresponding to the results of the value). 2. Implementation of the data report related to the list page. Compare each metrics whith the previous results. If any of the metrics are a 5% improvement( or regression), there is one aspect that is progressive (or regressive). This means there may be aspects of "improvement", "regression" and "status quo" in one test result.This is the report List page for an example: http://140.211.168.111/#/machineInfo/pmJEjJjSk3WREM3Q 3.The details page of the test report is implemented. The test results in the "read only" and "read & write" mode of this report are detailed in this page, as well as the link of previous report and other details. Here is an example: http://140.211.168.111/#/detailInfo/KScN58FCUWRD2fuC2R7VeY 4. A simple user center was implemented to allow users to browse the owning machine and apply for a new machine (I originally planned to access the community's user information interface at logon verification, but this seems to require some waiting...So this part is temporarily not present.) 5. Implement the action of approvaling machine and sending notification email in django admin. We plan to add some useful features and write test cases to pg performance farm in the future. If anyone has any good ideas or opinions. Please feel free to email my mentors and me. And You are also welcome to leave a message on this issue page: https://github.com/PGPerfFarm/pgperffarm/issues Best Regards, Hongyuan Ma
[GSoC] Summery of pg performance farm
Hello hackers, This is a late summary of pg performance farm in gsoc. Although it is not yet perfect, but it has began to take shape. 1. The implementation of the basic test results upload interface to ensure that the upload operation for the atomic level (including different client numbers in different modes (rw, ro) corresponding to the results of the value). 2. Implementation of the data report related to the list page. Compare each metrics whith the previous results. If any of the metrics are a 5% improvement( or regression), there is one aspect that is progressive (or regressive). This means there may be aspects of "improvement", "regression" and "status quo" in one test result.This is the report List page for an example: http://140.211.168.111/#/machineInfo/pmJEjJjSk3WREM3Q 3.The details page of the test report is implemented. The test results in the "read only" and "read & write" mode of this report are detailed in this page, as well as the link of previous report and other details. Here is an example: http://140.211.168.111/#/detailInfo/KScN58FCUWRD2fuC2R7VeY 4. A simple user center was implemented to allow users to browse the owning machine and apply for a new machine (I originally planned to access the community's user information interface at logon verification, but this seems to require some waiting...So this part is temporarily not present.) 5. Implement the action of approvaling machine and sending notification email in django admin. We plan to add some useful features and write test cases to pg performance farm in the future. If anyone has any good ideas or opinions. Please feel free to email my mentors (cc'ed) and me. And You are also welcome to leave a message on this issue page: https://github.com/PGPerfFarm/pgperffarm/issues Best Regards, Hongyuan Ma
[GSOC 18] Perf Farm——Description of patch
Hi, This email is a description of 0001-add-apps-directory.patch. In this patch, I created the apps directory and created two applications, test_item and user_operation. The test_item application model classes are: TestBranch, TestItem The model classes in the user_operation application are: Alias, UserMachine, TestResult The relationship between User and UserMachine is 1 : m The relationship between TestResult and UserMachine is: n : 1 The relationship between TestItem and TestResult is 1 : m I also created the db_tools directory. The scripts in this directory make it easy for developers to import simulation data during development. Here are some of my ideas: Since django-rest-framework does not support django 1.8, I am anxious that in the next patch, I can upgrade the Django version to the TLS version, which is now 1.11. I'm not sure if there is an existing alias database. If not, maybe I should write a web crawler that collects aliases (such as plants‘ names) from web pages (https://en.wikipedia.org/wiki/List_of_garden_plants). In addition, can you please provide PGAUTH_REDIRECT and PGAUTH_KEY constants for use in development. I plan to change the return type of the auth module to json format.
[PATCH] add apps directory; add model classes; add db_tools directory;
--- .gitignore | 2 ++ web/apps/__init__.py| 0 web/apps/test_item/__init__.py | 0 web/apps/test_item/admin.py | 3 ++ web/apps/test_item/models.py| 35 +++ web/apps/test_item/tests.py | 3 ++ web/apps/test_item/views.py | 3 ++ web/apps/user_operation/__init__.py | 0 web/apps/user_operation/admin.py| 3 ++ web/apps/user_operation/models.py | 58 web/apps/user_operation/tests.py| 3 ++ web/apps/user_operation/views.py| 3 ++ web/db.sqlite3 | Bin 0 -> 50176 bytes web/db_tools/data/__init__.py | 0 web/db_tools/data/test_branch_date.py | 11 ++ web/db_tools/import_test_branch_data.py | 22 web/pgperffarm/settings.py | 8 - web/pgperffarm/urls.py | 2 +- 18 files changed, 154 insertions(+), 2 deletions(-) create mode 100644 web/apps/__init__.py create mode 100644 web/apps/test_item/__init__.py create mode 100644 web/apps/test_item/admin.py create mode 100644 web/apps/test_item/models.py create mode 100644 web/apps/test_item/tests.py create mode 100644 web/apps/test_item/views.py create mode 100644 web/apps/user_operation/__init__.py create mode 100644 web/apps/user_operation/admin.py create mode 100644 web/apps/user_operation/models.py create mode 100644 web/apps/user_operation/tests.py create mode 100644 web/apps/user_operation/views.py create mode 100644 web/db.sqlite3 create mode 100644 web/db_tools/data/__init__.py create mode 100644 web/db_tools/data/test_branch_date.py create mode 100644 web/db_tools/import_test_branch_data.py diff --git a/.gitignore b/.gitignore index 239c691..f6dc301 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,5 @@ migrations *.psp .DS_Store tags + +.idea \ No newline at end of file diff --git a/web/apps/__init__.py b/web/apps/__init__.py new file mode 100644 index 000..e69de29 diff --git a/web/apps/test_item/__init__.py b/web/apps/test_item/__init__.py new file mode 100644 index 000..e69de29 diff --git a/web/apps/test_item/admin.py b/web/apps/test_item/admin.py new file mode 100644 index 000..8c38f3f --- /dev/null +++ b/web/apps/test_item/admin.py @@ -0,0 +1,3 @@ +from django.contrib import admin + +# Register your models here. diff --git a/web/apps/test_item/models.py b/web/apps/test_item/models.py new file mode 100644 index 000..428dc0b --- /dev/null +++ b/web/apps/test_item/models.py @@ -0,0 +1,35 @@ +from datetime import datetime + +from django.db import models + +# Create your models here. +class TestBranch(models.Model): +""" +test brand +""" +branch_name = models.CharField(max_length=128, verbose_name="branch name", help_text="branch name") +add_time = models.DateTimeField(default=datetime.now, verbose_name="branch added time", help_text="branch added time") + +class Meta: +verbose_name = "test branch" +verbose_name_plural = "test branch" + +def __str__(self): +return self.branch_name + +class TestItem(models.Model): +""" +test_item +""" +test_name = models.CharField(max_length=128, verbose_name="test name", help_text="test name") +test_desc = models.TextField(verbose_name="test desc", help_text="test desc") +test_metric = models.DecimalField(max_digits=10, decimal_places=2, verbose_name="test metric", help_text="score>metric means improving;score<=metric means declining;") +test_branch = models.ForeignKey(TestBranch ,verbose_name="test branch", help_text="test branch") +add_time = models.DateTimeField(default=datetime.now, verbose_name="test added time") + +class Meta: +verbose_name = "test_item" +verbose_name_plural = "test_item" + +def __str__(self): +return self.test_name diff --git a/web/apps/test_item/tests.py b/web/apps/test_item/tests.py new file mode 100644 index 000..a6019d6 --- /dev/null +++ b/web/apps/test_item/tests.py @@ -0,0 +1,3 @@ +from django.test import TestCase + +# Create your test_item here. diff --git a/web/apps/test_item/views.py b/web/apps/test_item/views.py new file mode 100644 index 000..91ea44a --- /dev/null +++ b/web/apps/test_item/views.py @@ -0,0 +1,3 @@ +from django.shortcuts import render + +# Create your views here. diff --git a/web/apps/user_operation/__init__.py b/web/apps/user_operation/__init__.py new file mode 100644 index 000..e69de29 diff --git a/web/apps/user_operation/admin.py b/web/apps/user_operation/admin.py new file mode 100644 index 000..8c38f3f --- /dev/null +++ b/web/apps/user_operation/admin.py @@ -0,0 +1,3 @@ +from django.contrib import admin + +# Register your models here. diff --git a/web/apps/user_operation/models.py b/web/apps/user_operation/models.py new file mode 100644 index 000..ca426f1 --- /dev/null +++ b/web/apps/user_operati
Re:Re: Re: [GSOC 18] Performance Farm Project——Initialization Project
Hi Dave, I am willing to use React to re-create the front-end application. Since I plan to use separate front-end and back-end development methods, this means that there will be no html files in Django applications. Front-end and back-end applications will interact via restful api. I will use react to rewrite some existing html files if needed. Before initializing the react application, I want to learn more about your tendency toward front-end technology. - About React: Which version of React (15.x or 16) and react-router (2.x or 4) do you tend to use? - About Package Manager: Do you tend to use yarn or npm? - About the UI library: I guess you might prefer Bootstrap or Material-UI. At the same time I hope you can help me understand the functional requirements of this project more clearly. Here are some of my thoughts on PerfFarm: - I see this comment in the client folder (In line 15 of the "pgperffarm \ client \ benchmarks \ pgbench.py" file): ''' # TODO allow running custom scripts, not just the default ''' Will PerfFarm have many test items so that the test item search function or even the test item sort function is expected to be provided? - What value will be used to determine if a machine's performance is improving or declining? - After the user logs in to the site, I think it may be necessary to provide a page for the user to browse or manage his own machine. Is there any function that requires users to log in before they can use it? - When I registered a machine on BuildFarm, I did not receive the confirmation email immediately but several days later. In PerfFarm, when a user registers a machine, will the administrator review it before sending a confirmation email? - I see BuildFarm assigning an animal name to each registered machine. Will PerfFarm also have this interesting feature? My postgresql.org community account is: - Username: maleicacid - Email: cs_maleica...@163.com I hope to get the commit permission of the pgperffarm.git repository. I am willing to continue coding and complete the project on the existing code.Looking forward to your reply. Best Regards, Hongyuan Ma (cs_maleica...@163.com) At 2018-03-13 03:03:08, "Dave Page" wrote: Hi On Mon, Mar 12, 2018 at 9:57 AM, Hongyuan Ma wrote: Hi Dave, Thank you for your reminder. This is indeed my negligence. In fact, I have browsed the code in the pgperffarm.git repository, including the client folder and the web folder. However, I found that the web folder has some unfinished html files and does not contain model class files. And the project used Django 1.8 without importing the Django REST Framework (When I talked to Mark about the PerfFarm project, he told me he insisted on using Django 1.11 and considered using the REST framework). So I mistakenly thought that the code in the web folder had been shelved. Nope, not at all. It just wasn't updated to meet the latest PG infrastructure requirements yet (basically just the update to Django 11). The rest should be fine as-is. In the newly initialized Django application, I upgraded its version and assigned the directory to make the project structure clearer. I want to use a separate front-end development approach (The front-end applications will use vue.js for programming.). I hope you can allow me to use it instead of the old one. I am willing to integrate the auth module into this new application as soon as possible. I would much prefer to see jQuery + React, purely because there are likely more PostgreSQL developers (particularly from the pgAdmin team) that know those technologies. It is important to consider long term maintenance as well as ease of initial development with any project. Thanks. Regards, Hongyuan Ma (cs_maleica...@163.com) 在 2018-03-12 02:26:43，"Dave Page" 写道： Hi Maybe I’m missing something (I’ve been offline a lot recently for unavoidable reasons), but the perf farm project already has a Django backend initialised and configured to work with community auth, on community infrastructure. https://git.postgresql.org/gitweb/?p=pgperffarm.git;a=summary On Sunday, March 11, 2018, Hongyuan Ma wrote: Hello, mark I initialized a Django project and imported the Django REST Framework. Its github address is: https://github.com/PGPerfFarm/server-code I created some model classes and also wrote scripts in the dbtools folder to import simulation data for development. I am hesitant to use admin or xadmin as the administrator's backend for the site (I prefer to use xadmin). I also initialized the website's front-end application. Here is its address: https://github.com/PGPerfFarm/front-end-code.git I wrote the header component as shown: I hope this effect can enhance the future user experience:). This application uses vue.js and element-ui, but if you insist on using other js libraries or not using js, please let me know.
Re:Google Summer of Code: Potential Applicant
Hi, Maybe you can try searching for the name of the tutor for the item you are interested in on this page below. https://www.postgresql.org/list/pgsql-hackers/ Then you can find the email address of the tutor you want to contact, and then share your thoughts with him. It is exactly what I have done. Hope it can help you. Regards, Hongyuan Ma (cs_maleica...@163.com) At 2018-03-12 21:48:21, "Christos Maris" wrote: Hey there, My name is Christos (Chris for short) and I would love to work with you via the GSoC program this summer. I am sending this mail because I am in a need for some instructions on how to find a mentor and a project to work on. Can anyone help me with that? Also is there any potential mentor here? Any info helps. Thanks a lot in advance!
Re:Re: [GSOC 18] Performance Farm Project——Initialization Project
Hi Dave, Thank you for your reminder. This is indeed my negligence. In fact, I have browsed the code in the pgperffarm.git repository, including the client folder and the web folder. However, I found that the web folder has some unfinished html files and does not contain model class files. And the project used Django 1.8 without importing the Django REST Framework (When I talked to Mark about the PerfFarm project, he told me he insisted on using Django 1.11 and considered using the REST framework). So I mistakenly thought that the code in the web folder had been shelved. In the newly initialized Django application, I upgraded its version and assigned the directory to make the project structure clearer. I want to use a separate front-end development approach (The front-end applications will use vue.js for programming.). I hope you can allow me to use it instead of the old one. I am willing to integrate the auth module into this new application as soon as possible. Regards, Hongyuan Ma (cs_maleica...@163.com) 在 2018-03-12 02:26:43，"Dave Page" 写道： Hi Maybe I’m missing something (I’ve been offline a lot recently for unavoidable reasons), but the perf farm project already has a Django backend initialised and configured to work with community auth, on community infrastructure. https://git.postgresql.org/gitweb/?p=pgperffarm.git;a=summary On Sunday, March 11, 2018, Hongyuan Ma wrote: Hello, mark I initialized a Django project and imported the Django REST Framework. Its github address is: https://github.com/PGPerfFarm/server-code I created some model classes and also wrote scripts in the dbtools folder to import simulation data for development. I am hesitant to use admin or xadmin as the administrator's backend for the site (I prefer to use xadmin). I also initialized the website's front-end application. Here is its address: https://github.com/PGPerfFarm/front-end-code.git I wrote the header component as shown: I hope this effect can enhance the future user experience:). This application uses vue.js and element-ui, but if you insist on using other js libraries or not using js, please let me know. I will empty this project and then rewrite it as you wish. My next step is to determine the necessary api interface and the corresponding components of the front-end application. Then implement them one by one. Finally, as you can see, I created an organization named PGPerfFarm on github without permission. I sent you an invitation letter, and after you join, I am willing to hand over the administrator of the organization to you. Regards, Hongyuan Ma (cs_maleica...@163.com) -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
[GSOC 18] Performance Farm Project——Initialization Project
Hello, mark I initialized a Django project and imported the Django REST Framework. Its github address is: https://github.com/PGPerfFarm/server-code I created some model classes and also wrote scripts in the dbtools folder to import simulation data for development. I am hesitant to use admin or xadmin as the administrator's backend for the site (I prefer to use xadmin). I also initialized the website's front-end application. Here is its address: https://github.com/PGPerfFarm/front-end-code.git I wrote the header component as shown: I hope this effect can enhance the future user experience:). This application uses vue.js and element-ui, but if you insist on using other js libraries or not using js, please let me know. I will empty this project and then rewrite it as you wish. My next step is to determine the necessary api interface and the corresponding components of the front-end application. Then implement them one by one. Finally, as you can see, I created an organization named PGPerfFarm on github without permission. I sent you an invitation letter, and after you join, I am willing to hand over the administrator of the organization to you. Regards, Hongyuan Ma (cs_maleica...@163.com)
Re:Re: [GSOC 18] Performance Farm Project
Hi Robert, Thanks for your advice! I agree very much with you. It is necessary to analyze and determine its functional requirements before developing a website. Hopefully in the exchange with Mr. Wong, we can discuss the function of this website as detail as possible. I see the Build Farm project seems to use perl script to upload data to the site. I guess the Performance Farm project may also use a similar script (Although I am not familiar with perl, I would like to learn it if needed). I think that after discussing the functional requirements, we can gradually determine the data table structure, data format and web interface. In my imagination, the perl script just needs to send the request in the format of the web interface, and the web application can receive the data it sends. Last but not least, I promise that if I write the code for this project, I will only use the famous frameworks and libraries. I will not look for semi-finished products and then modify or improve it. Regards, Hongyuan Ma (cs_maleica...@163.com)
