$ ./profile-stats -c myscript.prof Wed Nov 25 10:40:11 2015 myscript.prof
7892315 function calls (7891054 primitive calls) in 7.940 seconds Ordered by: internal time List reduced from 1518 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 9086 2.693 0.000 6.706 0.001 inspect.py:247(getmembers) 1952494 1.394 0.000 1.880 0.000 inspect.py:59(isclass) 9092 1.030 0.000 1.030 0.000 {dir} 1952642 0.600 0.000 0.600 0.000 {getattr} 1972765 0.504 0.000 0.504 0.000 {isinstance} 3 0.334 0.111 0.334 0.111 {method 'perform' of 'pycurl.Curl' objects} 1883918 0.284 0.000 0.284 0.000 {method 'append' of 'list' objects} 9087 0.221 0.000 0.221 0.000 {method 'sort' of 'list' objects} 9051 0.172 0.000 6.911 0.001 reflectionhelper.py:51(isModuleMember) 1 0.124 0.124 0.354 0.354 errors.py:17(<module>) 1 0.088 0.088 0.230 0.230 params.py:8(<module>) 8879 0.070 0.000 6.998 0.001 params.py:367(__setattr__) 1 0.047 0.047 5.182 5.182 api.py:23(<module>) 1 0.025 0.025 4.743 4.743 brokers.py:22(<module>) 1 0.023 0.023 0.030 0.030 connectionspool.py:17(<module>) 1 0.022 0.022 0.053 0.053 lxml.etree.pyx:1(PyMODINIT_FUNC PyInit_etree(void)) 118 0.019 0.000 4.684 0.040 params.py:45277(__init__) 5 0.015 0.003 0.024 0.005 {built-in method strptime} 1 0.012 0.012 0.013 0.013 socket.py:45(<module>) 10 0.011 0.001 0.015 0.002 collections.py:288(namedtuple) So it is not the classes, it is the code inspecting them on import. Nir On Wed, Nov 25, 2015 at 7:49 AM, John Hunter <zhjw...@gmail.com> wrote: > Hi Nir, > > Attachment is my script and its profile. > Thanks a lot about your help! > > On Wed, Nov 25, 2015 at 5:49 AM, Nir Soffer <nsof...@redhat.com> wrote: > >> On Tue, Nov 24, 2015 at 3:49 PM, John Hunter <zhjw...@gmail.com> wrote: >> >>> >>> >>> On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <jhern...@redhat.com> >>> wrote: >>> >>>> On 11/24/2015 01:40 PM, John Hunter wrote: >>>> > Hi, >>>> > >>>> > On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <oourf...@redhat.com >>>> > <mailto:oourf...@redhat.com>> wrote: >>>> > >>>> > Hi >>>> > >>>> > I discussed it with Juan (cc-ed). >>>> > >>>> > There used to be a bug in the JDBC authenticion extension that >>>> > artificially delayed RESTAPI responses by 5 seconds: >>>> > >>>> > brute force prevention login delay should not be applied to >>>> successful >>>> > login requests >>>> > https://bugzilla.redhat.com/1255814 >>>> > >>>> > That matches the description of the issue, but in theory it has >>>> been >>>> > fixed. I would suggest him to check that he is using the right >>>> version >>>> > of the extension. >>>> > >>>> > I did not use the extension ovirt-engine-extension-aaa-jdbc, and I >>>> don't >>>> > think this bug matches my problem, because even there is only one line >>>> > in the python script, it still cost like 3 seconds, I don't think >>>> this is a >>>> > reasonable time as when I import other package, it cost almost no >>>> time. >>>> > >>>> > Can you explain why this import line costs so much time? >>>> > >>>> >>>> If you are using 3.6 then you are using ovirt-engine-extension-aaa-jdbc, >>>> as it is enabled by default, but looks like it isn't related to your >>>> problem. >>>> >>>> That line takes a long time to execute because it has to process two >>>> large Python modules: the "params" module that contains a class per each >>>> type used by the API (393 classes) and the "brokers" module that >>>> contains a class per each resource used by the API (358 classes). That >>>> makes a total of 751 classes. In my environment it takes 0.9 seconds, >>>> approx. You may want to use the python profile in your environment and >>>> share the results: >>>> >>>> $ cat > profile.py <<. >>>> import cProfile >>>> cProfile.run("from ovirtsdk.api import API") >>>> . >>>> >>>> $ python profile.py >>>> >>>> I won't be surprised to see this taking those 3 seconds in a slower >>>> environment. >>>> >>>> But even if this takes those 3 seconds it shouldn't be a big problem, >>>> because you shouldn't be running that "from ... import ..." line >>>> frequently. Your program should do this once only, when it starts. >>>> >>>> Assume that I have two functions to implement, one is to list all the >>> vms belong >>> to the user, and the other is to retrieve one vm's virt-viewer >>> connection file, as >>> far as I can see, I have to write two python scripts and import the >>> ovirtsdk.api in both >>> scripts, each script has to take the 3 seconds :( >>> >> >> No, you have two functions, which can be in the same module, or in >> different modules, depending on how you want to organize your code. >> >> Python imports are done only once, on the first time you import a module >> anywhere. The module is stored in sys.modules, and the next import fetch >> the module object from sys.modules. >> >> Can you share a profile of a simple script importing the Python sdk? >> >> Do: >> >> python -m cProfile -o myscript.prof myscript.py >> >> The profile will be stored in myscript.prof. >> >> For viewing the profile, I recommend to use the profile-stats tool: >> https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats >> >> >>> >>> How can I run the "from ... import ..." just once ? >>> >>> >>>> > >>>> > In addition we also know that retrieving large lists of objects >>>> with the >>>> > SDK is slow: >>>> > >>>> > [RFE][performance] - generate large scale list running to slow. >>>> > https://bugzilla.redhat.com/1221238 >>>> > >>>> > We don't have a solution for that yet. >>>> > >>>> > CC-ing Juan in case you have additional questions. >>>> > >>>> > >>>> > On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <zhjw...@gmail.com >>>> > <mailto:zhjw...@gmail.com>> wrote: >>>> > >>>> > Hi guys, >>>> > >>>> > I am using the ovirt-engine-sdk-python to communicate with the >>>> > ovirt-engine, >>>> > I am ok to list the vms but the processing time is too long, >>>> > like 4.5 seconds, >>>> > and this line: >>>> > from ovirtsdk.api import API >>>> > take almost 3 seconds. >>>> > >>>> > This seems a little bit longer than I expected it to be, so I >>>> am >>>> > asking is there >>>> > a quicker way to communicate with the ovirt-engine? >>>> > >>>> >>>> -- >>>> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta >>>> 3ºD, 28016 Madrid, Spain >>>> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat >>>> S.L. >>>> >>> >>> >>> >>> -- >>> Best regards >>> Junwang Zhao >>> Department of Computer Science &Technology >>> Peking University >>> Beijing, 100871, PRC >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > > > -- > Best regards > Junwang Zhao > Department of Computer Science &Technology > Peking University > Beijing, 100871, PRC >
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel