Le lundi 18 mai 2020 08:56:07 UTC+2, Mathias Herberts a écrit :
>
> 1.3M comparisons on 435k metadata usually does not take 25s, this is a
> typical log in one of our own systems:
>
> Search returned 27 results in 183.488054 ms, inspected 182304 metadatas in
> 1 classes (1 matched) and performed 546912 comparisons
>
>
> what is the load average on your Directory? What is the time a /find
> request actually takes (the path is slightly different so result may give
> some additional infos).
>
>
>
> On Friday, May 15, 2020 at 5:35:21 PM UTC+2, A. Hébert wrote:
>>
>> I can't really provides those lines with certainty... We are using HA
>> sharded directories. I could find an example of what may like a slow query:
>> Search returned 14 results in 25218.834073 ms, inspected 435067
>> metadatas in 1 classes (1 matched) and performed 1306154 comparisons.
>>
>> For information in all the sharded directory logs, I didn't found a log
>> with 14 results [ ... ] in 14 classes or with 14 matched neither.
>>
>> Le vendredi 15 mai 2020 09:45:12 UTC+2, Mathias Herberts a écrit :
>>>
>>> Can you provide the directory log lines related to the searches you are
>>> performing? They should look like:
>>>
>>> Search returned AAA results in BBB ms, inspected CCC metadatas in DDD
>>> classes (EEE matched) and performed FFF comparisons
>>>
>>>
>>>
>>> On Thursday, May 14, 2020 at 4:53:33 PM UTC+2, A. Hébert wrote:
>>>>
>>>> I do apologise, I run my tests to fast, the 'extra' parameter isn't
>>>> working with a FIND statement only with FETCH. I couldn't really reduce by
>>>> a lot using this method (from 70 - 80 seconds) to (40 - 50 seconds) to
>>>> retrieve the 14 last values of those series. I had to split my statement
>>>> in
>>>> 3 or 4 FETCH, as the labels keys are different for several classes.
>>>>
>>>> Le jeudi 14 mai 2020 16:42:02 UTC+2, A. Hébert a écrit :
>>>>>
>>>>> Thanks for your proposal. Do you have an example for the 'extra'
>>>>> parameter? I couldn't make it work (only one series as result) on a
>>>>> Warp10
>>>>> 2.5.
>>>>> As a matter of fact this time I am really in the second case (I can't
>>>>> know the full set of labels only the id) and I really would like to be
>>>>> able
>>>>> to reduce the 10 seconds to FIND a single series.
>>>>>
>>>>> Le mardi 28 avril 2020 21:34:03 UTC+2, Mathias Herberts a écrit :
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> assuming you know the classes of those 14 series you want to access,
>>>>>> you could very well use the same approach using the 'gts' parameter with
>>>>>> a
>>>>>> list of GTS you build manually, i.e.:
>>>>>>
>>>>>> 'gts' [
>>>>>> NEWGTS 'class-1' RENAME { 'id' '42' } RELABEL
>>>>>> NEWGTS 'class-2' RENAME { 'id' '42' } RELABEL
>>>>>> ..
>>>>>> ]
>>>>>>
>>>>>> Of course you need to specify the full set of labels so this approach
>>>>>> will only work if you know what the labels are.
>>>>>>
>>>>>> Another approach, assuming you do not know what the labels are but
>>>>>> you know the class names, you could select the GTS using a single class
>>>>>> name then use the 'extra' parameter to specify the other 13 classes.
>>>>>>
>>>>>> On Monday, April 27, 2020 at 11:54:41 AM UTC+2, A. Hébert wrote:
>>>>>>>
>>>>>>> Hello Mathias,
>>>>>>>
>>>>>>> I am back again on this story of performance for queries due to the
>>>>>>> Index size.
>>>>>>>
>>>>>>> First of all, I wanted to say that in many use-cases the FETCH with
>>>>>>> the "gts" key improve a lot of query in our side.
>>>>>>>
>>>>>>> However we still have massive slow queries, for example when we want
>>>>>>> to compute a FIND on a specific label on one of our most large
>>>>>>> application.
>>>>>>> The selector query is '~.*{id=42}' and this queries takes more than 70
>>>>>>> seconds (server side). The FINDSTATS provides the following result
>>>>>>> "{"classes.estimate":14,"gts.estimate":14,"labelnames.estimate":13,"labelvalues.estimate":12,"error.rate":0.008125}".
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Even when setting one series name on the selector 'test{id=42}', we
>>>>>>> still get series than takes more than 10 seconds to be executed (server
>>>>>>> side).
>>>>>>>
>>>>>>> I would be to help to improve those queries. What can I look like to
>>>>>>> start with?
>>>>>>>
>>>>>>> Le mercredi 11 mars 2020 17:01:50 UTC+1, A. Hébert a écrit :
>>>>>>>>
>>>>>>>> For sure on a batch of hundred queries an average of 150ms, the
>>>>>>>> best resolved in only 28ms, and the worst in 1s
>>>>>>>>
>>>>>>>> Le mercredi 11 mars 2020 16:52:34 UTC+1, Mathias Herberts a écrit :
>>>>>>>>>
>>>>>>>>> Did you see an improvement in terms of performance?
>>>>>>>>>
>>>>>>>>
--
You received this message because you are subscribed to the Google Groups "Warp
10 users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/warp10-users/06c0ede6-709b-4d97-839e-a565802aeea7%40googlegroups.com.
