[
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439612#comment-16439612
]
Adam Szita commented on HIVE-19077:
-----------------------------------
[~djaiswal], sorry if this caused an inconvenience for you. The goal is to
prevent wasting resources when running ptest on the same patch multiple times.
It's just pointless and causes long queues to progress slowly.
Issues standing in queue multiple times is just like when you go grocery
shopping and you're already in line at cashier but you realise you also need
e.g. milk. You go grab it and join the END of the queue. So I believe it's a
fair punishment for losing one's spot in the queue for making amends on their
patch.
I know this is not ideal - but for now it's an easy fix to speed up the queue a
little. I've been monitoring the precommit queue size for quite some time now,
and in average there's 11 entries in queue for 10 distinct ones, at some times
it is 15 to 10.
That said I also understand that for big changes like the one you're having
this makes your life very difficult. What I can propose to this is that ptest
client can look for a certain tag on / text in the Jira ticket, one which if
set, overrides this restriction (this could work similarly like the (NO
PRECOMMIT TESTS)) and everything works as it used to. The downside of this is
that it opens up a cheating opportunity but I'm hoping we're better than
exploiting such thing on normal sized patches...
Please let me know what you think.
> Handle duplicate ptests requests standing in queue at the same time
> -------------------------------------------------------------------
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
> Reporter: Adam Szita
> Assignee: Adam Szita
> Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch,
> HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I
> noticed that sometimes huge queues can build up, that contain jira's more
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now
> the current ptest infra seems to be smart enough to always deal with the
> latest patch, so what will happen is that the same patch will be tested 2
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for
> Jira X, then it can take a look on the current queue, and see if X is there
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your
> original place in the queue will be gone (like as a penalty for changing it),
> but overall it saves resources for the whole community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
