[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13847064#comment-13847064 ] Ted Yu commented on HADOOP-10163: - I will try to digest the above code. However, for JIRA with long discussion, e.g. HBASE-8755, searching comments for attachment Id seems inefficient. > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846852#comment-13846852
]
Brock Noland commented on HADOOP-10163:
---
also later we match the file name.
{noformat}
# validate the patch name, parse branch if needed
shopt -s nocasematch
PATCH_NAME=$(basename $PATCH_URL)
# Test examples:
# HIVE-123.patch HIVE-123.1.patch HIVE-123.D123.patch HIVE-123.D123.1.patch
HIVE-123-tez.patch HIVE-123.1-tez.patch
# HIVE-.patch, HIVE-.XX.patch HIVE-.XX-branch.patch
HIVE--branch.patch
if [[ $PATCH_NAME =~ ^HIVE-[0-9]+(\.[0-9]+)?(-[a-z0-9-]+)?\.(patch|patch.\txt)$
]]
then
if [[ -n "${BASH_REMATCH[2]}" ]]
then
BRANCH=${BASH_REMATCH[2]#*-}
else
echo "Assuming branch $BRANCH"
fi
# HIVE-.D.patch or HIVE-.D.XX.patch
elif [[ $PATCH_NAME =~ ^(HIVE-[0-9]+\.)?D[0-9]+(\.[0-9]+)?\.(patch|patch.\txt)$
]]
then
echo "Assuming branch $BRANCH"
else
echo "Patch $PATCH_NAME does not appear to be a patch"
exit 1
fi
{noformat}
> Attachment Id for last tested patch should be passed to test-patch.sh
> -
>
> Key: HADOOP-10163
> URL: https://issues.apache.org/jira/browse/HADOOP-10163
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> In HBASE-10044, attempt was made to filter attachments according to known
> file extensions.
> However, that change alone wouldn't work because when non-patch is attached,
> QA bot doesn't provide attachment Id for last tested patch.
> This results in the modified test-patch.sh to seek backward and launch
> duplicate test run for last tested patch.
> If attachment Id for last tested patch is provided, test-patch.sh can decide
> whether there is need to run test.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846844#comment-13846844
]
Brock Noland commented on HADOOP-10163:
---
Here is what we do with hive where $JIRA_TEXT is a file containing the entire
HTML page:
{noformat}
# pull attachments from JIRA (hack stolen from hadoop since rest api doesn't
show attachments)
PATCH_URL=$(grep -o '"/jira/secure/attachment/[0-9]*/[^"]*' $JIRA_TEXT | \
grep -v -e 'htm[l]*$' | sort | tail -1 | \
grep -o '/jira/secure/attachment/[0-9]*/[^"]*')
if [[ -z "$PATCH_URL" ]]
then
echo "Unable to find attachment for $JIRA_NAME"
exit 1
fi
# ensure attachment has not already been tested
ATTACHMENT_ID=$(basename $(dirname $PATCH_URL))
if grep -q "ATTACHMENT ID: $ATTACHMENT_ID" $JIRA_TEXT
then
echo "Attachment $ATTACHMENT_ID is already tested for $JIRA_NAME"
exit 1
fi
{noformat}
> Attachment Id for last tested patch should be passed to test-patch.sh
> -
>
> Key: HADOOP-10163
> URL: https://issues.apache.org/jira/browse/HADOOP-10163
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> In HBASE-10044, attempt was made to filter attachments according to known
> file extensions.
> However, that change alone wouldn't work because when non-patch is attached,
> QA bot doesn't provide attachment Id for last tested patch.
> This results in the modified test-patch.sh to seek backward and launch
> duplicate test run for last tested patch.
> If attachment Id for last tested patch is provided, test-patch.sh can decide
> whether there is need to run test.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846820#comment-13846820 ] Ted Yu commented on HADOOP-10163: - The above scenario actually may happen to Hive: Patch A was tested on day 1. B.htm is attached to same JIRA on day 2. test-patch.sh would filter out B.htm and run patch A again. > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846765#comment-13846765
]
Ted Yu commented on HADOOP-10163:
-
Here is proposed change from HBASE-10044 :
{code}
+relativePatchURL=`$GREP -o '"/jira/secure/attachment/[0-9]*/[^"]*'
$PATCH_DIR/jira | $EGREP '(\.txt$|\.patch$|\.diff$)' | sort | tail -1 | $GREP
-o '/jira/secure/attachment/[0-9]*/[^"]*'`
{code}
Suppose patch A was tested on day 1 - JIRA stayed in Patch Available status.
Some more comments were then added.
Tread dump named thread.out was attached.
QA bot would be triggered due to the new attachment. However the above change
would filter out the dump file and go back to patch A which has been run.
The attachment ID of last patch should be known so that test-patch.sh can
decide that no new patch has been attached and bail out.
> Attachment Id for last tested patch should be passed to test-patch.sh
> -
>
> Key: HADOOP-10163
> URL: https://issues.apache.org/jira/browse/HADOOP-10163
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> In HBASE-10044, attempt was made to filter attachments according to known
> file extensions.
> However, that change alone wouldn't work because when non-patch is attached,
> QA bot doesn't provide attachment Id for last tested patch.
> This results in the modified test-patch.sh to seek backward and launch
> duplicate test run for last tested patch.
> If attachment Id for last tested patch is provided, test-patch.sh can decide
> whether there is need to run test.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846759#comment-13846759 ] Brock Noland commented on HADOOP-10163: --- We don't iterate jira comments, it's a simple (jira text contains "ATTACHMENT ID XXX"). Not sure if this holds for other projects but in hive a specific attachment is only tested once. > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846646#comment-13846646 ] Ted Yu commented on HADOOP-10163: - Do you have sample code on how to iterate through JIRA comments ? Thanks > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846634#comment-13846634 ] Brock Noland commented on HADOOP-10163: --- Yep > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846585#comment-13846585 ] Brock Noland commented on HADOOP-10163: --- We handled this scenario in Hive by placing "ATTACHMENT ID " in the HiveQA comment: https://issues.apache.org/jira/browse/HIVE-5973?focusedCommentId=13846235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13846235 If Hadoop/HBase did that then they could eliminate duplicate runs as we have done in Hive. > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10163) Attachment Id for last tested patch should be passed to test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846592#comment-13846592 ] Ted Yu commented on HADOOP-10163: - That approach would imply scanning comments until "ATTACHMENT ID " is found, right ? > Attachment Id for last tested patch should be passed to test-patch.sh > - > > Key: HADOOP-10163 > URL: https://issues.apache.org/jira/browse/HADOOP-10163 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ted Yu > > In HBASE-10044, attempt was made to filter attachments according to known > file extensions. > However, that change alone wouldn't work because when non-patch is attached, > QA bot doesn't provide attachment Id for last tested patch. > This results in the modified test-patch.sh to seek backward and launch > duplicate test run for last tested patch. > If attachment Id for last tested patch is provided, test-patch.sh can decide > whether there is need to run test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
