[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread Shubham Sandeep Rastogi via lldb-commits 

https://github.com/rastogishubham created 
https://github.com/llvm/llvm-project/pull/83312

The timeout for this test was set to 1.0s which is very low, it should be a 
default of 10s and be increased by a factor of 10 if ASAN is enabled. This will 
help reduce the falkiness of the test, especially in ASAN builds.

>From 53c507046527e04b7faa70ea17cd59b45e724f55 Mon Sep 17 00:00:00 2001
From: Shubham Sandeep Rastogi 
Date: Wed, 28 Feb 2024 10:28:55 -0800
Subject: [PATCH] Increase timeout to reduce test failure rate.

The timeout for this test was set to 1.0s which is very low, it should
be a default of 10s and be increased by a factor of 10 if ASAN is
enabled. This will help reduce the falkiness of the test, especially in
ASAN builds.
---
 .../API/tools/lldb-dap/launch/TestDAP_launch.py   | 15 +--
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py 
b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
index 04d741c1d47201..635ec4f0baf190 100644
--- a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
+++ b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
@@ -44,7 +44,8 @@ def test_termination(self):
 self.dap_server.request_disconnect()
 
 # Wait until the underlying lldb-dap process dies.
-self.dap_server.process.wait(timeout=10)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+self.dap_server.process.wait(timeout=timeoutval)
 
 # Check the return code
 self.assertEqual(self.dap_server.process.poll(), 0)
@@ -334,14 +335,15 @@ def test_commands(self):
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue again and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the second breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
@@ -402,21 +404,22 @@ def test_extra_launch_commands(self):
 self.verify_commands("launchCommands", output, launchCommands)
 # Verify the "stopCommands" here
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
 self.continue_to_exit()
 # Get output from the console. This should contain both the
 # "exitCommands" that were run after the second breakpoint was hit
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("exitCommands", output, exitCommands)
 
 @skipIfWindows

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread via lldb-commits 

llvmbot wrote:




@llvm/pr-subscribers-lldb

Author: Shubham Sandeep Rastogi (rastogishubham)


Changes

The timeout for this test was set to 1.0s which is very low, it should be a 
default of 10s and be increased by a factor of 10 if ASAN is enabled. This will 
help reduce the falkiness of the test, especially in ASAN builds.

---
Full diff: https://github.com/llvm/llvm-project/pull/83312.diff


1 Files Affected:

- (modified) lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py (+9-6) 


``diff
diff --git a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py 
b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
index 04d741c1d47201..635ec4f0baf190 100644
--- a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
+++ b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
@@ -44,7 +44,8 @@ def test_termination(self):
 self.dap_server.request_disconnect()
 
 # Wait until the underlying lldb-dap process dies.
-self.dap_server.process.wait(timeout=10)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+self.dap_server.process.wait(timeout=timeoutval)
 
 # Check the return code
 self.assertEqual(self.dap_server.process.poll(), 0)
@@ -334,14 +335,15 @@ def test_commands(self):
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue again and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the second breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
@@ -402,21 +404,22 @@ def test_extra_launch_commands(self):
 self.verify_commands("launchCommands", output, launchCommands)
 # Verify the "stopCommands" here
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
 self.continue_to_exit()
 # Get output from the console. This should contain both the
 # "exitCommands" that were run after the second breakpoint was hit
-output = self.get_console(timeout=1.0)
+output = self.get_console(timeout=timeoutval)
 self.verify_commands("exitCommands", output, exitCommands)
 
 @skipIfWindows

``




https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread via lldb-commits 

github-actions[bot] wrote:




:warning: Python code formatter, darker found issues in your code. :warning:



You can test this locally with the following command:


``bash
darker --check --diff -r 
c1b8c6cf41df4a148e7a89c3a3c7e8049b0a47af...53c507046527e04b7faa70ea17cd59b45e724f55
 lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
``





View the diff from darker here.


``diff
--- TestDAP_launch.py   2024-02-28 18:40:35.00 +
+++ TestDAP_launch.py   2024-02-28 18:44:39.052136 +
@@ -42,11 +42,11 @@
 # The lldb-dap process should finish even though
 # we didn't close the communication socket explicitly
 self.dap_server.request_disconnect()
 
 # Wait until the underlying lldb-dap process dies.
-timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+timeoutval = 10 * (10 if ("ASAN_OPTIONS" in os.environ) else 1)
 self.dap_server.process.wait(timeout=timeoutval)
 
 # Check the return code
 self.assertEqual(self.dap_server.process.poll(), 0)
 
@@ -333,11 +333,11 @@
 
 # Continue after launch and hit the first breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+timeoutval = 10 * (10 if ("ASAN_OPTIONS" in os.environ) else 1)
 output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue again and hit the second breakpoint.
 # Get output from the console. This should contain both the
@@ -402,11 +402,11 @@
 # Verify all "launchCommands" were found in console output
 # After execution, program should launch
 self.verify_commands("launchCommands", output, launchCommands)
 # Verify the "stopCommands" here
 self.continue_to_next_stop()
-timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+timeoutval = 10 * (10 if ("ASAN_OPTIONS" in os.environ) else 1)
 output = self.get_console(timeout=timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue and hit the second breakpoint.
 # Get output from the console. This should contain both the

``




https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread Walter Erquinigo via lldb-commits 

https://github.com/walter-erquinigo requested changes to this pull request.

could you create instead a variable at the base test class level that can be 
used by other DAP tests when setting timeouts? I'm pretty sure at least one 
other test file uses timeouts.

https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread Walter Erquinigo via lldb-commits 

https://github.com/walter-erquinigo edited 
https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-28 Thread Shubham Sandeep Rastogi via lldb-commits 

https://github.com/rastogishubham updated 
https://github.com/llvm/llvm-project/pull/83312

>From 563ef808aa3b06e97b89e7b52d518705e97d9e14 Mon Sep 17 00:00:00 2001
From: Shubham Sandeep Rastogi 
Date: Wed, 28 Feb 2024 10:28:55 -0800
Subject: [PATCH] Increase timeout to reduce test failure rate.

The timeout for this test was set to 1.0s which is very low, it should
be a default of 10s and be increased by a factor of 10 if ASAN is
enabled. This will help reduce the falkiness of the test, especially in
ASAN builds.
---
 .../test/tools/lldb-dap/lldbdap_testcase.py  |  1 +
 .../test/API/tools/lldb-dap/launch/TestDAP_launch.py | 12 ++--
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git 
a/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py 
b/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
index 288cc8cf9a48c8..f586a28f511a45 100644
--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
@@ -6,6 +6,7 @@
 
 
 class DAPTestCaseBase(TestBase):
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
 NO_DEBUG_INFO_TESTCASE = True
 
 def create_debug_adaptor(self, lldbDAPEnv=None):
diff --git a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py 
b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
index 04d741c1d47201..0760d358d9c0b3 100644
--- a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
+++ b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
@@ -44,7 +44,7 @@ def test_termination(self):
 self.dap_server.request_disconnect()
 
 # Wait until the underlying lldb-dap process dies.
-self.dap_server.process.wait(timeout=10)
+
self.dap_server.process.wait(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 
 # Check the return code
 self.assertEqual(self.dap_server.process.poll(), 0)
@@ -334,14 +334,14 @@ def test_commands(self):
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue again and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the second breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
@@ -402,21 +402,21 @@ def test_extra_launch_commands(self):
 self.verify_commands("launchCommands", output, launchCommands)
 # Verify the "stopCommands" here
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
 self.continue_to_exit()
 # Get output from the console. This should contain both the
 # "exitCommands" that were run after the second breakpoint was hit
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("exitCommands", output, exitCommands)
 
 @skipIfWindows

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-29 Thread Walter Erquinigo via lldb-commits 

https://github.com/walter-erquinigo edited 
https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-29 Thread Walter Erquinigo via lldb-commits 

https://github.com/walter-erquinigo approved this pull request.

Other than a missing comment, this LGTM. Thanks!

https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-29 Thread Walter Erquinigo via lldb-commits 


@@ -6,6 +6,7 @@
 
 
 class DAPTestCaseBase(TestBase):
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)

walter-erquinigo wrote:

Please add a comment here mentioning what the purpose of this variable is

https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-29 Thread Shubham Sandeep Rastogi via lldb-commits 

https://github.com/rastogishubham updated 
https://github.com/llvm/llvm-project/pull/83312

>From cef723783edd5ba89fd5b9b0ec800fa1439ffb90 Mon Sep 17 00:00:00 2001
From: Shubham Sandeep Rastogi 
Date: Wed, 28 Feb 2024 10:28:55 -0800
Subject: [PATCH] Increase timeout to reduce test failure rate.

The timeout for this test was set to 1.0s which is very low, it should
be a default of 10s and be increased by a factor of 10 if ASAN is
enabled. This will help reduce the falkiness of the test, especially in
ASAN builds.
---
 .../test/tools/lldb-dap/lldbdap_testcase.py  |  3 +++
 .../test/API/tools/lldb-dap/launch/TestDAP_launch.py | 12 ++--
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git 
a/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py 
b/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
index 288cc8cf9a48c8..23f650d2d36fdd 100644
--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py
@@ -6,6 +6,9 @@
 
 
 class DAPTestCaseBase(TestBase):
+# set timeout based on whether ASAN was enabled or not. Increase
+# timeout by a factor of 10 if ASAN is enabled.
+timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
 NO_DEBUG_INFO_TESTCASE = True
 
 def create_debug_adaptor(self, lldbDAPEnv=None):
diff --git a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py 
b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
index 04d741c1d47201..0760d358d9c0b3 100644
--- a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
+++ b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
@@ -44,7 +44,7 @@ def test_termination(self):
 self.dap_server.request_disconnect()
 
 # Wait until the underlying lldb-dap process dies.
-self.dap_server.process.wait(timeout=10)
+
self.dap_server.process.wait(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 
 # Check the return code
 self.assertEqual(self.dap_server.process.poll(), 0)
@@ -334,14 +334,14 @@ def test_commands(self):
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue again and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the second breakpoint was hit
 self.continue_to_breakpoints(breakpoint_ids)
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
@@ -402,21 +402,21 @@ def test_extra_launch_commands(self):
 self.verify_commands("launchCommands", output, launchCommands)
 # Verify the "stopCommands" here
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue and hit the second breakpoint.
 # Get output from the console. This should contain both the
 # "stopCommands" that were run after the first breakpoint was hit
 self.continue_to_next_stop()
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("stopCommands", output, stopCommands)
 
 # Continue until the program exits
 self.continue_to_exit()
 # Get output from the console. This should contain both the
 # "exitCommands" that were run after the second breakpoint was hit
-output = self.get_console(timeout=1.0)
+output = 
self.get_console(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
 self.verify_commands("exitCommands", output, exitCommands)
 
 @skipIfWindows

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [lldb] Increase timeout to reduce test failure rate. (PR #83312)

2024-02-29 Thread Shubham Sandeep Rastogi via lldb-commits 

https://github.com/rastogishubham closed 
https://github.com/llvm/llvm-project/pull/83312
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits