Skip to content

tools: show the full test output, on known flaky test failure with v test folder/ - #22716

Merged
spytheman merged 1 commit into
vlang:masterfrom
spytheman:show_full_flaky_test_output_on_failures
Oct 31, 2024
Merged

spytheman merged 1 commit into
vlang:masterfrom
spytheman:show_full_flaky_test_output_on_failures

Conversation

@spytheman

@spytheman spytheman commented Oct 31, 2024 •

Copy link
Copy Markdown
Contributor

Make diagnosing flaky tests easier, on the CI and locally, without having to run them manually first.

It now shows all the lines below starting with >>>:

                 retrying 1/3 of vlib/x/vweb/tests/vweb_should_listen_on_both_ipv4_and_ipv6_by_default_test.v ; known flaky: true ...
                 retrying 2/3 of vlib/x/vweb/tests/vweb_should_listen_on_both_ipv4_and_ipv6_by_default_test.v ; known flaky: true ...
                 retrying 3/3 of vlib/x/vweb/tests/vweb_should_listen_on_both_ipv4_and_ipv6_by_default_test.v ; known flaky: true ...
>>> flaky failures so far:
>>>>>> ---------------------------------------------------------------------------------------------------- retry: 0 ; max_retry: 3 ; r.exit_code: 1 ; trimmed_output.len: 1149
>>>>>> 2024-10-31T16:19:23.728582Z [DEBUG] testsuite_begin
>>>>>> 2024-10-31T16:19:23.728601Z [INFO ] working curl_executable found at: /usr/local/bin/curl
>>>>>> 2024-10-31T16:19:23.728604Z [DEBUG] starting watchdog thread to ensure the test will always exit one way or another...
>>>>>> 2024-10-31T16:19:23.728838Z [DEBUG] starting webserver...
>>>>>> [Vweb] Running app on http://localhost:23013/
>>>>>> 2024-10-31T16:19:23.735121Z [DEBUG] webserver started
>>>>>> 2024-10-31T16:19:23.748160Z [INFO ] > test_curl_connecting_through_ipv4_works
>>>>>> 2024-10-31T16:19:23.748577Z [INFO ] > test_net_http_connecting_through_ipv4_works
>>>>>> 2024-10-31T16:19:23.765041Z [INFO ] > test_curl_connecting_through_ipv6_works
>>>>>> /space/v/oo/vlib/x/vweb/tests/vweb_should_listen_on_both_ipv4_and_ipv6_by_default_test.v:86: ✗ fn test_net_http_connecting_through_ipv6_works failed propagation with error: dial_tcp failed for address ::1:23013
>>>>>> tried addrs:
>>>>>>  0.0.0.0:0: net: socket error: 111; code: 111
>>>>>> 
>>>>>>    86 |  res := http.get('http://[::1]:${port}/')!
>>>>>> 2024-10-31T16:19:23.765479Z [DEBUG] testsuite_end
>>>>>> ---------------------------------------------------------------------------------------------------- retry: 1 ; max_retry: 3 ; r.exit_code: 1 ; trimmed_output.len: 1149
>>>>>> 2024-10-31T16:19:23.786566Z [DEBUG] testsuite_begin
>>>>>> 2024-10-31T16:19:23.786598Z [INFO ] working curl_executable found at: /usr/local/bin/curl
>>>>>> 2024-10-31T16:19:23.786602Z [DEBUG] starting watchdog thread to ensure the test will always exit one way or another...

etc...

Huly®: V_0.6-21163

@spytheman
spytheman merged commit d48cb18 into vlang:master Oct 31, 2024
@spytheman
spytheman deleted the show_full_flaky_test_output_on_failures branch October 31, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant