1916662 - Run Marionette tests for TSAN builds as well

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, enhancement)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Right now Marionette tests are only run for ASAN builds but miss TSAN. By inspecting the YAML definition I cannot see why:
https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/marionette.yml

Maybe it requires some setting outside of that specific file?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

these tests timeout after 90 minutes:
https://treeherder.mozilla.org/jobs?repo=try&revision=5a60e3051b14a49884920af7f5f9648a6330bdbf

we might want to wait until we can chunk the tests.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sounds fine. Marking bug 1916456 as dependency.

Also please note that the 2 perma failing restart tests (that would have to be fixed first as well) are causing an extra 16min delay! I should take a look at those first as well. Not sure if they are actually based on the same underlying issue. I would have to test, but this won't happen this week.

https://treeherder.mozilla.org/logviewer?job_id=472911439&repo=try&lineNumber=60738
https://treeherder.mozilla.org/logviewer?job_id=472911439&repo=try&lineNumber=61353

Comment 3

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

this didn't work as well as I would have liked:
https://treeherder.mozilla.org/jobs?repo=try&revision=a73e75b536b03a62cd73b4364c9e46531a1d115e

first 2 chunks barely ran anything, and the 3rd chunk ran too much and hit the time limit. more work when I am back from PTO

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The tricky part are the restart tests! Each and every shutdown and startup of Firefox takes a good amount of time. If we have trouble to balance out between the different chunks I can expect more failures in the future if more of such tests are getting added.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

TSAN builds are slow as well. So we might need bug 1921411 resolved first too.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

Attachment: Bug 1916662 - Enable marionette tests for tsan builds. r=whimboo

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8341f229117b
Enable marionette tests for tsan builds. r=webdriver-reviewers,releng-reviewers,bhearsum,jdescottes

Comment 8

[Serban Stanca [:SerbanS]]

Backed out for causing tsan marionette failures.

• Backout link
• Push with failures
• Failure Log
• Failure line: SUMMARY: ThreadSanitizer: data race /builds/worker/checkouts/gecko/toolkit/xre/ProfileReset.h:72:35 in ProfileResetCleanupAsyncTask::Run()

Comment 9

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/790e21bff175
Backed out changeset 8341f229117b for causing tsan marionette failures.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

thanks, this appears to be something tsan is good at catching, I filed a bug just so I can move forward with this and ideally the tsan race will get fixed soon.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Joel, a patch is already attached on bug 1925378 so I would prefer to actually wait with landing the patch on this bug until the bug is fixed.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Bug 1925378 is fixed now. So we can try to land the patch again, right?

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2be4fadbf41d
Enable marionette tests for tsan builds. r=whimboo,webdriver-reviewers,releng-reviewers,bhearsum,jdescottes

Comment 14

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/2be4fadbf41d