1294465 - Test Pilot WebExtension experiments are not working on the tabs opened before installation

Status: RESOLVED INCOMPLETE

(Add-on SDK Graveyard :: General, defect)

Description

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Ads QA, and Solo QA teams)]

Attachment: Experiments are not working on tabs opened before Test Pilot installation.mp4

[Notes]:
- This issue was initially logged on Test Pilot but after discussing with the developers there we inclined to think it's more of a WebExtension issue 

[Affected versions]:
- Nightly 51.0a1
- Test Pilot v0.8.5-dev-c071788
- Probably all experiments available trough Test Pilot

[Affected Platforms]:
- All Windows

[Prerequisites]:
- Test Pilot environment pref for dev server was already added in about:config (string "testpilot.env" with "dev" value). 
- At least one tab is opened with any website in the browser.

[Steps to reproduce]:

1. Open a new tab and go to https://testpilot.dev.mozaws.net page.
2. Install Test Pilot add-on and after enable "Blok" experiment.
3. On the tab that was opened before Test Pilot installation and on the tab with Blok experiment page, navigate to a page known for tracking user actions (eg: http://goo.gl/CY4AUu)
4. Observe the top part of the website on both tabs.
5. Open a new tab and navigate again to the same page on both the new tab and on the onboarding page tab opened after test Pilot installation.
6. Observe again the top part of the website on both tabs.

[Expected result]:
In all the 4 tabs the Blok bar is displayed on top, and the page content that tracks user actions is blocked.

[Actual result]:
Step 4 - The Blok bar is not displayed, and the content that tracks user actions is visible on both tabs that were opened before Test Pilot installation.
Step 6 - The Blok bar is displayed, and the content that tracks user actions is not visible on both tabs that were opened after Test Pilot Installation. 

[Additional Notes]:
- This issue probably affects all experiments from Test Pilot. Tested also with NoMore404s one and the archived page from Wayback Machine is not provided on the same tabs.
- This issue only occurs on the tabs opened before Test Pilot installation (including the tab from which Test Pilot is installed).
- Added a screen recording with the issue.

Comment 1

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Updating title to clarify that WebExtensions, not other types of add-ons, seem to be affected

Comment 2

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

To be clear, the Blok and NoMore404s extensions are WebExtensions, and seem to be affected.

Universal Search, Activity Stream, and Tab Center are non-WebExtension add-ons, and this bug hasn't been observed there.

Comment 3

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1399562