Closed Bug 1425075 Opened 8 years ago Closed 8 years ago

Split tracking protection blocklist by tracker categories

Categories

(Cloud Services :: Server: Shavar, enhancement)

Product:
Cloud Services
Component:
Server: Shavar
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: esawin, Assigned: groovecoder)

References

Details

(Whiteboard: [geckoview:klar])

Currently, the tracking protection base list (base-track-digest256) contains ad, analytic and social category trackers. To support Focus-like [1] blocking settings, we need to split the base list up into category-based lists. Note: this is related to bug 1335646, but would go further by splitting the misc list further up. [1] https://blog.mozilla.org/blog/2017/06/20/firefox-focus-new-to-android-blocks-annoying-ads-and-protects-your-privacy/
Status: NEW → ASSIGNED
I've implemented this in a branch of the list-creation repo here: https://github.com/mozilla-services/shavar-list-creation/pull/50 To push this live we: 1. Merge the list-creation PR to add this new logic 2. Merge a PR adding the ads, analytics, and social configs to https://github.com/mozilla-services/shavar-list-creation-config/blob/master/stage.ini 3. Test the resulting lists on the stage server 4. Send the same PR adding ads, analytics, and social configs to https://github.com/mozilla-services/shavar-list-creation-config/blob/master/prod.ini 5. Test the resulting lists on the prod server :esawin - is there an ETA for supporting Focus-like blocking settings? Will it be in Focus, Fennec, GeckoView, somewhere else?
Flags: needinfo?(esawin)
Luke flagged this for me and I wanted to weigh in quickly. Even if we break out these categories, the default blocking behavior should stay the same. Meaning that on desktop we currently block everything but the content category. On Focus we block ads, socials, and analytics, but not the content category. The same stuff should get blocked by the default behavior on desktop even if we break out the categories.
Also, :sebastian - (how) does Focus split up the "Disconnect" category and put its contents into "ads", "analytics" and/or "social" ?
Flags: needinfo?(s.kaspari)
(In reply to Luke Crouch [:groovecoder] from comment #3) > :esawin - is there an ETA for supporting Focus-like blocking settings? Will > it be in Focus, Fennec, GeckoView, somewhere else? We're planning to implement a tracking protection API in GeckoView enabling support for Focus-like apps in Q1. The GeckoView-based Focus build would be based on this new API. Currently, there are no plans to support such settings in Fennec or on desktop, this would need to run through the respective product peers.
Flags: needinfo?(esawin)
Thanks :esawin. When you start work on the GeckoView TP API, please ping back here and we can sync up the *names* of the lists that GeckoView will download & use. Once we decide the names of the lists, I can add them to the config and they'll be generated and served within 30m of deployment.
(In reply to Luke Crouch [:groovecoder] from comment #7) > Once we decide the names of the lists, I can add them to the > config and they'll be generated and served within 30m of deployment. To match what's displayed in the Focus prefs, I suggest these internal names: - ads-track-digest256 - social-track-digest256 - analytics-track-digest256 - content-track-digest256 (no change to this one)
I put those names in a PR to the staging config: https://github.com/mozilla-services/shavar-list-creation-config/pull/25 :rbillings, :ckolos - no rush, but when you're ready we can review + merge the 2 PRs to do a stage test.
Flags: needinfo?(s.kaspari)
Flags: needinfo?(rbillings)
Flags: needinfo?(ckolos)
I'm back from PTO & am ready to work on this.
Flags: needinfo?(rbillings)
Luke, what are the next steps for this bug? Are you waiting for esawin to comment on the list names (comment 7)? Or just for rbillings and ckolos to review and merge your PRs?
Flags: needinfo?(lcrouch)
Whiteboard: [geckoview:klar]
(In reply to François Marier [:francois] from comment #8) > (In reply to Luke Crouch [:groovecoder] from comment #7) > > Once we decide the names of the lists, I can add them to the > > config and they'll be generated and served within 30m of deployment. > > To match what's displayed in the Focus prefs, I suggest these internal names: > > - ads-track-digest256 > - social-track-digest256 > - analytics-track-digest256 > - content-track-digest256 (no change to this one) These names look fine to me. I've blocked the GeckoView implementation (bug 1423229) on this bug, so I can update the lists accordingly before landing.
Also, we need to update the Gecko prefs (e.g., "urlclassifier.trackingTable") to include the split lists, are we going to handle it in this bug?
(In reply to Eugen Sawin [:esawin] from comment #13) > Also, we need to update the Gecko prefs (e.g., > "urlclassifier.trackingTable") to include the split lists, are we going to > handle it in this bug? That's assuming that we want to move Gecko to the new split lists and eventually stop serving the base list.
:rbillings and I will work on merging & deploying my PR this week. We should let :francois decide if we should update the urlclassifier.trackingTable value along with this change. That will be a significant change so I imagine we will want a follow-up bug.
Flags: needinfo?(lcrouch) → needinfo?(francois)
We'll have to add the new lists to browser.safebrowsing.provider.mozilla.lists so that they work properly in geckoview, but until product decides to split the lists, there's no need to update urlclassifier.trackingTable (and the about:preferences code) to use the new lists.
Flags: needinfo?(francois)
(In reply to François Marier [:francois] from comment #16) > We'll have to add the new lists to > browser.safebrowsing.provider.mozilla.lists so that they work properly in > geckoview... I guess that's something I can handle in bug 1423229, I'll prepare the patch. To clarify, does the union of the ads, social, analytics lists contain all trackers from the original base list? What do we want to do with the disconnect category, serve it as its own list? From the GeckoView perspective, we could offer it as an other/misc category, however, I'm not sure how much, if any, overlap the list has with the other lists.
(In reply to Eugen Sawin [:esawin] from comment #20) > (In reply to François Marier [:francois] from comment #16) > > We'll have to add the new lists to > > browser.safebrowsing.provider.mozilla.lists so that they work properly in > > geckoview... > > I guess that's something I can handle in bug 1423229, I'll prepare the patch. > > To clarify, does the union of the ads, social, analytics lists contain all > trackers from the original base list? The original base default list categories are Ads, Analytics, Social, and Disconnect. (https://github.com/mozilla-services/shavar-list-creation/blob/master/lists2safebrowsing.py#L81) > > What do we want to do with the disconnect category, serve it as its own > list? From the GeckoView perspective, we could offer it as an other/misc > category, however, I'm not sure how much, if any, overlap the list has with > the other lists. I included the Disconnect list to maintain parity and backwards compat. But it just contains Facebook, Google, and Twitter domains, which area already included in the other Social & Analytics categories too. So it is likely redundant. https://github.com/mozilla-services/shavar-prod-lists/blob/master/disconnect-blacklist.json#L8910
Flags: needinfo?(ckolos)
Verified in production, the new and existing lists look good.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
(In reply to Luke Crouch [:groovecoder] from comment #22) > I included the Disconnect list to maintain parity and backwards compat. But > it just contains Facebook, Google, and Twitter domains, which area already > included in the other Social & Analytics categories too. So it is likely > redundant. > > https://github.com/mozilla-services/shavar-prod-lists/blob/master/disconnect- > blacklist.json#L8910 It seems like some ad trackers (e.g., doubleclick.net and adwords.google.com) are only included in in the Disconnect (Google) category and not in Advertisement. Would it be possible to instead split (or copy) out the Disconnect trackers to their respective categories?
Flags: needinfo?(lcrouch)
Or should we block the Disconnect category by default with TP enabled in GeckoView?
Moved further discussion to bug 1459037.
Flags: needinfo?(lcrouch)
Depends on: 1465528
Blocks: 1536380
You need to log in before you can comment on or make changes to this bug.