1400187 - TAAR: Build a module for the ensemble methods

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P2)

Description

[Florian Hartmann]

The Jupyter notebook created earlier [1] should be converted into a general Python module. This module should have extensive test coverage

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1400186

Comment 1

[Mark Reid [:mreid]]

Florian, are you currently working on this? Are you planning to work on it in Q4?

Comment 2

[mlopatka]

Florian is currently prototyping different approaches for combining the recommendations that the different TAAR modules provide.
Once that is ready to go, Alessio and I are planning to productionize it and evaluate the feasibility of calling all four TAAR modules per recommendation request.

Assuming performance is sufficient and the ensemble method looks promising on a validation set, we aim to have a followup SHIELD study in Q4 to compare addon installations and addon retention between Linear-TAAR and Ensemble-TAAR.

Comment 3

[mlopatka]

Comment copied over from duplicate bug for a nice history of this approach:

The current execution of the TAAR recommendation driver selects a recommendation strategy based on available user information. Specific availability of fields I.e. other installed addons, Legacy addons installed, telemetry info in our H-base instance determine which of the recommender systems provide the recommendations.

However it is possible to combine the different sources of information by allow each module to contribute a set of recommended addons and accompanying "confidence" of the output. This would allow a reweighting of a finalized recommendation list that integrates more information into the final list that any individual recommender module has access to.

In order to ensure comparable confidence values on recomendations, each module would have to normalize its own confidence score based on the the range and probability density of such a confidence value being produced.

Additionally, the recommendations passed to RecommendationManager would have to include an ordered list of confidence; or provide a list of tuples including confidence per guid in its output.

Comment 5

[Firefox Bug Husbandry Bot]

Changed to new component, per bug 1425844.

Comment 6

[Victor Ng [:vng] [:crankycoder]]

This was shipped in the TAARv2 experiment.