1290210 - Decide upon storage layout for games telemetry data

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, defect, P2)

Description

[Mark Reid [:mreid]]

For Firefox telemetry submissions, we partition the data storage by a number of dimensions for conveniently fetching a subset of the data at a time.

These dimensions include things like submission date, firefox version, channel, etc.

Aside from submission date, what are the interesting dimensions by which we might want to partition this data?

Comment 1

[Jukka Jylänki]

Does this affect how the data is stored? Or is this only about developing a suitable presentation/views to the data?

The data that is recorded is documented here: https://s3.amazonaws.com/mozilla-games/tmp/browser_feature_test/example.html

Comment 2

[:Dominik Strohmeier [:dstrohmeier]]

(In reply to Jukka Jylänki from comment #1)
> Does this affect how the data is stored? Or is this only about developing a
> suitable presentation/views to the data?

From what I have understood, these dimensions are the ones that may allow for interesting partitions in terms of slicing the data for analysis. So, it's about analysis and presentation, not about storing the data.

I'd suggest that these dimensions for partitioning the data are:
submission date, arch, platform, os, osVersion, browserProduct, formFactor, bitness

Comment 3

[Roberto Agostino Vitillo (:rvitillo)]

(In reply to Dominik Strohmeier from comment #2)
> (In reply to Jukka Jylänki from comment #1)
> > Does this affect how the data is stored? Or is this only about developing a
> > suitable presentation/views to the data?
> 
> From what I have understood, these dimensions are the ones that may allow
> for interesting partitions in terms of slicing the data for analysis. So,
> it's about analysis and presentation, not about storing the data.

Partitioning is helpful to speed up analyses as the query engine doesn't have to fetch all the data if you are interested only in a specific sub population, like e.g. "os = darwin".

Comment 4

[Mark Reid [:mreid]]

This is no longer a priority. Closing bugs, as they won't be worked on in the foreseeable future.