1250889 - Please add submissionDate to the longitudinal data set

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect)

Description

[Mark Reid [:mreid]]

Please add the submissionDate field to the longitudinal dataset.

Other fields that might be of interest, and are not available in the payload itself, but which I don't have an immediate use for:
DNT - the status of the Do Not Track flag
Size - the size in bytes of the original json payload
sampleId - a number 0 <= n < 100 that partitions the clientId space into 100 groups. This field will only be useful if/when the dataset is increased in size beyond 1%.
creationTimestamp - a pre-parsed version of the creationDate field

Comment 1

[Anthony Zhang [:azhang] (last day at Mozilla: 2016-04-29)]

The DNT_USAGE histogram should have Do Not Track info - is that what you meant by DNT status?

Ref: https://github.com/mozilla/telemetry-batch-view/pull/30

Comment 2

[Mark Reid [:mreid]]

(In reply to Anthony Zhang [:azhang] from comment #1)
> The DNT_USAGE histogram should have Do Not Track info - is that what you
> meant by DNT status?
> 
> Ref: https://github.com/mozilla/telemetry-batch-view/pull/30

The web server that receives Telemetry submissions stores the actual DNT header value the client sends. This is more comprehensive than the DNT_USAGE histogram, because it is included for all document types (even ones without histograms), and that histogram is only being collected for clients who opt in to extended telemetry.

You should find in the "fields" section of the Heka message.

Comment 3

[Roberto Agostino Vitillo (:rvitillo)]

https://github.com/mozilla/telemetry-batch-view/commit/c765287506e935d74662eac7836cca1d221848fa

Comment 4

[Roberto Agostino Vitillo (:rvitillo)]

https://github.com/mozilla/telemetry-batch-view/pull/30