1271450 - Need to make daily oom crash data accessible via Presto for re:dash

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[David Durst [:ddurst]]

This bug is pending merge of https://github.com/mozilla-services/data-pipeline/pull/204.

Once the data is being created on S3, we'll want to make it accessible via Presto.

(CCing Roberto by request.)

Comment 1

[David Durst [:ddurst]]

I've just backfilled data for 4/27-5/10. I will now schedule the job per https://github.com/mozilla-services/data-pipeline/pull/204#issuecomment-218414937.

Comment 2

[Roberto Agostino Vitillo (:rvitillo)]

There are some changes required for the dataset to be imported into Presto:

1) The dataset name should not contain dashes and it should be changed to "crash_stats_oom"
2) A version is required to keep track of the schema, e.g. "v1"

You should change your notebook to output the data in "s3://net-mozaws-prod-us-west-2-pipeline-analysis/ddurst/crash_stats_oom/v1".

In the meantime I moved the existing data to the right prefix on S3 and imported the tables in Presto. Once you make those changes to the notebook we can close this bug.

Comment 3

[David Durst [:ddurst]]

Updated in job and PR 204.

(I assume you moved the existing data for 2016-05-11 as well? Thank you.)

Comment 4

[David Durst [:ddurst]]

Whoops: https://github.com/mozilla-services/data-pipeline/pull/207

Comment 5

[Roberto Agostino Vitillo (:rvitillo)]

(In reply to David Durst [:ddurst] from comment #3)
> Updated in job and PR 204.
> 
> (I assume you moved the existing data for 2016-05-11 as well? Thank you.)

Yes.