1430125 - Port HBaseAddonRecommenderView.scala to Python and DynamoDB

Status: VERIFIED FIXED

(Data Platform and Tools :: General, enhancement)

Description

[Victor Ng [:vng] [:crankycoder]]

The existing TAAR recommendation dataset is generated by :

https://github.com/mozilla/telemetry-batch-view/blob/master/src/main/scala/com/mozilla/telemetry/views/HBaseAddonRecommenderView.scala

That codebase streams data from the telemetry HBase instance to the TAAR HBase instance.

We need to port the codebase to stream from telemetry HBase into a new TAAR DynamoDB table instead.  Porting to python would be ideal to increase the number of potential reviewers as well.

Comment 1

[mlopatka]

I believe the new DynamoDB fill job is fully reviewed and landed in pyhton_mozetl and has been running successfully for about a week. Closing this bug as fixed.

Comment 2

[mlopatka]

@Anthony, can you verify?

Comment 3

[Victor Ng [:vng] [:crankycoder]]

For record keeping sake - the dynamo db fill job is over at:

https://github.com/mozilla/python_mozetl/blob/master/mozetl/taar/taar_dynamo.py

Comment 4

[Anthony Miyaguchi [:amiyaguchi]]

Some context for this bug:

original airflow PR:
https://github.com/mozilla/telemetry-airflow/pull/222

adding taar_dynamo to mozetl:
https://github.com/mozilla/python_mozetl/pull/196
https://github.com/mozilla/python_mozetl/pull/197
https://github.com/mozilla/python_mozetl/pull/199

scheduling taar_dynamo on airflow
https://github.com/mozilla/telemetry-airflow/pull/243

I can also verify that the job has been running for the last week.