1337995 - Pin or otherwise resolve python_moztelemetry protobuf dependency

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P2)

Description

[Wesley Dawson [:whd]]

At some point today a new release of on pypi for the protobuf dependency occurred that broke our python spark clusters, including scheduled jobs: https://pypi.python.org/pypi/protobuf/3.2.0

I manually patched the requirements in the bootstrap script to load 3.1.0, which works, because I off-hand do not know how to reference the appropriate version of 3.2.0 (the one uploaded on 2017-01-30, as opposed to the newer protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl).

We should either permanently pin protobuf to a working version or otherwise resolve this.

Comment 1

[Wesley Dawson [:whd]]

The temporary fix was made "permanent" by https://github.com/mozilla/emr-bootstrap-spark/pull/62 but we should still figure out what the right thing to do here is.

Comment 2

[Mauro Doglio [:mdoglio]]

Pinning protobuf seems the right thing to do short term. I'll open a pull request for python_moztelemetry.

Comment 3

[GitHub Autolander Bot]

Attachment: [python_moztelemetry] mozilla:bug-1337995-pin-protobuf-version > mozilla:master