Closed
Bug 1310486
Opened 9 years ago
Closed 8 years ago
Prevent Airflow from running past jobs in dev
Categories
(Data Platform and Tools :: General, defect, P2)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: bugzilla, Assigned: bugzilla)
References
Details
Attachments
(1 file)
Currently, a fresh dev deployment will try to re-run all jobs that are elapsed since the start date, since there's no database record for those runs. This is fine as long as the AWS credentials aren't set in dev. If the credentials happen to be available, however, the clusters for those jobs will actually spin up in aws and this becomes an expensive mistake.
We should figure out how to prevent this from happening and only kick off jobs in dev if they're explicitly triggered.
|
||
Comment 1•9 years ago
|
||
|
||
Updated•8 years ago
|
Component: Metrics: Pipeline → Scheduling
Product: Cloud Services → Data Platform and Tools
|
||
Comment 2•8 years ago
|
||
AFAICT this bug was fixed by the referenced PR from :sunahsuh. Further, this setting was made the default by airflow 1.8.0 so our new dev containers should also not exhibit this issue. Closing.
Assignee: nobody → ssuh
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
|
||
Updated•3 years ago
|
Component: Scheduling → General
You need to log in
before you can comment on or make changes to this bug.
Description
•