1438927 - Run main summary backfill going back one year

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Description

[Mark Reid [:mreid]]

Target date for starting backfill is Feb 21.

We should try to land dependent bugs before that.

Comment 1

[Mark Reid [:mreid]]

Update: given that not all the new fields have landed yet, this backfill won't happen before next week.

Comment 2

[bugzilla]

Update: We've run the airflow jobs up to March 1, 2017 and the copy over to the production location is scheduled for tomorrow evening.

Here's the final list of fields added in the backfill:

optional binary normalized_os_version (UTF8);
optional int32 cpu_count;
optional int32 cpu_cores;
optional binary cpu_vendor (UTF8);
optional int32 cpu_family;
optional int32 cpu_model;
optional int32 cpu_stepping;
optional int32 cpu_l2_cache_kb;
optional int32 cpu_l3_cache_kb;
optional int32 cpu_speed_mhz;
optional binary gfx_features_d3d11_status (UTF8);
optional binary gfx_features_d2d_status (UTF8);
optional binary gfx_features_gpu_process_status (UTF8);
optional binary gfx_features_advanced_layers_status (UTF8);
optional binary previous_build_id (UTF8);
optional binary session_id (UTF8);
optional binary subsession_id (UTF8);
optional binary previous_session_id (UTF8);
optional binary previous_subsession_id (UTF8);
optional binary session_start_date (UTF8);
optional int64 session_length;
optional binary update_channel (UTF8);
optional boolean update_enabled;
optional boolean update_auto_download;
optional int32 sandbox_effective_content_process_level;
optional int64 client_clock_skew;
optional int64 client_submission_latency;
optional binary user_pref_browser_search_region (UTF8);
optional group histogram_content_fx_new_window_ms (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_fx_tab_close_time_anim_ms (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_distribution_among_content (MAP) {
  repeated group key_value {
    required binary key (UTF8);
    optional group value (MAP) {
      repeated group key_value {
        required int32 key;
        optional int32 value;
      }
    }
  }
}
optional group histogram_content_memory_heap_allocated (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_resident_fast (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_total (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_unique (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_vsize (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_content_memory_vsize_max_contiguous (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_fx_new_window_ms (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_fx_tab_close_time_anim_ms (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_fx_tab_switch_total_e10s_ms (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_distribution_among_content (MAP) {
  repeated group key_value {
    required binary key (UTF8);
    optional group value (MAP) {
      repeated group key_value {
        required int32 key;
        optional int32 value;
      }
    }
  }
}
optional group histogram_parent_memory_heap_allocated (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_resident_fast (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_total (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_unique (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_vsize (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}
optional group histogram_parent_memory_vsize_max_contiguous (MAP) {
  repeated group key_value {
    required int32 key;
    optional int32 value;
  }
}

Comment 3

[bugzilla]

Forgot to close this out after the final copy step last week. Seems to have gone smoothly -- no one has reported any bugs and folks seem to be happily running their queries