Operations Engineering Runbooks

These runbooks are designed to provide information and instructions to the Operations Engineering Team when provisioning and supporting tools and services.

Services

What We Don’t Support

Refer to this runbook for a list of services not supported by Operations Engineering, Slack channel contacts, etc to refer users to.

• Not Supported by Operations Engineering

1Password

• Exporting passwords from 1Password
• 1Password User Account Recovery
• Groups and Vaults
• 1Password Chrome extension shortcut conflict
• Users unable to import via the 1Password App
• Recycle Inactive User Accounts
• Alternative ways to share vaults and passwords

AWS

• Requests for AWS Account Access
• AWS Credentials Remediation Process
• Add OIDC between AWS and GitHub
• YJAF AWS Password Resets
• YJAF AWS IAM Account Deletion

Auth0

• Creating an Auth0 Tenant

Certificates

• Manual SSL Certificate Processes
• Respond to expired certificates
• Configuring the Certificate Mappings File

CircleCI

• CircleCI
• CircleCI Troubleshooting

DNS

• Register new gov uk subdomain
• Register new service.gov.uk subdomain
• Register new justice.gov.uk or service.justice.gov.uk subdomain
• Register New Defensive Domain
• Domain Transfers for Non-gov.uk subdomains
• Delegate existing gov.uk subdomain
• Delegate existing service.gov.uk subdomain
• Delegation of subdomains
• Decommissioning Domains
• How to check for domain activity before decommissioning
• How to manually recover deleted DNS records
• How to delete a Hostedzone
• Redirecting Domains
• DNS for services using e-mail

Docker

• Add Docker User (Manual Process)

GitHub

• Add GitHub User
• Adding an SSH Key to GitHub
• Handling Third Party GitHub Requests
• Add GitHub collaborators from a fork PR
• Branch Protection Settings and Issues
• How to respond to a low GitHub seats alert
• How to respond to a low Github Actions minutes alert
• Dormant User Process
• Repository Terraform
• Review Organisation PAT Requests

OS Data Hub

• OS Data Hub API Key Management

PagerDuty

• Add a new Slack channel to a PagerDuty service

Renovate

• Renovate

Sentry.io

• Create a Sentry Internal Integration
• Disabling sending errors to a project in Sentry
• Respond to Sentry Usage Alerts

SonarCloud

• Adding a GitHub repository to MoJ SonarCloud

SSO

• Adding SSO to a tool

Internal

How to be Support

• How to be Support

Operations Engineering Communication Plan

• Operations Engineering Communication Plan

Operations Engineering Team

• New Joiners Guide
• Leavers Guide
• Ways of Working
• Ways of Engineering
• Working Out of Hours

MoJ Organisation Leavers

• Respond to Leavers

Internal Processes

• Add a Runbook
• Add a Slack Alert to our Alert Channel
• Manage Slack RSS Feeds
• Python Best Practice
• Risk Review
• Testing and Release Process
• Incident Response
• Post-Incident Review Proceses
• Incident Log
• Responding to Dependency Alerts
• Secret Naming Convention

Architecture Decision Records

This is a record of architectural decisions made by the Operations Engineering Team

To understand why we are recording decisions and how we are doing it, please see ADR-000

Status	ADR no.	Title
✅	ADR-000	Record Architecture Decisions
✅	ADR-001	Github Failover
⌛️	ADR-002	Sentry Spike Protection
✅	ADR-003	DNS Failover
✅	ADR-004	Docker SSO
✅	ADR-005	Github Standards Branch Protection
✅	ADR-006	Require Authentication for Team Reports
✅	ADR-007	Decomission Developer Portal
✅	ADR-008	Sentry Spike Protection
✅	ADR-009	1Password SSO
✅	ADR-010	1Password Manager Permissions
✅	ADR-011	GitHub Features as Opt In
✅	ADR-012	RSS Feed Aggregation Channel
❌	ADR-013	Archiving the DNS repo
✅	ADR-014	Risk Review
❌	ADR-015	Use of GitHub Actions Runner Controller
✅	ADR-016	Archive DNS-IAC
✅	ADR-017	Revert decision to mandate that repository access must be via a Team
✅	ADR-018	Standardisation of Repository Naming
✅	ADR-019	Management of Github Repositories through Terraform
✅	ADR-020	Bot Account Personal Access Token Standards
✅	ADR-021	Management of DNS Records through OctoDNS
✅	ADR-022	Decomission Maintenance Pages Platform

Statuses:

• 🤔 Proposed
• ✅ Accepted
• ❌ Rejected
• ⌛️ Superseded
• ♻️ Amended

This page was last reviewed on 12 November 2024. It needs to be reviewed again on 12 May 2025 by the page owner #operations-engineering-alerts .