Azure DevOps Environments with deployment gates: the approval workflow we replaced, the automated quality gate that replaced it, and the environment that had no protection rules until it deployed to prod
A developer with contributor access deployed directly to the production environment because it had no protection rules. The full environments setup across dev, staging, and prod, the REST-API-driven quality gates, and the audit query that surfaces unprotected environments across the org.
The deployment happened at 11:23 on a Thursday. A developer was trying to reproduce a customer issue and wanted the latest build on production to test against. The staging environment was behind by two commits and the fix they needed was in neither of those commits yet. So they opened the pipeline, picked the production environment from the dropdown, and ran it. No prompt. No approval request. No gate. The pipeline succeeded in four minutes and the build was live.
Nobody noticed for two hours. The build was fine, as it happened. The change the developer was carrying was a one-line config tweak that had already passed staging on a different branch. But the production environment had no protection rules, and the deployment happened outside any release process, with no audit trail beyond the pipeline run log.
The postmortem was short. There was no incident, no outage, no customer impact. The action item was: add protection rules to the production environment and audit every other environment in the organisation for the same gap. What came out of that audit was a broader rebuild of how we use Azure DevOps Environments, deployment gates, and quality checks. This is that rebuild.
What Azure DevOps Environments actually give you
An Environment in Azure DevOps is a logical deployment target that your pipeline YAML references in a deployment job. The reason to use Environments rather than a plain job is the surface they expose: resource tracking, deployment history, approvals, checks, and a clear audit trail per environment rather than per pipeline run.
The features that matter most for what we were building:
- Approvals and checks: gate a deployment on human approval, a REST API response, an Azure Function result, or a pipeline resource. These compose.
- Exclusive lock: only one deployment runs against an environment at a time, serialising concurrent pipeline runs rather than letting them race.
- Deployment history: a per-environment timeline of every deployment, who triggered it, from which pipeline, with which source version.
- Branch control: restrict which branches may deploy to an environment.
None of these are on by default. An Environment you create in the UI has no protection rules until you add them. That is how the Thursday deployment happened.
The audit
Before fixing anything, we needed to know the full scope of the gap. The Azure DevOps REST API exposes environment protection rules per project. A quick script across every project in the org:
ORG="https://dev.azure.com/your-org"
TOKEN=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv)
for PROJECT in $(az devops project list --org "$ORG" --query "value[].name" -o tsv); do
ENV_LIST=$(curl -s -u ":$TOKEN" "$ORG/$PROJECT/_apis/pipelines/environments?api-version=7.1-preview.1" | jq -r '.value[] | [.id, .name] | @tsv')
while IFS=$'\t' read -r ENV_ID ENV_NAME; do
CHECKS=$(curl -s -u ":$TOKEN" "$ORG/$PROJECT/_apis/pipelines/checks/configurations?resourceType=environment&resourceId=$ENV_ID&api-version=7.1-preview.1" | jq '.count')
if [ "$CHECKS" -eq 0 ]; then
echo "UNPROTECTED: $PROJECT / $ENV_NAME (id $ENV_ID)"
fi
done <<< "$ENV_LIST"
done
The output was 31 unprotected environments across 11 projects. Fourteen of them had "prod" or "production" in the name.
The environment structure
We settled on three environments per deployable service: dev, staging, and prod. The pipeline YAML references them in deployment jobs that run in sequence.
stages:
- stage: DeployDev
jobs:
- deployment: deploy_dev
environment: my-service-dev
strategy:
runOnce:
deploy:
steps:
- template: steps/deploy.yml
parameters:
environment: dev
- stage: DeployStaging
dependsOn: DeployDev
jobs:
- deployment: deploy_staging
environment: my-service-staging
strategy:
runOnce:
deploy:
steps:
- template: steps/deploy.yml
parameters:
environment: staging
- stage: DeployProd
dependsOn: DeployStaging
jobs:
- deployment: deploy_prod
environment: my-service-prod
strategy:
runOnce:
deploy:
steps:
- template: steps/deploy.yml
parameters:
environment: prod
Each environment is created with protection rules via the REST API rather than the UI, so the configuration is repeatable and auditable.
Protection rules per environment
dev: no approval required, no branch restriction. Deploys on every merge to main. This is the fast-feedback loop.
staging: branch control (only main and release/* may deploy), an automated quality gate that calls our integration test runner via an Azure Function check, and an exclusive lock so parallel pipelines queue rather than race.
prod: branch control (release/* only), a required human approval from anyone in the release-approvers group, the same integration test gate as staging, and an exclusive lock. The approval has a 48-hour timeout after which the deployment is rejected automatically.
Setting the branch control via REST:
curl -s -u ":$TOKEN" -X POST -H "Content-Type: application/json" "$ORG/$PROJECT/_apis/pipelines/checks/configurations?api-version=7.1-preview.1" -d '{
"type": { "id": "fe1de3ee-a436-41b4-bb20-f6eb4cb879a6", "name": "Task Check" },
"settings": {
"displayName": "Branch control",
"definitionRef": {
"id": "86b05a0c-73e6-4f7d-b3cf-e38f3b39a75b",
"name": "evaluatebranchProtection",
"version": "0.0.1"
},
"inputs": {
"allowedBranches": "refs/heads/release/*",
"ensureProtectionOfBranch": "false"
},
"retryOnFailure": false,
"timeoutMinutes": 0
},
"resource": { "type": "environment", "id": "'"$ENV_ID"'" }
}'
The quality gate Azure Function
The automated gate is an HTTP-triggered Azure Function that the environment check calls before any deployment proceeds to staging or prod. The function:
- Accepts the pipeline run id and environment name in the request body
- Queries the Azure DevOps Test Results API for the most recent test run associated with that pipeline run
- Returns HTTP 200 if pass rate is above threshold and no blocking test categories have failures
- Returns HTTP 500 otherwise, which the environment check treats as a gate failure
import logging
import os
import httpx
from azure.functions import HttpRequest, HttpResponse
ADO_ORG = os.environ["ADO_ORG"]
ADO_TOKEN = os.environ["ADO_PAT"]
PASS_THRESHOLD = float(os.environ.get("PASS_THRESHOLD", "0.98"))
def main(req: HttpRequest) -> HttpResponse:
body = req.get_json()
project = body["projectName"]
run_id = body["pipelineRunId"]
url = f"https://dev.azure.com/{ADO_ORG}/{project}/_apis/test/runs?buildIds={run_id}&api-version=7.1"
resp = httpx.get(url, auth=("", ADO_TOKEN))
runs = resp.json().get("value", [])
if not runs:
return HttpResponse("No test runs found", status_code=500)
latest = max(runs, key=lambda r: r["completedDate"])
total = latest.get("totalTests", 0)
passed = latest.get("passedTests", 0)
if total == 0:
return HttpResponse("No tests executed", status_code=500)
rate = passed / total
if rate < PASS_THRESHOLD:
return HttpResponse(f"Pass rate {rate:.1%} below threshold {PASS_THRESHOLD:.1%}", status_code=500)
return HttpResponse("Gate passed", status_code=200)
Branch control enforcement
The branch restriction on the prod environment means that if a developer tries to run the pipeline against main targeting the prod deployment job, the environment check rejects it before the deployment job starts. The pipeline run fails at the check stage, not mid-deployment. This is the key difference from the Thursday incident: the rejection happens before any infrastructure is touched.
Where we ended up
Thirty-one unprotected environments were brought into compliance over two sprints. Every prod environment in the org now requires a release branch, passes the quality gate, and requires a named human approval with a 48-hour expiry. The exclusive lock prevents concurrent deployments from racing. The audit query runs nightly and pages the platform team if any new unprotected environment appears.
The Thursday deployment would not happen today. A developer triggering the prod deployment job on main hits the branch control gate and gets a clear rejection message. The approval requirement means that even on a release branch, a solo engineer cannot push to prod without a second pair of eyes. The quality gate means that even with approval, a build with a degraded test suite does not reach production.
The actual change that stuck was less technical than process: making it impossible for the fast path to accidentally be the production path. The environments model in Azure DevOps is well-suited to that goal. The gap was that we left it unconfigured for two years because nothing had gone wrong yet. Now it is configured by default, enforced by a nightly audit, and the 31-environment gap is closed.