DEV Community: Rahimah Sulayman

Kubernetes on AKS: From Zero to Production Workloads - Publishing to GitHub and Cleaning Up Azure Resources (Part 9/9)

Rahimah Sulayman — Mon, 26 Oct 2026 20:50:19 +0000

Introduction

Building a cloud-native environment is only part of the engineering process. A complete project also needs version-controlled implementation artifacts, a reproducible record of the work, and a deliberate cleanup strategy that prevents unused cloud resources from continuing to incur charges.

In this final mission, I published the Kubernetes manifests to GitHub, verified that the lab namespace could be removed, deleted the Azure resource group, checked the status of the associated resource groups, and removed the obsolete Kubernetes context from the local kubeconfig.

The objective was to preserve the implementation as portfolio evidence while ensuring that the temporary Azure environment was decommissioned.

What This Part Covers

Committing the Kubernetes manifests to Git.
Publishing the repository to GitHub.
Verifying namespace removal.
Deleting the Azure resource group.
Checking that the AKS-managed resource group was removed.
Deleting the obsolete Kubernetes context.
Confirming what remains available after the cloud environment is deleted.

1. Review the Kubernetes Manifests

The project contains nine Kubernetes manifest files covering the workloads and resources implemented throughout the lab.

The files are organized under the manifests/ directory:

ls manifests

The output showed:

01-pod.yaml
02-deployment.yaml
03-service-clusterip.yaml
04-service-loadbalancer.yaml
05-configmap.yaml
06-statefulset.yaml
07-daemonset.yaml
08-job.yaml
09-cronjob.yaml

The manifests capture the main resources from the project:

Manifest	Resource
`01-pod.yaml`	Standalone Pod
`02-deployment.yaml`	Deployment
`03-service-clusterip.yaml`	Internal ClusterIP Service
`04-service-loadbalancer.yaml`	LoadBalancer Service
`05-configmap.yaml`	ConfigMap
`06-statefulset.yaml`	Headless Service and StatefulSet with persistent storage
`07-daemonset.yaml`	DaemonSet
`08-job.yaml`	Batch Job
`09-cronjob.yaml`	Scheduled CronJob

Keeping these resources in version control preserves the configuration used throughout the lab. The manifests provide a reference for the implementation and can be reviewed independently of the Azure environment in which they were deployed.

2. Commit the Manifests to Git

I committed the workload manifests with a message describing the implementation:

git commit -m "feat: AKS workloads (pods, deployment, services, config, statefulset, daemonset, job, cronjob)"

Git confirmed the commit:

[main 610883d] feat: AKS workloads (pods, deployment, services, config, statefulset, daemonset, job, cronjob)
9 files changed, 206 insertions(+)

The commit created the nine manifest files in the repository.

A descriptive commit message makes the change easier to identify in the project history. It also establishes a clear record of the implementation being preserved for the portfolio.

3. Configure the GitHub Remote

I configured the local repository to use the GitHub repository:

https://github.com/rahimahisah17/aks-kubernetes-lab.git

The command used was:

git remote add origin https://github.com/rahimahisah17/aks-kubernetes-lab.git 2>/dev/null || git remote set-url origin https://github.com/rahimahisah17/aks-kubernetes-lab.git

This command adds origin when it does not already exist or updates its URL when the remote has already been configured.

Using a remote allows the local Git repository to publish commits to GitHub and maintain a shared record of the implementation.

4. Push the Project to GitHub

With the remote configured, I pushed the main branch:

git push -u origin main

The command completed successfully:

Enumerating objects: 19, done.
Counting objects: 100% (19/19), done.
Compressing objects: 100% (18/18), done.
Writing objects: 100% (19/19), 3.70 KiB | 84.00 KiB/s, done.
Total 19 (delta 2), reused 0 (delta 0), pack-reused 0

To https://github.com/rahimahisah17/aks-kubernetes-lab.git
 * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

The push published the branch and configured it to track origin/main.

The project repository is available here:

GitHub — aks-kubernetes-lab

Publishing the manifests ensures that the configuration remains accessible even after the cluster and its cloud resources have been deleted.

The repository also serves as portfolio evidence of the project implementation, including the manifest structure and version-control history.

5. Delete the Kubernetes Lab Namespace

After the GitHub push succeeded, I removed the namespace containing the lab workloads:

kubectl delete namespace k8s-lab

Kubernetes returned:

namespace "k8s-lab" deleted

I then inspected the remaining namespaces:

kubectl get namespaces

The output listed the remaining namespaces:

NAME              STATUS   AGE
default           Active   5h9m
kube-node-lease   Active   5h9m
kube-public       Active   5h9m
kube-system       Active   5h9m

The k8s-lab namespace was no longer present.

Deleting a namespace removes the Kubernetes resources contained in it, subject to their deletion and finalization behavior. This removed the dedicated lab namespace and its namespaced workloads from the cluster.

However, deleting a Kubernetes namespace does not itself decommission the AKS cluster or remove every Azure resource associated with it. The Azure resource group therefore needed to be deleted separately.

6. Delete the Azure Resource Group

The Azure environment used the following configuration:

RESOURCE_GROUP=rg-aks-lab
CLUSTER_NAME=aks-lab
LOCATION=westeurope
NODE_SIZE=Standard_D2s_v3

I first attempted to delete the resource group using the environment variable:

az group delete --name $RESOURCE_GROUP --yes --no-wait

The command returned an error:

argument --name/-n/--resource-group/-g: expected one argument

The variable was not populated in that shell because I switched to another terminal, so Azure CLI did not receive a resource group name.

I then supplied the resource group name explicitly:

az group delete --name rg-aks-lab --yes --no-wait

This issued the resource group deletion request.

The --yes option skips the interactive confirmation, while --no-wait returns without waiting for the deletion operation to finish. As a result, the command's return does not, on its own, establish that the resources have already disappeared.

For cleanup operations, verification is essential because Azure resource deletion can continue asynchronously after the command returns.

7. Verify the Azure Resource Group Cleanup

The AKS deployment had a primary resource group and an AKS-managed resource group. I checked for both using:

az group list --query "[?starts_with(name, 'rg-aks-lab') || starts_with(name, 'MC_rg-aks-lab')].name" --output tsv

An initial check still showed:

rg-aks-lab
MC_rg-aks-lab_aks-lab_westeurope

This indicated that the deletion process had not yet removed both resource groups from the listing.

I repeated the check after the deletion process had progressed:

az group list --query "[?starts_with(name, 'rg-aks-lab') || starts_with(name, 'MC_rg-aks-lab')].name" --output tsv

The subsequent output was empty, confirming that neither matching resource group remained in the returned list.

The verification covered both the main lab resource group and the resource group created by AKS to manage supporting infrastructure.

This distinction matters because AKS can create Azure resources outside the visible workload definitions in the Kubernetes manifests. A complete cleanup check should account for the managed infrastructure, rather than checking only the primary resource group.

8. Remove the Obsolete Kubernetes Context

With the AKS environment decommissioned, I removed its local Kubernetes context:

kubectl config delete-context aks-lab

The command returned:

warning: this removed your active context, use "kubectl config use-context" to select a different one
deleted context aks-lab from /home/rahimahisah/.kube/config

The context had stored the connection configuration used by kubectl to access the aks-lab cluster.

Removing it cleans up the local kubeconfig after the cluster has been deleted. The warning also notes that the removed context had been active, so another context would need to be selected before using kubectl against a different cluster.

This step removes the local connection entry; it does not delete the local repository, manifests, or GitHub project.

9. What Remains After Cleanup?

The cleanup separates the cloud environment from the project artifacts.

Component	Final state
`k8s-lab` namespace	Deleted
Main Azure resource group `rg-aks-lab`	No longer present in the verification listing
AKS-managed resource group	No longer present in the verification listing
Local `aks-lab` kubeconfig context	Deleted
Local project directory	Preserved
Kubernetes YAML manifests	Preserved locally
Git commit history	Preserved in the repository
GitHub repository	Published and available

This separation is important for cloud labs. The resources used to demonstrate the implementation can be removed without discarding the configuration, documentation, and version-control history that make the work reproducible and useful as portfolio evidence.

Although the Azure resource groups were removed from the verification listing, the final Azure bill remains the authoritative source for confirming any outstanding or previously accrued charges. Deletion stops the continued use of the deleted resources, but it does not erase charges already incurred.

10. Operational Takeaways

This final mission brought together version control, cloud resource lifecycle management, and verification.

The main takeaways were:

Version-controlled infrastructure: Kubernetes manifests provide a durable record of the resource configuration used in the lab.
Repository publishing: Pushing the project to GitHub preserves the implementation beyond the lifetime of the AKS cluster.
Namespace cleanup: Removing the lab namespace deletes its dedicated Kubernetes workloads and resources.
Azure resource cleanup: Deleting the primary resource group initiates removal of the cluster and related Azure resources.
Asynchronous deletion: Commands using --no-wait require a separate verification step to confirm the resources have disappeared.
Managed-resource awareness: AKS can create a separate managed resource group that should be included in cleanup verification.
Local configuration hygiene: Removing an obsolete kubeconfig context prevents local configuration from retaining an unnecessary connection entry.
Cost awareness: Cloud cleanup should be verified rather than assumed, and actual billed amounts should be checked independently.

Conclusion: From Cluster Provisioning to Decommissioning

The nine-part series brought together AKS provisioning, Kubernetes workloads, networking, configuration, persistent storage, batch processing, scheduled automation, and cost optimization.

In this final mission, I committed the nine workload manifests, pushed the project to GitHub, deleted the k8s-lab namespace, issued the Azure resource group deletion request, verified that the primary and AKS-managed resource groups were no longer listed, and removed the obsolete Kubernetes context.

The result is a preserved project repository without the temporary cloud infrastructure used to run it.

The central operational principle is straightforward: infrastructure work is not complete until its implementation has been preserved and its temporary resources have been deliberately decommissioned. Version-controlled manifests retain the engineering evidence, while verified cleanup helps prevent unnecessary ongoing resource consumption.

This article is Part 9 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads - FinOps: Measure, Price and Cut Cluster Costs (Part 8/9)

Rahimah Sulayman — Mon, 26 Oct 2026 20:26:27 +0000

Introduction

Cloud-native infrastructure is not production-ready simply because its workloads are running. It must also be measurable, cost-aware, and configured to avoid paying for unnecessary capacity.

In this mission, I evaluated the resource consumption of my Azure Kubernetes Service (AKS) cluster, examined Kubernetes resource requests and limits, queried the Azure Retail Prices API to estimate compute costs, and implemented workload and node-pool scaling controls. I also removed an externally exposed LoadBalancer Service, tested the AKS stop/start lifecycle, and documented the findings and optimization actions in COST-REPORT.md.

The objective was to connect Kubernetes resource telemetry with Azure pricing and practical cost-control measures, while distinguishing observed utilization from requested capacity, configured limits, and estimated charges.

What This Part Covers

Measuring node and Pod resource usage.
Inspecting CPU and memory requests and limits.
Installing jq to process Azure pricing data.
Querying the Azure Retail Prices API for the VM size used by the cluster.
Estimating the monthly compute cost.
Configuring workload autoscaling with a Horizontal Pod Autoscaler (HPA).
Enabling cluster autoscaling for the AKS node pool.
Removing the public LoadBalancer Service.
Testing AKS stop/start operations.
Creating and committing a cost report.

1. Measure Actual Resource Utilization

I started with the node-level metrics:

kubectl top nodes

The output showed:

NAME                                CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)
aks-nodepool1-33590657-vmss000000   242m         12%      1479Mi          25%

At the time of measurement, the node was using approximately 12% of its reported CPU capacity and 25% of its memory capacity.

These figures provide a snapshot of actual resource consumption. They are useful for identifying whether the node is under pressure or operating with substantial unused capacity, although a single observation does not establish the workload's peak or long-term requirements.

Inspect the Pods consuming CPU

I then inspected the Pods across namespaces, sorted by CPU usage:

kubectl top pods -A --sort-by=cpu | head -15

The output included system components in kube-system and application workloads in k8s-lab, including:

csi-azurefile-node
azure-cns
metrics-server
konnectivity-agent
coredns
kube-proxy
web Deployment Pods
node-agent
store-0 and store-1

The displayed CPU consumption of the application Pods was low at the time of measurement. This helped establish that the cluster's workloads were not consuming a substantial share of the node's available CPU during the observed interval.

The distinction matters because a Kubernetes cluster can have a relatively low current workload while still reserving significant capacity through its configured resource requests.

2. Compare Resource Requests, Limits and Actual Usage

To understand how much capacity the workloads had requested or been permitted to use, I inspected the node's allocated resources:

kubectl describe nodes | grep -A 8 "Allocated resources"

The output showed:

Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests       Limits
  --------           --------       ------
  cpu                1187m (62%)    10692m (562%)
  memory             1330Mi (22%)   13443360Ki (226%)

The figures reveal three different aspects of resource management.

Metric	CPU	Memory
Observed node utilization	12%	25%
Allocated requests	62%	22%
Allocated limits	562%	226%

Actual utilization represents observed consumption at the time of measurement.

Requests represent the capacity declared by workloads for scheduling purposes. CPU requests of 62% indicate that workloads had reserved a significant proportion of the node's allocatable CPU capacity, even though observed CPU utilization was only 12%.

Limits define upper resource boundaries for individual containers. Their aggregated percentages can exceed 100% because Kubernetes permits resource overcommitment. The reported CPU limit of 562% therefore does not mean that the node was actually using more than five times its CPU capacity.

These measurements should not be interpreted interchangeably. Low current utilization can indicate an opportunity to review requests, but reducing them requires an understanding of the workload's peak behavior, performance requirements, and reliability objectives.

Verify the Deployment's resource settings

I inspected the container configuration for the web Deployment:

kubectl get deployment web -n k8s-lab -o json \
  | jq '.spec.template.spec.containers[] |
        {name: .name, requests: .resources.requests, limits: .resources.limits}'

The result showed:

{
  "name": "nginx",
  "requests": {
    "cpu": "50m",
    "memory": "64Mi"
  },
  "limits": {
    "cpu": "200m",
    "memory": "128Mi"
  }
}

Each NGINX container requested 50m CPU and 64Mi memory, with limits of 200m CPU and 128Mi memory.

With three replicas, those declared requests contribute to the Deployment's aggregate resource reservations. Repeating this comparison across workloads helps identify where requests may be substantially higher than typical consumption or where capacity may be constrained.

The goal is not to minimize requests indiscriminately. It is to align resource allocation with actual workload requirements while preserving sufficient headroom for bursts and normal operating variation.

3. Install jq for Pricing Data Analysis

To process the JSON returned by Azure's pricing endpoint, I installed jq in the Linux environment:

sudo apt-get install -y jq

The package manager installed jq and its required dependencies successfully.

This utility made it easier to filter the Azure Retail Prices API response and extract fields such as the product name, meter, price, unit of measure, region, and price type.

4. Retrieve VM Pricing from the Azure Retail Prices API

The cluster configuration identified the following environment values:

RESOURCE_GROUP=rg-aks-lab
CLUSTER_NAME=aks-lab
LOCATION=westeurope
NODE_SIZE=Standard_D2s_v3

I queried the Azure Retail Prices API for virtual machine pricing matching the region and VM size:

curl -sG "https://prices.azure.com/api/retail/prices" \
  --data-urlencode "\$filter=serviceName eq 'Virtual Machines' and armRegionName eq '$LOCATION' and armSkuName eq '$NODE_SIZE' and priceType eq 'Consumption'" \
  | jq -r '.Items[]
    | select(.meterName | test("Spot|Low Priority"; "i") | not)
    | [.productName, .skuName, .meterName, .retailPrice, .unitOfMeasure, .armRegionName, .priceType]
    | @tsv'

The response contained multiple pricing records for the requested VM SKU. The available records included different operating-system products and pricing types, so the SKU alone was not sufficient to select the correct rate.

I compared the returned fields to distinguish the applicable consumption rate from Windows-specific, DevTest, Spot, and Low Priority entries.

For the cost report, I used an observed public retail consumption rate of $0.12 per hour for the Standard_D2s_v3 VM in westeurope.

Estimate the monthly VM compute cost

Using a simplified month of 730 hours, the calculation was:

[
\text{Estimated monthly cost} =
\text{Hourly rate} \times 730
]

[
\$0.12 \times 730 = \$87.60
]

The resulting estimate was:

Item	Estimate
VM size	`Standard_D2s_v3`
Region	`westeurope`
Hourly rate used	$0.12
Assumed monthly hours	730
Estimated monthly VM compute cost	$87.60

This is a price-based estimate, not a confirmed Azure invoice amount. The calculation covers the VM rate used in the report and excludes additional AKS and Azure resource charges. Actual billing can also depend on the applicable pricing record, subscription or purchasing terms, usage duration, and associated resources.

For that reason, the estimate provides a useful baseline for cost planning, but it should not be presented as the complete operating cost of the cluster.

5. Identify and Remove Unnecessary Public Exposure

I reviewed the Services across namespaces:

kubectl get services -A

The output showed the internal web-svc ClusterIP Service, the headless store Service, and the web-public LoadBalancer Service.

The public Service was exposing NGINX using this external IP:

20.61.154.190

I inspected its configuration and endpoints:

kubectl describe service web-public -n k8s-lab

The Service used the selector app=web, exposed port 80, and routed traffic to the NGINX Pods. Its type was LoadBalancer, and the external IP had been assigned.

The lab no longer required this public endpoint for its remaining work, so I removed the Service:

kubectl delete service web-public -n k8s-lab

Kubernetes confirmed:

service "web-public" deleted from k8s-lab namespace

Removing the Service eliminated that Kubernetes-managed public exposure. The internal Service remained available for cluster-internal access.

This action also removed the LoadBalancer Service configuration that was no longer needed by the lab. Deleting the Service should not be treated as proof that every associated Azure charge disappears immediately; the resulting Azure resources and billing records still need to be considered when validating actual savings.

6. Configure Horizontal Pod Autoscaling

The web Deployment had been configured with three replicas. Rather than permanently fixing it at that count, I created a Horizontal Pod Autoscaler (HPA) to allow Kubernetes to adjust the number of Pods based on CPU utilization.

I configured the HPA as follows:

kubectl autoscale deployment web -n k8s-lab \
  --cpu-percent=70 \
  --min=1 \
  --max=5

Kubernetes returned:

horizontalpodautoscaler.autoscaling/web autoscaled

The command displayed a deprecation notice for --cpu-percent, recommending the newer --cpu option with a percentage or resource quantity format.

I verified the resulting HPA:

kubectl get hpa web -n k8s-lab

The output showed:

NAME   REFERENCE      TARGETS   MINPODS   MAXPODS   REPLICAS
web    Deployment/web cpu: 2%/70%  1      5         3

At the time of the check, the observed CPU utilization was approximately 2% against a 70% target. The HPA was configured to maintain between one and five replicas, with three replicas currently reported.

This configuration allows the replica count to respond to CPU utilization rather than requiring manual scaling for every workload fluctuation.

The HPA does not guarantee that replicas immediately scale down whenever usage is low, nor does it remove the need for appropriate resource requests and metrics. It evaluates workload metrics against its configuration and adjusts the desired replica count as conditions warrant.

7. Enable AKS Node-Pool Autoscaling

Pod autoscaling changes the number of application replicas, but it does not automatically increase the number of virtual machines in the cluster unless cluster-level scaling is configured.

I enabled the cluster autoscaler for nodepool1, allowing the node count to vary between one and three:

az aks nodepool update \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name nodepool1 \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 3

The command updated the node-pool configuration. Azure CLI also displayed a notice that the command behavior had been altered by the aks-preview extension.

I then queried the effective settings:

az aks nodepool show \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name nodepool1 \
  --query "{Pool:name,Count:count,Autoscaling:enableAutoScaling,Min:minCount,Max:maxCount}" \
  --output table

The output showed:

Pool       Count   Autoscaling   Min   Max
nodepool1  1       True          1     3

This verified that cluster autoscaling was enabled and that the configured minimum and maximum node counts were one and three.

The node pool was currently at one node. With autoscaling enabled, Kubernetes can request additional node capacity when scheduling demand requires it, subject to the configured limits and Azure's ability to provision the capacity.

HPA and cluster autoscaling serve different purposes

These settings control separate layers of the environment:

Horizontal Pod Autoscaler: adjusts the number of application Pods.
Cluster autoscaler: adjusts the number of AKS nodes when cluster capacity needs to change.

The HPA was configured for one to five web replicas, while the node pool was configured for one to three nodes. The two mechanisms work at different levels and should be configured together with appropriate resource requests, scheduling requirements, and capacity limits.

The maximum node count is a limit, not a target. Enabling autoscaling does not mean the cluster will remain at three nodes.

8. Test the AKS Stop/Start Lifecycle

I also exercised the AKS cluster's stop/start commands.

To stop the cluster, I ran:

az aks stop \
  --resource-group "$RESOURCE_GROUP" \
  --name "$CLUSTER_NAME"

I then started it again:

az aks start \
  --resource-group "$RESOURCE_GROUP" \
  --name "$CLUSTER_NAME"

Finally, I checked its reported power state:

az aks show \
  --resource-group "$RESOURCE_GROUP" \
  --name "$CLUSTER_NAME" \
  --query powerState.code \
  --output tsv

The resulting state was:

Running

This confirmed that the cluster was running after the stop/start sequence.

For a lab or development environment that is not needed continuously, stopping the cluster during idle periods can be part of a cost-control strategy. However, the cluster must first be checked for active workloads, scheduled operations, dependencies, and service availability requirements.

The stop/start exercise demonstrates the control operation; it is not, on its own, a measured comparison of the before-and-after Azure bill. Actual savings should be verified against the applicable billing records and the resources that continue to incur charges.

9. Document the Findings in COST-REPORT.md

To capture the analysis and decisions, I created COST-REPORT.md in the project repository:

cat << 'EOF' > COST-REPORT.md
# AKS FinOps Cost Report

## Cluster Configuration
- Resource Group: rg-aks-lab
- Cluster: aks-lab
- Region: westeurope
- Node Pool: nodepool1
- VM Size: Standard_D2s_v3
- Node Count: 1

## Resource Utilization
- Node CPU usage: 12%
- Node memory usage: 25%
- CPU requests: 62%
- Memory requests: 22%

## Cost Estimate
- Estimated VM rate: $0.12/hour
- Estimated monthly VM cost (730 hours): $87.60
- Estimate excludes additional AKS and Azure resource charges.

## Cost Optimization Actions
- Configured web deployment autoscaling between 1 and 5 replicas.
- Enabled cluster autoscaling for nodepool1 between 1 and 3 nodes.
- Reduced the node pool to 1 node.
- Removed the web-public LoadBalancer service.
- Stopped the AKS cluster when not in use.

## Notes
The VM price is a public retail-price estimate, not a confirmed invoice amount. Actual costs depend on billing rates and associated Azure resources.
EOF

The report consolidated the cluster configuration, observed utilization, VM price estimate, and optimization actions into a version-controlled artifact.

A concise cost report is useful because it distinguishes the measurements from the actions taken and documents what remains outside the estimate. In this case, it explicitly records that the $87.60 figure excludes additional AKS and Azure resource charges.

10. Commit the Cost Report

I checked the repository state:

git status

The report appeared as an untracked file. I staged it:

git add COST-REPORT.md

Then I committed it:

git commit -m "docs: AKS cost review and optimisation"

Git confirmed the commit:

[main 2725349] docs: AKS cost review and optimisation
1 file changed, 30 insertions(+)
create mode 100644 COST-REPORT.md

This committed the cost report to the main branch, preserving the analysis and optimization decisions alongside the lab's other project documentation.

11. Summary of Cost Optimization Decisions

The main findings and actions were:

Area	Finding or action	Operational purpose
Node utilization	12% CPU and 25% memory usage observed	Establish a snapshot of actual resource consumption
Resource requests	CPU requests at 62%; memory requests at 22%	Identify declared capacity reservations for further review
Resource limits	Aggregated CPU and memory limits above 100%	Understand configured overcommitment rather than actual usage
VM pricing	$0.12/hour used for the estimate	Establish a public retail-price baseline
Monthly estimate	$87.60 for 730 hours of VM compute	Quantify an illustrative compute cost
Workload scaling	HPA configured for one to five replicas	Match the `web` replica count to CPU demand
Node scaling	Cluster autoscaler configured for one to three nodes	Allow node capacity to adjust to scheduling demand
Public networking	`web-public` Service deleted	Remove an external endpoint no longer required by the lab
Cluster lifecycle	Stop/start operations tested	Establish an option for managing idle lab capacity
Documentation	`COST-REPORT.md` committed	Preserve the findings and optimization actions

These actions address different aspects of cost management. Autoscaling controls capacity, removing an unnecessary Service eliminates unneeded exposure, resource measurements inform future tuning, and stopping an idle cluster provides a lifecycle control. The retail-price estimate supports planning but does not replace actual cost analysis.

Conclusion

This FinOps review combined Kubernetes telemetry with Azure pricing data to establish a practical baseline for the AKS lab.

The observed node utilization was 12% CPU and 25% memory, while resource requests and limits revealed how declared capacity differed from current consumption. A public VM consumption rate of $0.12 per hour was used to estimate $87.60 for 730 hours of VM compute, excluding additional AKS and Azure resource charges.

I configured the web HPA to scale between one and five replicas, enabled node-pool autoscaling between one and three nodes, removed the public web-public Service, tested the AKS stop/start workflow, and committed the findings in COST-REPORT.md.

The key principle is that cost optimization should be based on measured usage, explicit assumptions, and deliberate trade-offs. Reducing infrastructure cost is valuable only when the resulting capacity and availability remain appropriate for the workload.

In Part 9, the series will conclude with the final mission, bringing the lab's implementation, documentation, and cleanup together.

This article is Part 8 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads -DaemonSets, Jobs and CronJobs (Part 7/9)

Rahimah Sulayman — Mon, 26 Oct 2026 19:25:12 +0000

Introduction

Kubernetes supports more than continuously running application workloads. It also provides controllers for node-level agents, one-off batch processing, and scheduled tasks.

In this mission, I implemented a DaemonSet to run a lightweight agent on each eligible AKS node, scaled the node pool to verify that the agent was scheduled on a newly added node, and then returned the pool to one node. I also configured a batch Job with three completions and a CronJob that runs every minute, verifying their execution through Job status and container logs.

These workload types address different operational requirements: DaemonSets distribute workloads across nodes, Jobs manage finite tasks, and CronJobs create Jobs on a recurring schedule.

What This Part Covers

Creating and validating a DaemonSet.
Injecting the node name into a container through the Downward API.
Scaling the AKS node pool and verifying DaemonSet placement.
Defining a batch Job with three successful completions.
Inspecting Job status and logs.
Scheduling recurring work with a CronJob.
Verifying scheduled executions and reviewing the resulting cluster resources.

1. Deploy a DaemonSet

A DaemonSet ensures that a copy of a specified Pod runs on each eligible node. This makes it suitable for node-level agents that need to operate wherever workloads are scheduled.

I defined the DaemonSet in manifests/07-daemonset.yaml:

cat << 'EOF' > manifests/07-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-agent
spec:
  selector:
    matchLabels:
      app: node-agent
  template:
    metadata:
      labels:
        app: node-agent
    spec:
      containers:
        - name: agent
          image: busybox:1.36
          command: ["sh", "-c", "while true; do echo \"$(date) agent running on node $NODE_NAME\"; sleep 30; done"]
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          resources:
            requests:
              cpu: 10m
              memory: 16Mi
            limits:
              cpu: 50m
              memory: 32Mi
EOF

Configuration details

The manifest defines a DaemonSet named node-agent with a single container using busybox:1.36.

The container runs a loop that writes a timestamp and the node name to its output every 30 seconds. This provides a lightweight way to confirm which node is hosting each agent.

The node name is injected through the Kubernetes Downward API:

env:
  - name: NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName

This avoids hardcoding a node name into the container command. Each Pod receives the name of the node on which it is running.

The resource configuration also defines explicit requests and limits:

Resource	Request	Limit
CPU	`10m`	`50m`
Memory	`16Mi`	`32Mi`

These values keep the agent's resource requirements small while making its resource boundaries explicit.

I applied the manifest:

kubectl apply -f manifests/07-daemonset.yaml

Kubernetes returned:

daemonset.apps/node-agent created

I then monitored the rollout:

kubectl rollout status daemonset/node-agent

The output confirmed:

daemon set "node-agent" successfully rolled out

2. Verify DaemonSet Placement and Logs

To inspect the DaemonSet's status, I ran:

kubectl get daemonset node-agent

The output showed:

NAME         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE
node-agent   1         1         1       1            1

At this stage, the cluster had one node, and the DaemonSet had one desired and ready Pod.

I checked the Pod's placement with:

kubectl get pods -l app=node-agent -o wide

The output identified the Pod as node-agent-rdmcq, running on:

aks-nodepool1-33590657-vmss000000

The Pod was in the Running state and had the internal IP 10.244.0.172.

To confirm that the agent was reporting the correct node, I inspected its recent logs:

kubectl logs -l app=node-agent --prefix --tail=1

The output included:

[pod/node-agent-rdmcq/agent] Mon Oct 26 10:02:52 UTC 2026 agent running on node aks-nodepool1-33590657-vmss000000

This verified that the agent was running and that the node name supplied through the Downward API was available inside the container.

3. Scale the AKS Node Pool to Verify DaemonSet Behavior

To verify how the DaemonSet responds when the cluster gains another node, I scaled the AKS node pool from one node to two:

az aks scale \
  --resource-group $RESOURCE_GROUP \
  --name $CLUSTER_NAME \
  --nodepool-name nodepool1 \
  --node-count 2

The Azure CLI returned the node pool configuration. Its output also displayed a notice that the command's behavior had been altered by the aks-preview extension.

Once the second node was available, I checked the cluster:

kubectl get nodes

The output showed two nodes in the Ready state:

NAME                                STATUS   ROLES    VERSION
aks-nodepool1-33590657-vmss000000   Ready    <none>   v1.36.4
aks-nodepool1-33590657-vmss000001   Ready    <none>   v1.36.4

I then checked the DaemonSet Pods again:

kubectl get pods -l app=node-agent -o wide

The output showed one agent Pod on each node:

Pod	Status	Node
`node-agent-rdmcq`	Running	`aks-nodepool1-33590657-vmss000000`
`node-agent-mdscg`	Running	`aks-nodepool1-33590657-vmss000001`

The second agent was running on the newly added node, demonstrating the DaemonSet's node-level placement behavior.

Unlike a Deployment, where the replica count is explicitly configured, a DaemonSet aims to maintain a Pod on every eligible node. As the eligible node count changes, the number of agent Pods changes accordingly.

Return the node pool to one node

After verifying the two-node behavior, I scaled the node pool back to one:

az aks scale \
  --resource-group $RESOURCE_GROUP \
  --name $CLUSTER_NAME \
  --nodepool-name nodepool1 \
  --node-count 1

This returned the lab environment to its previous node count. Scaling a node pool affects the cluster's compute capacity and is also relevant when managing cloud costs for a lab environment.

4. Create a Batch Job

A Kubernetes Job manages Pods that perform finite work and are expected to complete successfully. Unlike a continuously running Deployment, a Job tracks successful task completion.

I defined the Job in manifests/08-job.yaml:

cat << 'EOF' > manifests/08-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: report
spec:
  completions: 3
  parallelism: 3
  backoffLimit: 2
  ttlSecondsAfterFinished: 3600
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: report
          image: busybox:1.36
          command: ["sh", "-c", "echo \"Report part generated by $(hostname)\"; sleep 5"]
EOF

Job configuration

The Job is named report and uses busybox:1.36 to print a message identifying the Pod that generated each report part.

The important settings are:

completions: 3 specifies that three successful completions are required.
parallelism: 3 allows up to three Job Pods to run concurrently.
backoffLimit: 2 sets the permitted retry count for failed Pods before the Job is considered failed.
ttlSecondsAfterFinished: 3600 allows the completed Job to be eligible for automatic cleanup one hour after it finishes.
restartPolicy: Never prevents a failed container from being restarted within the same Pod.

This configuration models a small batch operation with three successful executions. Each Pod prints its own hostname, making it possible to distinguish the individual task executions in the logs.

I applied the Job:

kubectl apply -f manifests/08-job.yaml

The initial output confirmed:

job.batch/report created

My first verification attempt returned a NotFound error when waiting for the Job to complete, and the immediate Job listing showed no resources. I reapplied the manifest and checked the resulting Job again.

kubectl apply -f manifests/08-job.yaml

The subsequent execution completed successfully.

5. Verify Job Completion and Inspect the Logs

I checked the Job status:

kubectl get job report

The output showed:

NAME     STATUS     COMPLETIONS   DURATION
report   Complete   3/3           10s

The 3/3 completion count confirmed that the Job had achieved its three required successful completions.

To inspect the output from all the Job's Pods, I ran:

kubectl logs -l job-name=report --prefix

The logs included messages such as:

[pod/report-2v8b5/report] Report part generated by report-2v8b5
[pod/report-bxzps/report] Report part generated by report-bxzps
[pod/report-plnqc/report] Report part generated by report-plnqc

Each execution produced a message containing its own Pod hostname. This confirmed that the Job's three task executions ran and completed.

The logs also demonstrate why Job status and application output are complementary checks. The completion count confirms the controller's recorded outcome, while the logs show what the containers actually executed.

6. Schedule Recurring Work with a CronJob

A CronJob creates Jobs according to a schedule. It is useful for recurring operations such as periodic reporting, cleanup tasks, and scheduled maintenance.

I defined the CronJob in manifests/09-cronjob.yaml:

cat << 'EOF' > manifests/09-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: heartbeat
spec:
  schedule: "*/1 * * * *"
  successfulJobsHistoryLimit: 2
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: heartbeat
              image: busybox:1.36
              command: ["sh", "-c", "echo \"Heartbeat at $(date)\""]
EOF

The CronJob is named heartbeat and uses the schedule:

*/1 * * * *

This cron expression requests execution every minute.

The Job template uses a BusyBox container that prints a timestamp. Unlike the report Job, which required three successful completions, this CronJob creates a new Job each time its schedule is triggered.

The history settings control how many finished Jobs are retained by the CronJob controller:

successfulJobsHistoryLimit: 2 retains up to two successful Jobs.
failedJobsHistoryLimit: 1 retains up to one failed Job.

The Pod template uses restartPolicy: OnFailure, allowing a failed container to be restarted within its Pod according to Kubernetes' restart behavior.

I applied the manifest:

kubectl apply -f manifests/09-cronjob.yaml

Kubernetes confirmed:

cronjob.batch/heartbeat created

7. Verify Scheduled Executions

I inspected the CronJob:

kubectl get cronjob heartbeat

The output showed the every-minute schedule and the CronJob's status fields, including its suspension state, active Job count, and last scheduled execution.

The initial check occurred before the first scheduled run had been recorded, so the LAST SCHEDULE field was still empty.

After subsequent checks, I listed Jobs in creation order:

kubectl get jobs --sort-by=.metadata.creationTimestamp

The output included the completed report Job and a completed Job created by the heartbeat CronJob.

To inspect the latest scheduled Job's output, I ran:

kubectl logs job/$(kubectl get jobs --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')

The output was:

Heartbeat at Mon Oct 26 12:34:01 UTC 2026

This confirmed that the scheduled workload had executed and produced its expected message.

The naming pattern also makes the controller relationship visible: the CronJob is named heartbeat, while the Jobs it creates include a generated suffix.

8. Inspect the Combined Cluster State

After deploying the DaemonSet, Job, and CronJob, I inspected the broader set of resources:

kubectl get all,configmap,secret,pvc

ls manifests

The manifests folder contains nine yaml files.

The output showed the different workload types operating in the k8s-lab namespace.

The relevant resources included:

Resource	Observed state
`node-agent` DaemonSet	One desired and ready agent Pod after returning the node pool to one node
`report` Job	Complete, with `3/3` successful completions
`heartbeat` CronJob	Configured to run every minute
`heartbeat` Jobs	Completed executions recorded
`store` StatefulSet	`2/2` Pods ready
`web` Deployment	`3/3` replicas available
`web-svc` Service	Internal `ClusterIP`
`web-public` Service	`LoadBalancer` with external IP `20.61.154.190`
`store` Service	Headless Service with no cluster IP
StatefulSet PVCs	Two claims bound, each with `1Gi` capacity

9. DaemonSets, Jobs and CronJobs: Key Differences

Although all three resources can create and manage Pods, they serve distinct purposes.

Resource	Primary behavior	Typical use
DaemonSet	Maintains a Pod on each eligible node	Node-level agents, monitoring, logging, and other node-scoped services
Job	Runs Pods until a finite task reaches its required successful completion count	Batch processing, report generation, and one-off operations
CronJob	Creates Jobs according to a recurring schedule	Scheduled reports, periodic maintenance, and recurring automation

The choice depends on the execution model.

A DaemonSet follows the eligible node count. A Job tracks successful completion of finite work. A CronJob adds a time-based schedule for creating those Jobs.

These resources are complementary rather than interchangeable.

10. Operational Takeaways

This mission demonstrated several useful Kubernetes operations:

Node-level coverage: A DaemonSet automatically placed an agent on each eligible node, including the second node added during the test.
Runtime visibility: The Downward API supplied the node name to the agent without hardcoding infrastructure details into the container command.
Finite task execution: The report Job completed three successful tasks and exposed their outputs through logs.
Recurring automation: The heartbeat CronJob created scheduled Jobs that printed timestamps.
Execution verification: Job completion counts and container logs provided separate ways to verify task outcomes.
Compute management: Scaling the AKS node pool demonstrated the relationship between node count and DaemonSet placement.
Resource awareness: The final resource listing showed how node-level, batch, scheduled, stateless, and stateful workloads coexist in the same cluster.

Conclusion

The node-agent DaemonSet was deployed and verified on the AKS node pool. Scaling the pool to two nodes resulted in one agent Pod on each node, and the pool was subsequently returned to one node.

The report Job completed all three required executions, with logs confirming that each Pod generated its report message. The heartbeat CronJob also ran successfully on its configured every-minute schedule, producing timestamped output.

Together, these implementations demonstrate how Kubernetes handles node-scoped agents, finite batch work, and recurring tasks through dedicated controllers.

In Part 8, the series will continue with the next mission, building on the workload and operational capabilities established here.

This article is Part 7 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads - StatefulSets and Persistent Storage (Part 6/9)

Rahimah Sulayman — Mon, 26 Oct 2026 18:38:17 +0000

Introduction

Applications that maintain data need more than a reliable way to run containers. They also need predictable identities, stable network addresses, and storage that remains available when Pods are replaced.

In this mission, I deployed a StatefulSet on Azure Kubernetes Service (AKS), paired it with a headless Service, and provisioned persistent Azure Disk volumes for its replicas. I then wrote data from inside a Pod, deleted that Pod, and verified that the replacement could still access the data. Finally, I tested the StatefulSet's DNS identity from another Pod in the cluster.

The objective was to demonstrate how Kubernetes supports stateful workloads whose identity and data must survive individual Pod replacement.

What This Part Covers

Inspecting the available Kubernetes StorageClasses.
Creating a headless Service.
Defining a StatefulSet with two replicas.
Provisioning persistent storage through managed-csi.
Verifying Pod identities and PersistentVolumeClaims.
Writing data to a Pod's persistent volume.
Deleting and waiting for a StatefulSet Pod to return.
Verifying that data survives Pod replacement.
Testing stable DNS access through the headless Service.

1. Inspect the Available StorageClasses

Before defining the StatefulSet, I inspected the StorageClasses available in the cluster:

kubectl get storageclass

The output showed several StorageClasses for Azure Files and Azure Disk. The managed-csi StorageClass was available for persistent disk provisioning.

Its relevant properties included:

Property	Value
StorageClass	`managed-csi`
Provisioner	`disk.csi.azure.com`
Reclaim policy	`Delete`
Volume binding mode	`WaitForFirstConsumer`
Volume expansion	Enabled

The managed-csi StorageClass uses the Azure Disk Container Storage Interface (CSI) driver. It allows Kubernetes to provision Azure-managed disks for workloads that request persistent storage.

The WaitForFirstConsumer binding mode delays volume binding and provisioning until a Pod using the claim is scheduled. This allows storage provisioning to account for the Pod's scheduling requirements.

The reclaim policy is also important. Delete means that a dynamically provisioned volume can be deleted when its PersistentVolume is released, depending on the lifecycle of the associated storage resources. Deleting an individual StatefulSet Pod does not, by itself, release its PersistentVolumeClaim.

2. Define a Headless Service and StatefulSet

I created manifests/06-statefulset.yaml to define both the headless Service and the StatefulSet.

cat << 'EOF' > manifests/06-statefulset.yaml
apiVersion: v1
kind: Service
metadata:
  name: store
spec:
  clusterIP: None
  selector:
    app: store
  ports:
    - port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: store
spec:
  serviceName: store
  replicas: 2
  selector:
    matchLabels:
      app: store
  template:
    metadata:
      labels:
        app: store
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              cpu: 200m
          volumeMounts:
            - name: data
              mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes:
          - ReadWriteOnce
        storageClassName: managed-csi
        resources:
          requests:
            storage: 1Gi
EOF

Why use a headless Service?

The Service is configured with:

clusterIP: None

This makes it a headless Service. Instead of providing a conventional cluster IP, it enables DNS records to resolve directly to the endpoints of the selected Pods.

The Service selects Pods labelled app: store and exposes port 80. It also provides the network identity associated with the StatefulSet.

Why use a StatefulSet?

The StatefulSet is named store and declares two replicas:

serviceName: store
replicas: 2

Unlike ordinary Deployment-managed Pods, StatefulSet Pods have stable ordinal identities. The two replicas use these names:

store-0
store-1

The StatefulSet also uses the store headless Service as its governing Service, allowing the Pods to have predictable DNS identities.

Persistent storage

The manifest defines a volumeClaimTemplates entry named data. Each replica receives its own PersistentVolumeClaim requesting 1Gi of storage from the managed-csi StorageClass.

The important settings are:

accessModes:
  - ReadWriteOnce
storageClassName: managed-csi
resources:
  requests:
    storage: 1Gi

The container mounts the volume at:

/usr/share/nginx/html

This is the directory NGINX serves as its default document root. Writing a file there allows the test to demonstrate whether the contents remain available after a Pod is replaced.

Because each StatefulSet replica receives its own claim, store-0 and store-1 do not share the same per-replica volume.

3. Deploy the StatefulSet

I applied the manifest:

kubectl apply -f manifests/06-statefulset.yaml

Kubernetes confirmed that both resources were created:

service/store created
statefulset.apps/store created

I then monitored the StatefulSet rollout:

kubectl rollout status statefulset/store

The command reported:

Waiting for 1 pods to be ready...
partitioned roll out complete: 2 new pods have been updated...

After the rollout, I inspected the StatefulSet, Pods, and PersistentVolumeClaims:

kubectl get statefulset,pods,pvc -l app=store

The two Pods were running:

NAME        READY   STATUS    RESTARTS
pod/store-0 1/1     Running   0
pod/store-1 1/1     Running   0

The PersistentVolumeClaims were also bound:

NAME                    STATUS   CAPACITY   ACCESS MODES   STORAGECLASS
persistentvolumeclaim/data-store-0   Bound   1Gi   RWO   managed-csi
persistentvolumeclaim/data-store-1   Bound   1Gi   RWO   managed-csi

This verified that the StatefulSet had created both replicas and Kubernetes had provisioned a separate 1Gi persistent volume claim for each one.

The stable Pod names and individual claims are central to the StatefulSet model: a replacement for store-0 is associated with the identity and storage claim belonging to store-0, rather than being treated as an entirely unrelated replica.

4. Write Data to the Persistent Volume

To test persistence, I wrote a message to the NGINX document root inside store-0:

kubectl exec store-0 -- sh -c 'echo "Hello from $(hostname), written at $(date)" > /usr/share/nginx/html/index.html'

The command writes a message containing the Pod's hostname and the current date into index.html.

Because the document root is backed by the persistent volume mounted at /usr/share/nginx/html, the test data is stored on the volume rather than only in the container's writable layer.

This creates a clear verification point: after the Pod is replaced, the file should still contain the original message if the same persistent claim is attached successfully.

5. Delete the Pod and Wait for Replacement

I deleted store-0 to test the StatefulSet's recovery behavior:

kubectl delete pod store-0

Kubernetes confirmed that the Pod had been deleted from the k8s-lab namespace.

The StatefulSet controller then worked to restore the desired replica count. Because store-0 is part of a StatefulSet, Kubernetes recreates the Pod with the same ordinal identity and reuses its associated PersistentVolumeClaim.

I waited for the replacement Pod to become ready:

kubectl wait --for=condition=Ready pod/store-0 --timeout=180s

The command returned:

pod/store-0 condition met

This confirmed that store-0 had returned to the Ready state.

6. Verify That the Data Survived

With store-0 ready again, I inspected the file:

kubectl exec store-0 -- cat /usr/share/nginx/html/index.html

The output was:

Hello from store-0, written at Mon Oct 26 09:56:05 UTC 2026

The message remained available after the original Pod had been deleted and replaced.

This confirmed the key behavior being tested: the replacement Pod retained access to the data stored on the PersistentVolume associated with store-0.

The Pod itself is replaceable, but its identity and associated persistent storage allow the application to recover without losing the file created during the test.

This does not mean that every storage configuration automatically survives every failure scenario. Data persistence depends on the volume lifecycle, reclaim policy, storage backend, and how the claims and volumes are managed. In this test, the StatefulSet's per-replica claim remained available through Pod replacement.

7. Verify Stable DNS from Another Pod

The headless Service also provides predictable network identities for the StatefulSet Pods.

To verify DNS access, I launched a temporary BusyBox Pod and requested the content through the hostname store-0.store:

kubectl run dns-test --rm -i --restart=Never \
  --image=busybox:1.36 -- wget -qO- http://store-0.store

The response contained:

Hello from store-0, written at Mon Oct 26 09:56:05 UTC 2026

The temporary Pod returned the same content written before store-0 was deleted.

This verified two aspects of the implementation:

The name store-0.store resolved and allowed access to the NGINX endpoint.
The endpoint served the data that persisted across the Pod replacement.

The store-0.store hostname is derived from the StatefulSet Pod identity and its governing headless Service. This predictable naming is particularly useful for applications that need to communicate with specific replicas.

The temporary dns-test Pod was configured with --rm, so Kubernetes removed it when the command completed.

8. StatefulSet vs. Deployment

StatefulSets and Deployments both manage replicated workloads, but they address different operational requirements.

Capability	Deployment	StatefulSet
Pod identity	Typically interchangeable Pod names	Stable ordinal names such as `store-0`
Stable per-replica storage	Not provided automatically through a Deployment's usual model	Supported through `volumeClaimTemplates`
Common use cases	Stateless web applications and APIs	Stateful applications requiring stable identity or storage
Scaling and replacement	Reconciles desired replica count	Reconciles replicas while preserving ordinal identity
Networking	Commonly accessed through a Service	Can use a headless Service for per-Pod DNS identities

A StatefulSet is not automatically the right choice for every application that writes files. The appropriate workload resource depends on whether the application needs stable identity, ordered operations, or persistent storage tied to individual replicas.

In this mission, those features made the StatefulSet a suitable way to demonstrate data persistence across Pod replacement.

9. Operational Takeaways

This mission demonstrated how StatefulSets, headless Services, and persistent storage work together on AKS.

The main takeaways were:

Stable workload identity: StatefulSet replicas have predictable names that remain associated with their ordinals when Pods are replaced.
Persistent storage: Each replica receives its own PersistentVolumeClaim through volumeClaimTemplates.
Dynamic provisioning: The managed-csi StorageClass allows Kubernetes to provision Azure Disk storage for the claims.
Pod recovery: Deleting store-0 caused the StatefulSet controller to recreate the same logical replica.
Data continuity: The file written before deletion remained available after the replacement Pod became ready.
Stable DNS: A headless Service enables per-Pod DNS identities, allowing another Pod to access store-0 through store-0.store.
Storage lifecycle awareness: Persistent volume behavior depends on the claim and volume lifecycle, access modes, storage class, and reclaim policy—not only on the Pod resource.

Conclusion

The store StatefulSet was deployed with two replicas, a headless Service, and separate 1Gi persistent volume claims backed by the managed-csi StorageClass.

I wrote a file to store-0, deleted the Pod, waited for the replacement to become ready, and verified that the original file content remained available. A request to http://store-0.store from a temporary Pod returned the same content, confirming that the stable DNS identity and persistent storage were working together as intended.

This is the primary distinction between simply recreating a container and managing a stateful workload: the replacement can recover with its expected identity and retain access to its data.

In Part 7, the series will continue with the next mission, building on the workload and storage capabilities established here.

This article is Part 6 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads - Services, ConfigMaps and Secrets (Part 5/9)

Rahimah Sulayman — Mon, 26 Oct 2026 18:12:33 +0000

Introduction

A running application needs more than healthy Pods. It also needs a reliable way to receive traffic and a secure, maintainable approach to handling configuration.

In this mission, I extended the NGINX Deployment on Azure Kubernetes Service (AKS) by creating internal and external Services, testing application access, and separating configuration values from the container image. I also created a Kubernetes Secret, verified its stored value, and injected both ConfigMap and Secret data into the running Deployment as environment variables.

The implementation demonstrates how Kubernetes networking and configuration resources work together to make workloads accessible and configurable without rebuilding their images.

What This Part Covers

Creating a ClusterIP Service for internal application access.
Testing the NGINX application inside the cluster.
Exposing the Deployment through a LoadBalancer Service.
Verifying the public endpoint.
Creating a ConfigMap from literal values.
Creating a Secret for a sensitive configuration value.
Injecting ConfigMap and Secret values into the Deployment.
Verifying the environment variables inside a running container.

1. Create a ClusterIP Service

I started by defining an internal Service in manifests/03-service-clusterip.yaml.

cat << 'EOF' > manifests/03-service-clusterip.yaml
apiVersion: v1
kind: Service
metadata:
  name: web-svc
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80
EOF

The manifest defines a Service named web-svc with the following configuration:

type: ClusterIP creates an internal Service address that is reachable from within the cluster.
selector: app: web identifies the Pods that receive traffic through the Service.
port: 80 specifies the port exposed by the Service.
targetPort: 80 specifies the destination port on the selected Pods.

The selector connects the Service to the web Deployment's Pods, which carry the app: web label.

This is an important distinction: clients communicate with the Service rather than depending on an individual Pod IP address. Kubernetes maintains the Service's routing to the eligible backend Pods as the workload changes.

I applied the manifest:

kubectl apply -f manifests/03-service-clusterip.yaml

The Service provides a stable internal endpoint for the application. It does not, by itself, expose the application directly to the public internet.

2. Test Internal Application Access

After configuring the internal Service, I tested access to the NGINX application from inside the Kubernetes environment.

kubectl run curl-test --rm -i --restart=Never --image=busybox:1.36 -- wget -qO- http://web-svc

It starts a temporary Pod called curl-test inside the cluster, using the small busybox image. The Pod runs wget to fetch http://web-svc, and the Service’s name resolves through CoreDNS. --rm deletes the Pod afterwards, -i shows the output, and --restart=Never stops Kubernetes from restarting it once it finishes. The command returned the default “Welcome to nginx!” HTML, which showed the app was reachable through the ClusterIP Service from inside the cluster.

<title>Welcome to nginx!</title>

The response also contained the page's standard message confirming that the NGINX web server was installed and running.

This verified that the application could respond to an HTTP request from within the cluster.

Testing internal connectivity before exposing a workload externally is useful because it separates application and internal networking checks from the additional configuration required for public access.

3. Expose the Application with a LoadBalancer Service

For external access, I created a second Service in manifests/04-service-loadbalancer.yaml.

cat << 'EOF' > manifests/04-service-loadbalancer.yaml
apiVersion: v1
kind: Service
metadata:
  name: web-public
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80
EOF

The Service uses the same app: web selector as the internal Service, so it routes traffic to the Pods managed by the Deployment.

The key difference is:

type: LoadBalancer

On AKS, this Service type requests an Azure load balancer configuration that can expose the application through an external IP address.

I applied the manifest:

kubectl apply -f manifests/04-service-loadbalancer.yaml

Kubernetes confirmed that the Service was created:

service/web-public created

I then watched the Service status:

kubectl get service web-public --watch

The output showed:

NAME         TYPE           CLUSTER-IP   EXTERNAL-IP      PORT(S)
web-public   LoadBalancer   10.0.73.23   20.61.154.190   80:31236/TCP

The assigned addresses and port information were:

Cluster IP: 10.0.73.23
External IP: 20.61.154.190
Service port: 80
NodePort: 31236

The external IP made the NGINX application reachable through the load balancer.

Verify the public endpoint

I tested the external endpoint using curl:

curl -s http://$(kubectl get service web-public -o jsonpath='{.status.loadBalancer.ingress[0].ip}') | grep title

The response was:

<title>Welcome to nginx!</title>

This confirmed that the request reached the NGINX application through the public Service endpoint.

A LoadBalancer Service is useful when an application needs an externally reachable endpoint. However, public exposure should be deliberate: production workloads may also require network restrictions, TLS termination, ingress routing, and appropriate access controls.

4. Create a ConfigMap

Application configuration should not always be embedded in a container image. Kubernetes ConfigMaps provide a way to store non-sensitive configuration separately from application code and image builds.

I generated a ConfigMap manifest from literal values:

kubectl create configmap web-config \
  --from-literal=APP_ENV=production \
  --from-literal=WELCOME_MSG="Hello from AKS" \
  --dry-run=client -o yaml > manifests/05-configmap.yaml

The command generated the YAML locally rather than immediately creating the resource in the cluster. The output was saved to manifests/05-configmap.yaml.

The configuration contained two values:

Key	Value
`APP_ENV`	`production`
`WELCOME_MSG`	`Hello from AKS`

I then applied the generated manifest:

kubectl apply -f manifests/05-configmap.yaml

Kubernetes confirmed:

configmap/web-config created

To inspect the stored configuration, I ran:

kubectl get configmap web-config -o yaml

The output included:

data:
  APP_ENV: production
  WELCOME_MSG: Hello from AKS

This verified that the ConfigMap contained the expected configuration values.

Because the ConfigMap is separate from the container image, these values can be managed independently of the image build. ConfigMaps are intended for non-sensitive data and should not be used to store credentials or other confidential values.

5. Create a Kubernetes Secret

For the next configuration value, I created a Kubernetes Secret named web-secret.

kubectl create secret generic web-secret \
  --from-literal=API_KEY=change-me-123

Kubernetes returned:

secret/web-secret created

The Secret contains the key API_KEY, with a placeholder value used in this lab.

Unlike a ConfigMap, a Secret is designed for sensitive configuration data. However, creating a Kubernetes Secret does not automatically make the value safe: access controls, encryption at rest, and appropriate secret-management practices still matter.

Verify the stored value

I verified the value using:

kubectl get secret web-secret -o jsonpath='{.data.API_KEY}' | base64 --decode; echo

The output was:

change-me-123

Kubernetes stores Secret data in a base64-encoded representation in the API object. Base64 is an encoding mechanism, not encryption. Anyone authorized to retrieve and decode the value can read it.

The value used here is only a lab placeholder. Real API keys and production credentials should not be committed to a repository or exposed in terminal output, screenshots, or documentation.

6. Inject the ConfigMap into the Deployment

With the ConfigMap and Secret created, I configured the web Deployment to consume their values as environment variables.

First, I added the ConfigMap values:

kubectl set env deployment/web --from=configmap/web-config

Kubernetes confirmed:

deployment.apps/web env updated

This added the ConfigMap's key-value pairs to the container environment defined by the Deployment.

The values supplied by web-config were:

APP_ENV=production
WELCOME_MSG=Hello from AKS

7. Inject the Secret into the Deployment

I then added the Secret value to the Deployment:

kubectl set env deployment/web --from=secret/web-secret

Kubernetes returned:

deployment.apps/web env updated

This configured the container environment to include API_KEY from web-secret.

Changing the Deployment's Pod template triggered a rollout. I verified that the rollout completed:

kubectl rollout status deployment/web

The output confirmed:

deployment "web" successfully rolled out

The new Pods were created using the updated Deployment configuration.

Verify the environment variables

To verify that the values were available inside a running container, I executed:

kubectl exec deploy/web -- printenv APP_ENV WELCOME_MSG API_KEY

The output was:

production
Hello from AKS
change-me-123

This confirmed that the container received the values from both Kubernetes configuration resources.

The ConfigMap supplied the non-sensitive application configuration, while the Secret supplied the placeholder API key. Both were injected into the container environment without rebuilding the NGINX image.

For real workloads, avoid printing secrets to logs or terminal output unnecessarily. When applications support it, mounting secrets as files or retrieving credentials from an external secret manager may be more appropriate, depending on the security requirements.

8. ClusterIP vs. LoadBalancer

The two Services created in this mission serve different networking purposes.

Feature	`ClusterIP`	`LoadBalancer`
Primary purpose	Internal application access	External application access
Default reachability	Within the cluster	Through an external load balancer when provisioned
Stable Service endpoint	Yes	Yes
Typical use	Internal APIs and service-to-service traffic	Applications that require a public or externally reachable endpoint

Both Services use label selectors to route traffic to the appropriate Pods. The choice of Service type depends on where the clients are located and how the application should be exposed.

9. Operational Takeaways

This mission brought together Kubernetes networking and configuration management.

The main operational takeaways were:

Stable internal networking: A ClusterIP Service provides an internal endpoint independent of individual Pod IP addresses.
External access: A LoadBalancer Service can expose an application through an Azure-managed external endpoint.
Separation of configuration: ConfigMaps allow non-sensitive settings to be managed separately from the container image.
Secret handling: Kubernetes Secrets provide a dedicated resource for sensitive values, but they still require appropriate security controls.
Environment injection: ConfigMap and Secret values can be supplied to containers through the Deployment's environment configuration.
Rollout verification: Updating the Deployment's Pod template triggers a rollout, which can be monitored with kubectl rollout status.
Connectivity testing: Testing both internal and external access verifies different parts of the application networking path.

Conclusion

The web Deployment was made accessible through an internal ClusterIP Service and an external LoadBalancer Service. The public endpoint returned the NGINX welcome page, confirming that external access was working.

I also created web-config and web-secret, injected their values into the Deployment, and verified the resulting environment variables inside a running container.

The result is a workload with a stable networking interface and configuration managed independently of its container image. These patterns form an important foundation for applications that need controlled connectivity and maintainable configuration on AKS.

In Part 6, the series will continue with the next mission, building on the networking and configuration resources established here.

This article is Part 5 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads -Deployments, Self-Healing, Scaling and Rolling Updates (Part 4/9)

Rahimah Sulayman — Mon, 26 Oct 2026 17:05:48 +0000

Introduction

Running a container in Kubernetes is only the starting point. Production workloads also need mechanisms for maintaining the desired number of replicas, recovering from Pod failures, scaling with demand, and updating application versions without unnecessary disruption.

In this mission, I used a Kubernetes Deployment to manage an NGINX workload on Azure Kubernetes Service (AKS). I configured three replicas, added health probes and resource constraints, verified the rollout, deleted a Pod to observe self-healing, scaled the Deployment, updated the container image, inspected rollout history, and rolled back to the previous version.

What This Part Covers

Defining a Deployment with three NGINX replicas.
Configuring resource requests, limits, and health probes.
Verifying the rollout and associated ReplicaSet.
Observing automatic Pod replacement after deletion.
Scaling the Deployment from three replicas to five and back.
Updating the NGINX image.
Inspecting rollout history and recording the change cause.
Rolling back to the previous Deployment revision.

1. Define the Deployment Manifest

I created the Deployment manifest at manifests/02-deployment.yaml.

cat << 'EOF' > manifests/02-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  labels:
    app: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              cpu: 200m
              memory: 128Mi
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 3
            periodSeconds: 10
EOF

The manifest defines a Deployment named web, using the apps/v1 API and the app: web label to identify its Pods.

Replica management

replicas: 3
selector:
  matchLabels:
    app: web

The Deployment declares three desired replicas. Its selector matches Pods labelled app: web, while the Pod template applies that label to each replica it creates.

This relationship allows the Deployment controller to manage the workload through a ReplicaSet, which maintains the required number of Pods.

Rolling update strategy

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0

The update strategy controls how Kubernetes replaces old Pods during an update.

maxSurge: 1 allows one additional Pod above the desired replica count during a rollout.
maxUnavailable: 0 prevents the Deployment from intentionally reducing the number of available replicas below the desired count during the rolling update.

These settings prioritize availability during updates, provided the cluster has sufficient capacity and the replacement Pods can become ready.

Container resources and health checks

The NGINX container uses the nginx:1.27 image and exposes container port 80.

Its resource settings are:

Resource	Request	Limit
CPU	`50m`	`200m`
Memory	`64Mi`	`128Mi`

The manifest also defines a readiness probe that sends an HTTP request to / on port 80. It begins checking after three seconds and repeats every ten seconds.

A readiness probe helps Kubernetes determine whether a container is ready to receive traffic. The configuration also establishes resource boundaries for each replica, making resource expectations explicit.

2. Apply the Deployment and Verify the Rollout

I applied the manifest to the cluster:

kubectl apply -f manifests/02-deployment.yaml

Kubernetes returned:

deployment.apps/web created

I then monitored the rollout:

kubectl rollout status deployment/web

The command confirmed:

deployment "web" successfully rolled out

To inspect the Deployment, ReplicaSet, and Pods together, I ran:

kubectl get deployment,replicaset,pods -l app=web

The results showed:

NAME                 READY   UP-TO-DATE   AVAILABLE
deployment.apps/web  3/3     3            3

NAME                     DESIRED   CURRENT   READY
replicaset.apps/web-5b77b547f8  3   3         3

NAME                          READY   STATUS
pod/web-5b77b547f8-5jwm5       1/1     Running
pod/web-5b77b547f8-5tzd h      1/1     Running
pod/web-5b77b547f8-qpzqh       1/1     Running

The important result was that the Deployment had three available replicas, the ReplicaSet maintained three Pods, and all three Pods were running.

The Deployment and ReplicaSet provide different levels of workload management. The Deployment manages rollout and revision history, while the ReplicaSet maintains the required number of matching Pods.

Together, they provide more control over application availability than a standalone Pod.

3. Verify Self-Healing by Deleting a Pod

To test the Deployment controller's reconciliation behavior, I deleted one of the Pods selected by the app=web label.

I used the following command to identify a Pod and pass its name to kubectl delete:

kubectl delete pod $(kubectl get pods -l app=web -o jsonpath='{.items[0].metadata.name}')

Kubernetes confirmed that the selected Pod had been deleted from the k8s-lab namespace.

I then checked the Pods again:

kubectl get pods -l app=web

The output showed three running Pods. A replacement had been created to maintain the Deployment's desired replica count.

This is the Deployment controller's reconciliation behavior in action. The desired state remained three replicas, so Kubernetes worked to restore that state after one Pod was removed.

The replacement Pod did not need to retain the deleted Pod's name. Kubernetes created a new Pod from the Deployment's template through its ReplicaSet.

Operational takeaway: Deleting a Pod managed by a Deployment does not normally reduce the workload's desired replica count. The controller works to maintain that count.

4. Scale the Deployment

The next step was to change the desired replica count.

I scaled the Deployment from three replicas to five:

kubectl scale deployment web --replicas=5

Kubernetes returned:

deployment.apps/web scaled

To verify the result and inspect the Pod placement, I ran:

kubectl get pods -l app=web -o wide

The output showed five running Pods, each with a Pod IP and placement on the AKS node.

I then scaled the Deployment back to three replicas:

kubectl scale deployment web --replicas=3

This returned:

deployment.apps/web scaled

Scaling changes the number of Pod replicas managed by the Deployment. Scaling out from three to five increased the number of running instances, while scaling back to three reduced the desired replica count.

This operation was performed manually with kubectl scale. It was not an automatic response to CPU utilization or incoming traffic; automatic scaling requires additional configuration, such as a Horizontal Pod Autoscaler.

5. Update the NGINX Container Image

After verifying replica management, I updated the Deployment to use the nginx:1.27-alpine image:

kubectl set image deployment/web nginx=nginx:1.27-alpine

Kubernetes confirmed:

deployment.apps/web image updated

The new image uses the Alpine-based NGINX image variant. Changing the image in a Deployment triggers a rollout, allowing Kubernetes to replace Pods according to the configured update strategy.

I then annotated the Deployment with the reason for the change:

kubectl annotate deployment/web kubernetes.io/change-cause="switch to nginx 1.27-alpine"

The output confirmed:

deployment.apps/web annotated

Recording the change cause makes the intention behind a revision easier to identify when reviewing rollout history.

6. Verify the Rollout and Inspect Revision History

I checked the status of the updated Deployment:

kubectl rollout status deployment/web

The rollout completed successfully:

deployment "web" successfully rolled out

I then inspected the revision history:

kubectl rollout history deployment/web

The output showed two revisions:

deployment.apps/web

REVISION  CHANGE-CAUSE
1         <none>
2         switch to nginx 1.27-alpine

Revision 1 represented the original Deployment configuration, while revision 2 recorded the change to the Alpine image.

The history provides an operational record of Deployment revisions and their associated change causes. This can help identify which version is currently deployed and which revision to restore if an update needs to be reversed.

7. Roll Back to the Previous Revision

To demonstrate rollback, I reverted the Deployment to its previous revision:

kubectl rollout undo deployment/web

Kubernetes returned:

deployment.apps/web rolled back

The command also displayed a warning explaining that the Deployment had previously been managed with kubectl apply. A rollback does not update the kubectl.kubernetes.io/last-applied-configuration annotation, which can affect the behavior of future kubectl apply operations. Kubernetes recommended applying the previous configuration file when appropriate.

This warning is worth noting when mixing imperative changes, such as kubectl set image, with declarative manifest management. The live Deployment, its rollout history, and the last-applied configuration annotation are related, but they are not interchangeable.

To verify the image configured in the Deployment after the rollback, I ran:

kubectl get deployment web -o jsonpath='{.spec.template.spec.containers[0].image}'; echo

The output was:

nginx:1.27

This confirmed that the Deployment's Pod template had returned to the original image.

The rollback demonstrated how Deployment revision history can be used to reverse a change without manually recreating the entire workload.

8. Operational Takeaways

This mission demonstrated several core capabilities used to manage Kubernetes applications:

Desired-state management: A Deployment maintains the declared number of replicas through its controller and ReplicaSet.
Self-healing: When a managed Pod is deleted, Kubernetes creates a replacement to restore the desired replica count.
Scaling: The replica count can be increased or reduced to change the number of running application instances.
Health checks: Readiness probes help Kubernetes determine whether a container is ready to receive traffic.
Rolling updates: A Deployment can replace Pods gradually according to its update strategy.
Revision history: Rollout history provides a record of Deployment revisions and their recorded change causes.
Rollback: A previous Deployment revision can be restored when an update needs to be reversed.
Configuration management: Mixing imperative updates with declarative kubectl apply requires attention to the last-applied configuration and future changes.

These capabilities provide a stronger operational foundation than managing individual Pods directly. They make application changes more controlled and help maintain the intended state of a workload.

Conclusion

The web Deployment was successfully created with three NGINX replicas, and its rollout was verified. I confirmed that Kubernetes replaced a deleted Pod, scaled the Deployment to five replicas and back to three, updated the image to nginx:1.27-alpine, reviewed the rollout history, and restored the original nginx:1.27 image through rollback.

The key distinction is that a Deployment manages the desired state of an application rather than relying on individual Pods to remain unchanged. Replica management, health checks, rolling updates, and rollback together provide essential tools for operating workloads reliably on AKS.

In Part 5, the series will continue with the next mission, building on the workload-management capabilities established here.

This article is Part 4 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads - Pods, the Smallest Unit (Part 3/9)

Rahimah Sulayman — Mon, 26 Oct 2026 16:42:24 +0000

Introduction

A Kubernetes Pod is the smallest deployable unit in Kubernetes. It provides the execution environment for one or more containers, along with shared networking and storage resources.

In this mission, I deployed a standalone NGINX Pod to my Azure Kubernetes Service (AKS) cluster, defined its resource requirements, inspected its runtime state, reviewed its logs, accessed the application locally, and examined what happens when a standalone Pod is deleted.

The focus was on the Pod lifecycle and the operational details that matter when running containerized workloads on Kubernetes.

What This Part Covers

Defining a Pod using a Kubernetes YAML manifest.
Deploying the Pod to the k8s-lab namespace.
Verifying its status, IP address, and assigned node.
Inspecting container configuration and resource constraints.
Reviewing NGINX logs and verifying its version.
Accessing the application through port forwarding.
Observing the behavior of a standalone Pod after deletion.

1. Define the Pod Manifest

I created the manifest at manifests/01-pod.yaml in the project repository.

cat << 'EOF' > manifests/01-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: web-pod
  labels:
    app: web-pod
spec:
  containers:
    - name: nginx
      image: nginx:1.27
      ports:
        - containerPort: 80
      resources:
        requests:
          cpu: 50m
          memory: 64Mi
        limits:
          cpu: 200m
          memory: 128Mi
EOF

The manifest defines a Pod named web-pod containing one NGINX container.

Key configuration details

apiVersion: v1 specifies the Kubernetes API version for the Pod resource.
kind: Pod identifies the resource being created.
metadata.name assigns the name web-pod.
metadata.labels attaches the app: web-pod label, which can be used to identify and select the Pod.
image: nginx:1.27 specifies the NGINX container image.
containerPort: 80 documents the port on which the container is expected to receive HTTP traffic.
CPU request: 50m declares 50 millicores of CPU as the container's requested compute capacity.
Memory request: 64Mi declares 64 mebibytes of requested memory.
CPU limit: 200m sets the container's CPU usage limit.
Memory limit: 128Mi sets the container's memory limit.

Resource requests and limits serve different purposes. Requests influence scheduling decisions, while limits constrain the resources a container can consume. CPU limits can result in throttling, whereas exceeding a container's memory limit can lead to termination.

Defining these values makes the workload's resource expectations explicit rather than leaving them entirely unspecified.

2. Deploy the Pod

With the manifest in place, I applied it to the cluster:

kubectl apply -f manifests/01-pod.yaml

Kubernetes confirmed that the resource had been created:

pod/web-pod created

I then checked the Pod's status and placement:

kubectl get pods -o wide

The output showed:

NAME      READY   STATUS    RESTARTS   AGE   IP           NODE
web-pod   1/1     Running   0          25s   10.244.0.139 aks-nodepool1-33590657-vmss000000

The important observations were:

Ready: 1/1 — the Pod's single container was ready.
Status: Running — the Pod had started successfully.
Restarts: 0 — no container restarts had occurred at the time of inspection.
Pod IP: 10.244.0.139 — Kubernetes assigned the Pod an internal IP address.
Node: aks-nodepool1-33590657-vmss000000 — the scheduler placed the Pod on this AKS worker node.

The -o wide option exposes additional placement and networking information beyond the default Pod listing.

This verified that the manifest had been accepted and the NGINX workload was running in the k8s-lab namespace.

3. Inspect the Pod Configuration and Runtime State

To examine the Pod in more detail, I ran:

kubectl describe pod web-pod

The output confirmed that web-pod was running in the k8s-lab namespace, with the nginx container using the nginx:1.27 image.

The inspection also exposed the container's runtime information, resource configuration, readiness conditions, assigned node, and recent events.

Container readiness

The Pod reported the following conditions as true:

Initialized
Ready
ContainersReady
PodScheduled

These conditions provide useful operational information. A Pod's existence alone does not guarantee that its application is ready to serve traffic. Checking readiness and container state helps distinguish a successfully scheduled workload from one that is actually running and ready.

Resource configuration

The container's reported resource settings matched the manifest:

Resource	Request	Limit
CPU	`50m`	`200m`
Memory	`64Mi`	`128Mi`

The events also recorded the scheduling, image-pulling, container-creation, and container-start operations.

The image was pulled successfully, and Kubernetes started the container without reporting a restart.

This inspection connected the desired configuration in the YAML manifest with the state Kubernetes was reporting for the running workload.

4. Review the NGINX Logs

I retrieved the container logs with:

kubectl logs web-pod

The output showed the NGINX container entrypoint initializing its configuration and launching the NGINX process.

The logs included messages indicating that configuration was complete and that NGINX was ready to start up. They also showed the NGINX worker processes starting.

Reviewing logs is an important part of troubleshooting containerized applications. They can reveal startup failures, configuration problems, and application-level errors that may not be obvious from the Pod status alone.

In this case, the startup output was consistent with a successfully initialized NGINX container.

5. Verify the NGINX Version

To inspect the NGINX version from inside the running container, I executed:

kubectl exec web-pod -- nginx -v

The command returned:

nginx version: nginx/1.27.5

This verified the version of NGINX running inside the container.

The manifest specified the nginx:1.27 image tag, while the running binary reported version 1.27.5. This illustrates why inspecting the runtime can provide useful information beyond the image tag declared in a manifest.

6. Access the Application with Port Forwarding

The Pod was running NGINX on container port 80. To access it from my local machine without creating a Kubernetes Service or exposing the application publicly, I used port forwarding:

kubectl port-forward pod/web-pod 8080:80

The terminal reported:

Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

This forwarded local port 8080 to port 80 on the Pod.

With the port-forwarding process running, I opened the following address in my browser:

http://localhost:8080

The browser displayed the default Welcome to nginx! page, confirming that the NGINX web server was accessible through the local port-forwarding connection.

Port forwarding provided a convenient way to test the application without creating a public endpoint. The connection remained dependent on the kubectl port-forward process, so it was useful for this verification rather than as a production traffic-routing solution.

Important: The port-forwarding command occupies the terminal while it is running. I kept that process active while accessing the page in the browser.

7. Delete the Standalone Pod

After verifying the application, I deleted the Pod:

kubectl delete pod web-pod

Kubernetes confirmed:

pod "web-pod" deleted from k8s-lab namespace

I then checked the namespace for remaining Pods:

kubectl get pods

The result was:

No resources found in k8s-lab namespace.

This confirmed that the standalone Pod had been removed and that no Pods remained in the namespace at the time of the check.

Why the Pod was not recreated

A standalone Pod does not have a higher-level workload controller responsible for maintaining a desired number of replicas.

Deleting it therefore removed the workload without automatically creating a replacement.

This behavior is an important distinction between running a Pod directly and managing workloads through resources such as Deployments or StatefulSets. Controllers can reconcile the actual state with the declared desired state, including replacing Pods when necessary.

For this mission, deleting the standalone Pod demonstrated its lifecycle and the absence of automatic replacement.

8. Operational Takeaways

This mission established a complete workflow for deploying and inspecting a basic Kubernetes workload.

The key operational points were:

Declarative configuration: A YAML manifest defines the desired Pod configuration and can be stored alongside the rest of the project.
Resource management: CPU and memory requests and limits establish explicit resource expectations for the container.
Runtime inspection: kubectl get pods -o wide and kubectl describe pod expose different levels of information about scheduling, readiness, networking, and container state.
Log inspection: kubectl logs helps reveal what happens during application startup.
Container-level verification: kubectl exec can be used to inspect the running container, including its software version.
Local application access: Port forwarding provides a practical way to test a Pod without creating a public endpoint.
Pod lifecycle: A standalone Pod is not automatically recreated after deletion because no higher-level controller manages its desired state.

Conclusion

The web-pod workload was successfully deployed to AKS, inspected at both the Kubernetes and container levels, accessed through local port forwarding, and subsequently deleted.

The exercise demonstrated how Kubernetes manifests translate into running workloads and how the Kubernetes CLI supports verification throughout the Pod lifecycle.

A standalone Pod is useful for understanding the fundamentals and performing targeted tests. For workloads that require resilience, scaling, and automatic replacement, higher-level controllers provide the management capabilities needed to maintain the desired state.

In the next part of this series, I will continue building on the AKS environment and explore the next workload-management concept in the lab.

This article is Part 3 of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads -Creating the AKS Cluster (Part 2/9)

Rahimah Sulayman — Mon, 26 Oct 2026 16:13:41 +0000

Introduction

Provisioning a Kubernetes cluster is the next step after preparing the development environment. In this part of the series, I moved from local tooling setup to creating an Azure Kubernetes Service (AKS) cluster, connecting kubectl to it, verifying the cluster, and preparing a dedicated namespace for workloads.

The objective was to establish a working Kubernetes environment on Azure that could support the deployments and experiments in the missions that follow.

What This Part Covers

Loading the lab configuration.
Creating an Azure resource group.
Provisioning an AKS cluster.
Retrieving cluster credentials and configuring kubectl.
Verifying the cluster and its system components.
Creating and selecting a Kubernetes namespace.

1. Load the Lab Configuration

I started from the project directory created in Part 1:

cd /mnt/c/Users/Admin/Desktop/labs/aks-kubernetes-lab

Before provisioning any Azure resources, I loaded the environment variables defined in lab.env:

source lab.env

The environment file provides the configuration used by the commands in this mission, including the resource group name, cluster name, Azure region, and node size.

Using variables instead of repeatedly hardcoding resource names makes the commands easier to maintain and reuse throughout the lab.

2. Check the Resource Group and Create It

Before creating the resource group, I checked whether it already existed:

az group show --name $RESOURCE_GROUP --query location --output tsv 2>/dev/null

The command queries the resource group's location. Redirecting standard error to /dev/null suppresses error messages if the resource group does not exist.

I then created the resource group:

az group create --name $RESOURCE_GROUP --location $LOCATION --output table

The command returned the following result:

Location    Name
----------  ----------
westeurope  rg-aks-lab

This confirmed that the resource group rg-aks-lab was available in westeurope.

An Azure resource group provides a logical container for related resources. In this lab, it serves as the main resource group for the AKS environment.

3. Provision the AKS Cluster

With the resource group created, I provisioned the AKS cluster using the Azure CLI:

az aks create \
  --resource-group $RESOURCE_GROUP \
  --name $CLUSTER_NAME \
  --location $LOCATION \
  --tier free \
  --node-count 1 \
  --node-vm-size $NODE_SIZE \
  --generate-ssh-keys \
  --output table

Here is what the main options do:

--resource-group specifies the Azure resource group that will contain the cluster.
--name sets the AKS cluster name.
--location specifies the Azure region.
--tier free selects the Free tier for AKS cluster management.
--node-count 1 configures one initial worker node.
--node-vm-size specifies the virtual machine size for the node.
--generate-ssh-keys generates SSH keys if required.
--output table presents the returned information in a table format.

The command displayed a notice that the aks-preview extension had altered its behavior. It also warned that the new node pool would enable SSH access and recommended using --ssh-access disabled to disable SSH access for better security.

The command subsequently returned cluster information, including the Kubernetes version, provisioning state, resource group, and cluster name.

The key result was:

ProvisioningState: Succeeded
ResourceGroup:     rg-aks-lab
Name:              aks-lab
Location:          westeurope

The provisioning state of Succeeded confirmed that Azure had completed the cluster creation operation.

4. Connect kubectl to the AKS Cluster

Creating the cluster is only part of the process. I also needed to configure my local Kubernetes client to communicate with it.

I retrieved the cluster credentials with:

az aks get-credentials \
  --resource-group $RESOURCE_GROUP \
  --name $CLUSTER_NAME \
  --overwrite-existing

The command returned:

Merged "aks-lab" as current context in /home/rahimahisah/.kube/config

This merged the AKS cluster credentials into the local kubeconfig file and selected the aks-lab context.

To confirm the active context, I ran:

kubectl config current-context

The output was:

aks-lab

This confirmed that kubectl was configured to use the AKS cluster context.

Kubernetes contexts allow a single kubeconfig file to hold connection details for multiple clusters. Checking the active context before running commands is a useful habit because it helps prevent accidentally targeting the wrong environment.

5. Verify the Kubernetes Cluster

After connecting kubectl, I checked the cluster's node status:

kubectl get nodes -o wide

The output showed:

NAME                                STATUS   ROLES    AGE     VERSION
aks-nodepool1-33590657-vmss000000   Ready    <none>   3m18s   v1.36.4

The node was in the Ready state, indicating that Kubernetes considered it ready to accept workloads.

The wider output also included information about the node's internal IP address, operating system image, kernel version, and container runtime.

The node was running Ubuntu 24.04.5 LTS, and the Kubernetes version shown was v1.36.4.

Inspect the cluster components

Next, I checked the cluster information:

kubectl cluster-info

The output confirmed that the Kubernetes control plane was running and displayed the endpoints for CoreDNS and the metrics server.

I then inspected the system pods in the kube-system namespace:

kubectl get pods --namespace kube-system

The output showed system components in the Running state, including:

azure-cns
azure-ip-masq-agent
cloud-node-manager
coredns
coredns-autoscaler
csi-azuredisk-node
csi-azurefile-node
kube-proxy
metrics-server
konnectivity-agent

These components support the operation of the cluster. For example, CoreDNS provides cluster DNS services, the CSI node components support storage integration, and the metrics server provides resource metrics used by Kubernetes tools and autoscaling components.

Checking the node, cluster information, and system pods provided an initial verification that the cluster was accessible and its core components were running.

Screenshot 4: Node status, cluster information, and system pods.

6. Create a Dedicated Kubernetes Namespace

With the cluster verified, I created a namespace for the lab workloads:

kubectl create namespace k8s-lab

Kubernetes confirmed the operation:

namespace/k8s-lab created

I then configured the current context to use the new namespace by default:

kubectl config set-context --current --namespace=k8s-lab

The output confirmed that the aks-lab context had been modified.

To verify the namespace configured in the current context, I ran:

kubectl config view --minify --output 'jsonpath={..namespace}'; echo

The result was:

k8s-lab

This means that subsequent kubectl commands that do not explicitly specify a namespace will use k8s-lab by default.

Namespaces provide a way to organize Kubernetes resources within a cluster. Creating a dedicated namespace keeps the lab workloads separate from system components running in kube-system.

7. What Learned

This mission connected the local tooling setup from Part 1 to a working Kubernetes environment hosted on Azure.

The key takeaways were:

Azure resource organization: Resource groups provide a logical boundary for related Azure resources.
Cluster provisioning: The Azure CLI can provision an AKS cluster using a repeatable set of configuration options.
Kubernetes authentication and context: Retrieving credentials allows kubectl to communicate with the selected cluster.
Cluster verification: Checking node readiness, cluster information, and system pods helps establish whether the environment is operational.
Namespace management: A dedicated namespace helps organize lab workloads and avoid mixing them with system resources.

One important habit I reinforced was verifying the active Kubernetes context before working with cluster resources. It is a small check that can help prevent mistakes when managing multiple environments.

Conclusion

The AKS cluster was provisioned successfully, the local Kubernetes client was connected to it, and the k8s-lab namespace was created and selected as the default namespace for the current context.

The environment is now ready for the next stage of the series, where I will begin working with Kubernetes resources and workloads.

In Part 3, the series will continue with the next mission in the lab, building on the cluster and namespace established here.

This article is part of the nine-part series, “Kubernetes on AKS: From Zero to Production Workloads.”

Kubernetes on AKS: From Zero to Production Workloads -Setting Up the Foundation (Part 1/9)

Rahimah Sulayman — Mon, 26 Oct 2026 15:31:25 +0000

Series: Kubernetes on AKS: From Zero to Production Workloads

Part 1: Workspace, Tools and Azure Sign-In

Environment: Ubuntu on WSL, VS Code terminal, Azure CLI, kubectl and Git

Before deploying workloads to Kubernetes, there is some groundwork to do. A reliable environment needs more than a cluster: it needs a structured project directory, version control, the right command-line tools, authenticated access to Azure, and the necessary resource providers.

In this first part of my nine-part series, I'll walk through how I prepared the environment for a hands-on Azure Kubernetes Service (AKS) lab. I'll cover the project workspace, Git initialization, Azure CLI installation, Azure authentication, Kubernetes command-line tools, and Azure resource provider registration. I'll also share an installation error I encountered, how I investigated it, and the workaround that allowed me to continue.

1. Creating the project workspace

I started by creating a dedicated directory for the lab and navigating into it.

mkdir -p /mnt/c/Users/Admin/Desktop/labs/aks-kubernetes-lab
cd /mnt/c/Users/Admin/Desktop/labs/aks-kubernetes-lab
pwd

Here's what each command does:

mkdir -p creates the project directory, including any missing parent directories.
cd moves into the directory.
pwd prints the current working directory so we can verify the location.

The terminal returned:

/mnt/c/Users/Admin/Desktop/labs/aks-kubernetes-lab

Confirming the working directory matters because subsequent commands create files and perform Git operations relative to that location.

A dedicated workspace keeps the project's documentation, Kubernetes manifests, and configuration organized as the lab grows.

2. Initializing Git

With the workspace ready, I initialized a Git repository using main as the initial branch.

git init -b main

The terminal confirmed:

Initialized empty Git repository in /mnt/c/Users/Admin/Desktop/labs/aks-kubernetes-lab/.git/

Git provides version control, allowing us to track changes, review previous versions, and maintain a history of the files created throughout the project.

The -b main option establishes main as the initial branch.

At this point, the repository existed locally. It had not yet been connected to a remote GitHub repository.

3. Creating the initial project structure

Next, I created a directory to hold the Kubernetes manifests.

mkdir -p manifests

I then created an initial README.md file:

printf '# Kubernetes on AKS\n\nProduction Kubernetes workloads on Azure Kubernetes Service, built as a hands-on lab.\n' > README.md

To inspect the file, I ran:

cat README.md

The output was:

# Kubernetes on AKS

Production Kubernetes workloads on Azure Kubernetes Service, built as a hands-on lab.

This README was intentionally minimal. It established the repository's entry point, while the manifests/ directory provided a dedicated location for Kubernetes YAML files.

The initial project structure looked like this:

aks-kubernetes-lab/
├── manifests/
└── README.md

This represents the initial structure, not the final repository layout.

4. Staging the README

Before creating the first commit, I staged the README and checked the repository status.

git add README.md
git status

Git reported:

On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   README.md

The output confirmed that README.md was staged and ready to be included in a commit.

Git's staging area lets us choose which changes belong in a commit. This is useful when a project contains multiple modifications but we want to record only a specific set of changes.

5. Adding `.gitignore` and creating the first commit

Before committing the initial files, I created a .gitignore file.

printf 'lab.env\n*.log\n' > .gitignore

Its contents were:

lab.env
*.log

The lab.env rule tells Git to ignore a file named lab.env, while *.log ignores files ending in .log.

This is a useful precaution when working with cloud infrastructure. Environment files may contain configuration values that should not be published, and log files can add unnecessary clutter to a repository.

Important: .gitignore does not remove files that have already been committed or staged. It is a preventive measure, not a replacement for proper secret management.

I staged the files and created the initial commit:

git add README.md .gitignore
git commit -m "chore: initialise AKS lab workspace"

Git confirmed the commit:

[main (root-commit) 7bdb9aa] chore: initialise AKS lab workspace
2 files changed, 5 insertions(+)
create mode 100644 .gitignore
create mode 100644 README.md

This established the first checkpoint in the repository's history. Subsequent changes could now be recorded as separate commits.

6. Installing Azure CLI

The next step was to install the Azure command-line interface (Azure CLI).

Azure CLI allows us to manage Azure resources from a terminal. It is particularly useful for repeatable infrastructure workflows, where commands can be documented and reused instead of relying exclusively on the Azure portal.

I ran the installation command provided by the lab:

curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

The command downloads and executes Microsoft's installation script for Debian-based Linux environments. Because it runs with elevated privileges, it is important to use a trusted installation source and understand what the command does.

The terminal showed the package repositories being updated and Azure CLI being installed.

I then verified the installation:

az version

The output included:

{
  "azure-cli": "2.91.0",
  "azure-cli-core": "2.91.0",
  "azure-cli-telemetry": "1.1.0"
}

This confirmed that Azure CLI was installed and available from the terminal.

7. Signing in to Azure

With Azure CLI installed, I authenticated using device-code sign-in.

az login --use-device-code

The CLI instructed me to open Microsoft's device-login page in a browser and enter the displayed code. After completing the browser authentication, I selected the available subscription.

The terminal showed the selected subscription as Azure subscription 1 under Default Directory.

Device-code authentication is useful when the terminal cannot conveniently open a browser or when authentication needs to be completed in a separate browser session.

Verifying the active account

I checked the active Azure account using:

az account show --output table

The output displayed account information, including the subscription name, state, and tenant details.

This verification is important because the subscription selected in Azure CLI determines where subsequent Azure commands will operate. Accidentally using the wrong subscription can result in resources being created in the wrong environment.

Security note: Before publishing terminal screenshots, redact device codes, access tokens, credentials, and any identifiers you don't want to expose.

8. Installing Kubernetes command-line tools

Azure CLI manages Azure resources, but we also need kubectl to interact with Kubernetes.

kubectl is the Kubernetes command-line client. It allows us to inspect resources, apply manifests, examine workload status, and communicate with the Kubernetes API when a cluster is available and the appropriate credentials and permissions are configured.

The lab's initial installation command was:

sudo az aks install-cli

Troubleshooting: The default installation path failed

The command encountered an error:

Connection error while attempting to download client
([Errno 2] No such file or directory: '/usr/local/bin/kubectl')

The terminal also showed that /usr/local/bin existed.

This was a useful reminder that an installation command can fail at its target path even when Azure CLI itself is working. Rather than repeatedly retrying the same command, I inspected the directory and switched to a user-local installation location.

First, I checked the directory:

ls -ld /usr/local/bin

Then I created a directory under my home folder:

mkdir -p ~/.local/bin

I reran the installation with explicit paths:

az aks install-cli \
  --install-location ~/.local/bin/kubectl \
  --kubelogin-install-location ~/.local/bin/kubelogin

This instructed the installer to place kubectl and kubelogin in my user-local binary directory.

The installer completed the downloads and advised that ~/.local/bin needed to be included in PATH so the shell could find the commands.

Adding the installation directory to `PATH`

I added the directory to my Bash configuration:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc

Then I reloaded the configuration:

source ~/.bashrc

Finally, I verified the Kubernetes client installation:

kubectl version --client

The terminal returned:

Client Version: v1.37.1
Kustomize Version: v5.8.1

This confirmed that the kubectl client was installed and available from the shell.

What resolved the problem?

The workaround used a directory within my home folder rather than the original default installation location. Updating PATH then allowed the shell to locate the downloaded binaries.

The failure occurred during local tooling setup, before a Kubernetes cluster was involved. Distinguishing local installation problems from cluster or workload problems makes troubleshooting more systematic.

9. Registering Azure resource providers

Before creating Azure resources, the subscription may need the relevant resource providers registered.

A resource provider exposes the management operations for a category of Azure resources. For this lab, I registered four providers:

az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.Compute
az provider register --namespace Microsoft.Network
az provider register --namespace Microsoft.Storage

These providers correspond to the services used throughout the lab:

Resource provider	Purpose
`Microsoft.ContainerService`	AKS and related container-service management
`Microsoft.Compute`	Virtual machines and compute resources
`Microsoft.Network`	Networking resources, including virtual networks and load balancers
`Microsoft.Storage`	Azure storage resources

Registering a provider does not create an AKS cluster or provision the associated resources. It enables the subscription to use the relevant provider's operations.

Verifying registration

I checked the registration state of the AKS provider:

az provider show \
  --namespace Microsoft.ContainerService \
  --query registrationState \
  --output tsv

The result was:

Registered

This confirmed that Microsoft.ContainerService was registered. The screenshot documents the verification of this provider; it does not independently confirm the registration state of all four providers.

10. Preparing the lab environment

The lab used an environment file to keep its configuration values consistent across subsequent commands.

cat << 'EOF' > lab.env
export RESOURCE_GROUP=rg-aks-lab
export LOCATION=westeurope
export CLUSTER_NAME=aks-lab
export NODE_SIZE=Standard_D2s_v3
EOF

I loaded the variables into the current shell:

source lab.env

Then I checked their values:

echo "$RESOURCE_GROUP $LOCATION $CLUSTER_NAME $NODE_SIZE"

The terminal returned:

rg-aks-lab westeurope aks-lab Standard_D2s_v3

Using variables avoids repeatedly typing the same resource names and configuration values throughout a lab. It also reduces the chance of accidentally using inconsistent names in later commands.

The values describe the intended lab configuration; they do not mean the AKS cluster has already been created.

Security reminder: lab.env is included in .gitignore. Keep credentials and secrets out of source control, and check the repository's status before committing changes.

Key takeaways

A dedicated workspace and Git history provide a maintainable starting point for infrastructure projects.
.gitignore helps prevent selected local files from being added to the repository, but it does not replace proper secret management.
Azure CLI and kubectl have different roles: one manages Azure resources, while the other interacts with Kubernetes.
When an installer fails at its default path, inspecting the environment and selecting a suitable installation location can provide a practical workaround.
Always verify the active Azure account and subscription before creating or modifying resources.
Registering a resource provider enables its operations; it does not create resources by itself.

What's next?

In Part 2, I'll continue with the next stage of the lab: provisioning and configuring the Azure Kubernetes Service infrastructure, following the lab guide.

We'll move from preparing the local environment to establishing the Azure infrastructure on which the Kubernetes workloads will run.

This is the foundation for the remaining parts of the series, covering Kubernetes workloads, networking, configuration, storage, automation, scaling, cost optimization, and cleanup.

This article documents the commands and outcomes captured during the lab. Installation results and version numbers reflect the recorded environment and may differ on other systems. Review commands, permissions, and potential Azure charges before running them in your own subscription.

S3 Has No Folders. Here's What You're Really Using.

Rahimah Sulayman — Thu, 22 Oct 2026 10:41:57 +0000

Overview

In my previous article, I created an S3 bucket and uploaded a single object. This time I'm organizing that bucket: creating folders, uploading multiple files at once, moving an object between folders, and finally deleting the bucket.

Along the way I'll explain a fact that trips up many people: Amazon S3 doesn't have folders. What the console shows you is something else, and once you understand it, a lot of S3 behavior starts to make sense.

By the end of this article, you'll be able to:

Create folders (and nested folders) in a bucket
Upload several files in one go
Explain what a prefix is and how it differs from a real folder
Move an object from one folder to another, and understand what S3 does behind the scenes
Empty and delete a bucket correctly

Where we're starting

I'm using the same bucket from the first article, aws-lab1-rahimah, which currently holds one object, datacenter.png.

Step 1: Create a folder

Inside the bucket, I clicked Create folder. The form asks for a folder name, so I entered food. Notice the hint under the field: folder names can't contain a /, because S3 uses that character itself to mark folder boundaries (more on that shortly).

The page also has a Server-side encryption section with two options:

Don't specify an encryption key: the folder uses the bucket's default encryption setting. I kept this one.
Specify an encryption key: choose a key just for this folder object.

Two warnings on that page are worth reading. The encryption settings apply only to the folder object itself, not to the files you later put inside it. And if your bucket policy requires a specific encryption key, you must specify that same key here, or folder creation will fail.

After clicking Create folder, S3 confirmed with a green banner, and the bucket now lists Objects (2): datacenter.png and food/. The Type column shows Folder for food/.

Opening food/ shows an empty folder: Objects (0).

So what is a "folder" in S3?

Here's the important part. S3 uses a flat structure. There is a bucket, and inside it there are objects. That's it. There are no real directories nested inside each other.

What the console calls a folder is built from three ideas:

Key: the full name of an object. A file I upload into food/ has the key food/burger.png, not just burger.png.
Prefix: the beginning part of a key. For food/burger.png, the prefix is food/.
Delimiter: the character that separates the parts of a key. In S3, it's the forward slash /.

When you click Create folder, S3 creates a tiny, empty object whose key ends in / (here, food/). The console then uses the / delimiter to display objects with the same prefix as if they lived in a folder. It's a presentation layer that makes a flat store feel familiar.

Coming from Azure? It works much the same way in Blob Storage. A container holds blobs in a flat namespace, and "folders" are just / characters in blob names (virtual directories). Real directories only appear when you enable the hierarchical namespace on a storage account (Data Lake Storage Gen2).

Step 2: Upload multiple files into the folder

With food/ open, I clicked Upload, then Add files. In the file picker, I selected three images at once: burger.png, cake.png and pizza.png.

The Upload page lists all three (3 total, 8.2 MB), and the Destination is now s3://aws-lab1-rahimah/food/. That path is the S3 URI of the folder, which is exactly the prefix the files will receive.

I clicked Upload, and the status page reported Upload succeeded: 3 files, 8.2 MB, 100%, with 0 failures.

Back at the bucket's root, there are still only two entries, datacenter.png and food/. The three new files live inside the folder.

Opening food/ shows Objects (3): burger.png, cake.png and pizza.png, each with the Standard storage class and its own last-modified time.

Step 3: Open a file and see the prefix in the URL

To prove the prefix idea, I selected burger.png and clicked Open. The image loaded in a new tab, and the address bar shows the key right in the path:

https://aws-lab1-rahimah.s3.eu-north-1.amazonaws.com/food/burger.png?X-Amz-Algorithm=...

See food/burger.png? That's the full key. There's no directory lookup involved. The long X-Amz-... parameters after it are the presigned URL signature I covered in the first article, which is why the file opens even though the bucket is private.

Step 4: Create a nested folder

Next, I wanted a sub-folder. Inside food/, I clicked Create folder again and named it menu. S3 confirmed with Successfully created folder "menu".

Now food/ contains burger.png, cake.png, pizza.png and the new menu/ folder. The key of that new folder is food/menu/, which is just a longer prefix. Nesting folders only means chaining more prefixes with /.

I deleted the datacenter.png and created an empty folder called menu/.

Step 5: Move a file into the new folder

I wanted cake.png to live in menu/. I ticked the checkbox next to cake.png and opened the Actions menu. It offers a lot of options:

Download as and Share with a presigned URL
Calculate total size
Copy and Move
Query with S3 Select
Under Edit actions: Rename object, Edit storage class, Edit server-side encryption, Edit tags, and Make public using ACL

I chose Move. The page opens with an important note at the top: this action creates a copy of the object with a new last-modified date, and depending on the bucket's versioning status, a delete marker will be inserted or the original object will be permanently deleted. Keep that sentence in mind, because it explains what move really does (more below).

The Destination field expects an S3 path in the format s3://bucket-name/optional-prefix/. You can type it, or click Browse S3 and navigate to the folder. I used Browse S3, went into aws-lab1-rahimah > food/, and selected the menu/ folder.

The browser also lets you open menu/ to check what's inside. At this point, it says there are no objects under this path, which is expected for a brand-new folder.

After clicking Choose destination, the form filled itself in:

Destination: s3://aws-lab1-rahimah/food/menu/
Destination bucket name: aws-lab1-rahimah
Destination prefix: food/menu/

Below that, Destination details shows the settings of the target bucket: Bucket Versioning is Disabled, Default encryption is SSE-S3, and Object Lock is Disabled. Because versioning is off, the console shows a recommendation to enable it. The Specified objects table confirms the one file being moved, cake.png (2.7 MB).

Last, Additional copy settings offers three choices. I kept the default, Copy source settings, which carries over the source object's storage class, tags, metadata and encryption. The other two let you skip those settings or specify new ones.

I clicked Move. The status page reported Successfully moved objects: 1 object, 2.7 MB, with 0 failures.

Checking the results:

food/ now shows Objects (3): burger.png, menu/ and pizza.png. cake.png is gone from here.

menu/ shows Objects (1): cake.png.

What "move" really does

Look at the Last modified time on the moved file. It changed to a newer timestamp than when I originally uploaded it. That's the giveaway: S3 has no true "move" or "rename" operation. The console does two steps for you:

Copy the object to the new key (food/cake.png becomes food/menu/cake.png).
Delete the original.

This matters for two reasons. First, because versioning was disabled on this bucket, the original was permanently deleted. If versioning were enabled, S3 would insert a delete marker instead, and the old version could be recovered. Second, the same logic applies to Rename object: it's also a copy followed by a delete, since a rename just means giving the object a new key.

For anything you can't afford to lose, enable versioning before you start moving things around.

Step 6: Delete the bucket

When the lab is finished, clean up so nothing lingers in your account. On the General purpose buckets page, I selected the radio button next to aws-lab1-rahimah. The toolbar offers Copy ARN, Empty, Delete and Create bucket.

I clicked Delete first, and S3 stopped me with this error: "This bucket is not empty. Buckets must be empty before they can be deleted." The page also asks you to type the bucket's name to confirm deletion, but the Delete bucket button stays disabled until the bucket is empty.

The solution is to empty the bucket first:

Click Empty bucket (on that error banner, or Empty on the bucket list). Or go back to the bucket, select the objects and click delete.
Confirm by typing permanently delete when prompted, then click Empty. This can never be retrieved because versioning is not enabled.

Return to Delete bucket, type aws-lab1-rahimah in the confirmation field, and click Delete bucket.

Emptying removes every object, including the folder objects, and it cannot be undone. That's why S3 makes you confirm twice.

Key takeaways

S3 has no real folders. A "folder" is a prefix that the console displays using the / delimiter.
An object's key is its full name, including the prefix. burger.png inside food/ has the key food/burger.png.
Creating a folder just creates a small, empty object whose key ends in /.
You can upload several files at once, and they all receive the destination folder's prefix.
Move and Rename are really copy + delete, which is why the last-modified date changes.
With versioning disabled, a move permanently removes the original. With versioning enabled, a delete marker is added instead.
A bucket must be emptied before it can be deleted.
In Azure terms, this is the same flat-namespace model as Blob Storage virtual directories.

Conclusion

Folders in S3 are a convenience, not a structure. Once you think in terms of keys and prefixes, things like moving files, setting lifecycle rules and writing IAM policies for food/* all make sense, because they all work on prefixes.

Next up in this series, I'll look at S3 permissions and what it takes to make an object public safely.

Which S3 feature should I cover after that: versioning, lifecycle rules or static website hosting? Let me know in the comments.

Getting Started with AWS S3: Creating a Bucket, Uploading an Object, and Understanding Why the Object URL Says Access Denied

Rahimah Sulayman — Tue, 20 Oct 2026 23:11:02 +0000

Introduction

I'm starting a new series on AWS, and this first article covers Amazon S3 (Simple Storage Service), one of the most widely used services on AWS. I did this lab in my own personal AWS free-tier account, and I'll explain every term as it comes up.

By the end, you'll know how to:

Create an S3 bucket with a consistent naming convention
Read the bucket's ARN and properties
Upload a file to the bucket
Find an object's URL, and understand why opening the image works while opening the Object URL doesn't

What is Amazon S3?

Amazon S3 is an object storage service. Instead of organizing data in a traditional disk with a file system, S3 stores data as objects inside buckets. You can store and retrieve any amount of data from anywhere, and you pay only for what you use.

Three terms to know first:

Bucket: a container for your data. Every object in S3 lives inside a bucket.
Object: the thing you store. It's the file itself plus its metadata (size, type, last modified date and so on).
Key: the object's unique name inside the bucket. For my file, the key is datacenter.png.

Step 1: Open the S3 service

In the AWS Management Console, I typed S3 into the search bar and selected S3 (Scalable Storage in the Cloud) from the results.

This opens the Amazon S3 landing page, where Create bucket is the first thing you see.

Step 2: Create a bucket

I clicked Create bucket, and the only thing I had to provide was the bucket name. The region was already set for me (more on that below), and I left everything else at the defaults to keep things simple.

Region

I didn't pick the region on this page. The Create bucket page shows it as plain text, not a dropdown: Europe (Stockholm) eu-north-1. That's because S3 creates a bucket in whichever region your console is currently set to, and mine was already set to Stockholm. I can't change it because this is a free plan. If you want a different region, switch it with the region selector in the top bar of the console before you click Create bucket.

A region is a physical geographic location where AWS runs data centers. Your bucket's data stays in the region it was created in, and the region affects latency (how close the data is to your users) and pricing.

Bucket type

I kept General purpose, which AWS recommends for most use cases. The other option, Directory, is for low-latency workloads that use a single Availability Zone.

Bucket namespace

I kept Global namespace. A namespace is the set of names in which yours has to be unique. In the global namespace, a bucket name must be unique across all AWS accounts, in every region. If someone else already has my-bucket, you can't use it. The console also offers an Account Regional namespace (labelled "recommended"). Buckets created there are unique only within your own account, so no other AWS account can ever claim the same name. I stayed with the global namespace for this lab.

Bucket name and the prefix naming convention

I named my bucket aws-lab1-rahimah.

The aws-lab1- part is a prefix I chose as a naming convention. A consistent prefix gives you three things:

It keeps names unique, since adding your own name or project helps avoid clashes in the global namespace.
It groups related resources together (aws-lab1-..., aws-lab2-...).
It makes resources easy to find and to clean up later.

S3 bucket names must be 3 to 63 characters long, can contain only lowercase letters, numbers, periods and hyphens, and must begin and end with a letter or number.

Default encryption

Further down the page, Default encryption was already set to Server-side encryption with Amazon S3 managed keys (SSE-S3). This means S3 automatically encrypts every new object at rest, using keys that AWS manages for you. You don't have to do anything. I left it as it was and clicked Create bucket.

S3 confirmed with a green banner: Successfully created bucket "aws-lab1-rahimah". The Objects tab shows Objects (0) because the bucket is empty.

Step 3: Look at the bucket's properties and ARN

I opened the Properties tab. The Bucket overview shows three things:

AWS Region: Europe (Stockholm) eu-north-1
Creation date: October 20, 2026
Amazon Resource Name (ARN): arn:aws:s3:::aws-lab1-rahimah

What is an ARN?

An Amazon Resource Name (ARN) is the unique identifier AWS gives every resource. It's how you point at a specific resource in IAM policies, logs and other services. The general shape is:

arn:partition:service:region:account-id:resource

For S3 buckets, the region and account sections are empty, which is why you see three colons in a row (:::). The bucket name is the final part.

The Properties tab also shows two settings that were off by default:

Bucket Versioning: Disabled. Versioning keeps multiple versions of an object, so you can recover from accidental overwrites or deletions. With it disabled, uploading a file with the same key replaces the old one.
Multi-factor authentication (MFA) delete: Disabled. This extra layer of security requires MFA before you can permanently delete object versions.

Step 4: Upload a file to the bucket

Back on the Objects tab, I clicked Upload.

On the Upload page, I clicked Add files and picked an image called datacenter.png from my Downloads folder. You can also drag and drop files, or use Add folder to upload a whole folder.

The page then listed the file as 1 total, 2.4 MB, type image/png, with the destination shown as s3://aws-lab1-rahimah.

Scrolling down, you'll see two collapsed sections:

Permissions: where you can grant public access and access to other AWS accounts.
Properties: where you can set the storage class, encryption settings and tags.

I left both alone and clicked Upload.

The next page showed Upload succeeded, with a summary of 1 file, 2.4 MB, 100% succeeded and 0 failed.

Back in the bucket, Objects (1) now lists datacenter.png, size 2.4 MB, and storage class Standard.

Storage class controls how your data is stored and what it costs. Standard is the default and is meant for frequently accessed data. Other classes, such as Glacier, are cheaper for data you rarely need to retrieve.

Step 5: Explore the object's details

I clicked datacenter.png to open its details. The Object overview shows:

Owner: the canonical ID of the AWS account that owns the object
AWS Region: eu-north-1
Last modified, Size and Type
Key: datacenter.png
S3 URI: s3://aws-lab1-rahimah/datacenter.png
ARN: arn:aws:s3:::aws-lab1-rahimah/datacenter.png
Entity tag (ETag): an identifier for this specific version of the object's content
Object URL: https://aws-lab1-rahimah.s3.eu-north-1.amazonaws.com/datacenter.png

Two of these are easy to confuse:

The S3 URI (s3://bucket/key) is what AWS tools like the AWS CLI and SDKs use to refer to an object.
The Object URL is the HTTPS address of the object on the internet, in the form https://bucket-name.s3.region.amazonaws.com/key.

The ARN here is the bucket's ARN with the object key added on the end.

Step 6: Open the image vs. use the Object URL

This is the part that surprised me, and it taught me the most. There are two ways to view the uploaded image, and they behave differently.

Option 1: Click "Open"

Clicking the Open button displays the image in a new browser tab. The address bar shows a very long URL that carries extra parameters such as X-Amz-Algorithm=AWS4-HMAC-SHA256.

That is a presigned URL. When you click Open, the console uses your own credentials to generate a temporary, signed link that grants access to the object for a short time. S3 sees the signature, confirms that a permitted user created it, and serves the file.

Option 2: Paste the Object URL

When I copied the plain Object URL into the browser, I got this:

<Error>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message>
  ...
</Error>

Why the difference?

The Object URL is an unauthenticated request. When you paste it into a browser, there's no signature and nothing that tells S3 who you are, so S3 treats you as an anonymous visitor. S3 buckets and objects are private by default, and only the owner can access them unless permissions are explicitly granted. So the anonymous request is refused with AccessDenied.

The error is not a bug. It shows S3's security model working as designed: nothing is public until you deliberately make it so.

Making objects public

If you need public access, for example to host images on a website, you'd have to change two things: turn off the relevant Block Public Access settings on the bucket, and then grant read access through a bucket policy or object permissions. Only do this for data that's genuinely meant to be public. For private files, presigned URLs are a safer way to share something temporarily.

Key Takeaways

A bucket is a container and an object is a file inside it, identified by its key.
Bucket names must be globally unique, so a naming prefix like aws-lab1- helps keep names unique and organized.
The region determines where your data physically lives.
An ARN uniquely identifies any AWS resource, and an object's ARN is its bucket's ARN plus the key.
The S3 URI is for AWS tools, while the Object URL is the HTTPS address.
S3 is private by default. Opening an object from the console works because it uses a presigned URL, while the plain Object URL returns AccessDenied.

Clean up

To avoid unexpected charges, delete what you created. You must empty a bucket before you can delete it:

Select datacenter.png and click the box, make sure you tick it, so that the Delete button is not greyed out.

Then confirm deletion.

Go back to the bucket list, select aws-lab1-rahimah and click Delete object.

Conclusion

In this first lab, I created an S3 bucket, uploaded an object, and learned why access to it is blocked by default. Next in the series, I'll build on this with more S3 features and other AWS services.

If you're learning AWS, let me know in the comments which service I should cover next.

Configure a project and repository structure to support secure pipelines.

Rahimah Sulayman — Fri, 16 Oct 2026 08:37:18 +0000

In this lab, you will learn how to configure a project and repository structure in Azure DevOps to support secure pipelines. This lab covers best practices for organizing projects and repositories, assigning permissions, and managing secure files.

Before you start

You’ll need an Azure subscription, Azure DevOps organization, and the eShopOnWeb application to follow the labs.

Follow the steps to validate your lab environment.

Validate your lab environment

In preparation for the labs, it is crucial to have your environment correctly set up. This section will guide you through the setup process, ensuring all prerequisites are met.

The labs require Microsoft Edge or an Azure DevOps-supported browser.
Set up an Azure Subscription: If you don’t already have an Azure subscription, create one by following the instructions on this page or visit https://azure.microsoft.com/free to sign up for a free.
Set up an Azure DevOps organization: If you don’t already have an Azure DevOps organization that you can use for the labs, create one by following the instructions on this page, or at Create an organization or project collection.
Git for Windows download page. This will be installed as part of prerequisites for this lab.
Visual Studio Code. This will be installed as part of prerequisites for this lab.
Azure CLI. Install the Azure CLI on the self-hosted agent machines.
.NET SDK - Latest version. Install the .NET SDK on the self-hosted agent machines.

Instructions to create an Azure DevOps Organization (you only have to do this once)

Note: Start at step 3, if you do already have a personal Microsoft Account setup and an active Azure Subscription linked to that account.

1.Use a private browser session to get a new personal Microsoft Account (MSA) at https://account.microsoft.com.

2.Using the same browser session, sign up for a free Azure subscription at https://azure.microsoft.com/free.

3.Open a browser and navigate to Azure portal at https://portal.azure.com, then search at the top of the Azure portal screen for Azure DevOps. In the resulting page, click Azure DevOps organizations.

4.Next, click on the link labelled My Azure DevOps Organizations or navigate directly to https://aex.dev.azure.com.

5.On the We need a few more details page, select Continue.

6.In the drop-down box on the left, choose Default Directory, instead of Microsoft Account.

7.If prompted (“We need a few more details”), provide your name, e-mail address, and location and click Continue.

8.Back at https://aex.dev.azure.com with Default Directory selected click the blue button Create new organization.

9.Accept the Terms of Service by clicking Continue.

10.If prompted (“Almost done”), leave the name for the Azure DevOps organization at default (it needs to be a globally unique name) and pick a hosting location close to you from the list.

11.Once the newly created organization opens in Azure DevOps, select Organization settings in the bottom left corner.

12.At the Organization settings screen select Billing (opening this screen takes a few seconds).

13.Select Setup billing and on the right-hand side of the screen, select your Azure Subscription and then select Save to link the subscription with the organization.

14.Once the screen shows the linked Azure Subscription ID at the top, change the number of Paid parallel jobs for MS Hosted CI/CD from 0 to 1. Then select SAVE button at the bottom.

Note: You may wait a couple of minutes before using the CI/CD capabilities so that the new settings are reflected in the backend. Otherwise, you will still see the message “No hosted parallelism has been purchased or granted”.

15.In Organization Settings, go to section Pipelines and click Settings.

16.Toggle the switch to Off for Disable creation of classic build pipelines and Disable creation of classic release pipelines

Note: The Disable creation of classic release pipelines switch sets to On hides classic release pipeline creation options such as the Release menu in the Pipeline section of DevOps projects.

17.In Organization Settings, go to section Security and click Policies.

18.Toggle the switch to On for Allow public projects

Note: Extensions used in some labs might require a public project to allow using the free version.
That option no longer exists.
Instructions to create and configure the Azure DevOps project (you only have to do this once)

Note: make sure you completed the steps to create your Azure DevOps Organization before continuing with these steps.

To follow all lab instructions, you’ll need set up a new Azure DevOps project, create a repository that’s based on the eShopOnWeb application, and create a service connection to your Azure subscription.

Create the team project

First, you’ll create an eShopOnWeb Azure DevOps project to be used by several labs.

1.Open your browser and navigate to your Azure DevOps organization.

2.Select the New Project option and use the following settings:

name: eShopOnWeb
visibility: Private
Advanced: Version Control: Git
Advanced: Work Item Process: Scrum

3.Select Create.

Import eShopOnWeb git repository
Now, you’ll import the eShopOnWeb into your git repository.

1.Open your browser and navigate to your Azure DevOps organization.

2.Open the previously created eShopOnWeb project.

3.Select the Repos > Files, Import a Repository and then select Import.

4.On the Import a Git Repository window, paste the following URL https://github.com/MicrosoftLearning/eShopOnWeb and select Import:

5.The repository is organized the following way:

.ado folder contains Azure DevOps YAML pipelines.
.devcontainer folder container setup to develop using containers (either locally in VS Code or GitHub Codespaces).
.azure folder contains Bicep & ARM infrastructure as code templates.
.github folder container YAML GitHub workflow definitions.
src folder contains the .NET 8 website used on the lab scenarios.

6.Leave the web browser window open.

7.Go to Repos > Branches.

8.Hover on the main branch then click the ellipsis on the right of the column.

9.Click on Set as default branch.

Create a service connection to access Azure resources

Next, you will create a service connection in Azure DevOps which will allow you to deploy and access resources in your Azure subscription.

1.Start a web browser, navigate to the Azure DevOps portal with the eShopOnWeb project open and select Project settings in the bottom left corner of the portal.

2.Select the Service connections under Pipelines, and then select Create service connection button.

3.On the New service connection blade, select Azure Resource Manager and Next (may need to scroll down).

4.Select Workload Identity federation (automatic) and Next.

Note: You can also use Workload identity federation (manual) if you prefer to manually configure the service connection. Follow the steps in the Azure DevOps documentation to create the service connection manually.

5.Fill in the empty fields using the information:

Subscription: Select your Azure subscription. Then sign in to your azure account.

Resource group: Select the resource group where you want to deploy resources.
Service connection name: Type azure subs. This name will be referenced in YAML pipelines to access your Azure subscription.

6.Make sure the Grant access permission to all pipelines option is unchecked and select Save.

Note: The Grant access permission to all pipelines option is not recommended for production environments. It is only used in this lab to simplify the configuration of the pipeline.

Note: If you see an error message indicating you don’t have the necessary permissions to create a service connection, try again, or configure the service connection manually.

You have now completed the necessary prerequisite steps to continue with the labs.

Instructions

Exercise 1: Configure a secure project structure

In this exercise, you will configure a secure project structure by creating a new project and assigning it project permissions. Separating responsibilities and resources into different projects or repositories with specific permissions supports security.

Task 1: Create a new team project

1.Navigate to the Azure DevOps portal at https://aex.dev.azure.com and open your organization.

2.Open your organization settings at the bottom left corner of the portal and then Projects under the General section.

3.Select the New Project option and use the following settings:
- name: eShopSecurity
- visibility: Private
- Advanced: Version Control: Git
- Advanced: Work Item Process: Scrum

4.Select Create to create the new project.

5.You can now switch between the different projects by clicking on the Azure DevOps icon in the upper left corner of the Azure DevOps portal.

You can manage permissions and settings for each project separately by going to the Project settings menu and selecting the appropriate team project. If you have multiple users or teams working on different projects, you can also assign permissions to each project separately.

Task 2: Create a new repository and assign project permissions

1.Select the organization name in the upper left corner of the Azure DevOps portal and select the new eShopSecurity project.

2.Select the Repos menu.

3.Select the Initialize button to initialize the new repository by adding the README.md file.

4.Open the Project settings menu in the lower left corner of the portal and select Repositories under the Repos section.

5.Select the new eShopSecurity repository and select the Security tab.

Note: Ensure that you select the Security tab in the specific repository only, and not for all repositories in the project. If you select all repositories, you may lose access to other repositories in the project.

6.Remove the Inherit permissions from parent by unchecking the Inheritance toggle button.

7.Select the Contributors group and select the Deny dropdown for all permissions except Manage permissions and Read. This will prevent all users from the Contributors group from accessing the repository.

Note: In a real world scenario, you will deny the manage permissions to the Contributors group as well. For this lab, we are allowing the Contributors group to manage permissions to allow you to complete the lab.

8.Select your user under Users and select the Allow button to allow all permissions.

Note: If you don’t see your name in the Users section, enter your name in the Search for users or groups text box and select it in the list of results.

9.Your changes will be saved automatically.

Now only the user you assigned permissions and the administrators can access the repository. This is useful when you want to allow specific users to access the repository and run pipelines from the eShopOnWeb project.

Exercise 2: Configure a pipeline and template structure to support secure pipelines

Task 1: Import and run the CI pipeline

Let’s start by importing the CI pipeline named eshoponweb-ci.yml.

1.Navigate to the Azure DevOps portal at https://aex.dev.azure.com and open your organization.

2.Open the eShopOnWeb project in Azure DevOps.

3.Go to Pipelines > Pipelines.

4.Select the Create Pipeline button.

5.Select Azure Repos Git (Yaml).

6.Select the eShopOnWeb repository.

7.Select Existing Azure Pipelines YAML File.

8.Select the /.ado/eshoponweb-ci.yml file then click on Continue.

9.Select the Run button to run the pipeline.

Note: Your pipeline will take a name based on the project name. You will rename it to easier identify the pipeline.

10.Go to Pipelines > Pipelines and select the recently created pipeline. Select the ellipsis and then select Rename/move option.

11.Name it eshoponweb-ci and select Save.

Task 2: Import and run the CD pipeline

Note: In this task, you will import and run the CD pipeline named eshoponweb-cd-webapp-code.yml.

1.Go to Pipelines > Pipelines.

2.Select New pipeline button.

3.Select Azure Repos Git (Yaml).

4.Select the eShopOnWeb repository.

5.Select Existing Azure Pipelines YAML File.

6.Select the /.ado/eshoponweb-cd-webapp-code.yml file then select Continue.

7.In the YAML pipeline definition, set the variables section to:

variables:
 resource-group: 'YOUR-RESOURCE-GROUP-NAME'
 location: 'centralus'
 templateFile: 'infra/webapp.bicep'
 subscriptionid: 'YOUR-SUBSCRIPTION-ID'
 azureserviceconnection: 'YOUR-AZURE-SERVICE-CONNECTION-NAME'
 webappname: 'YOUR-WEB-APP-NAME'

8.Replace the values of the variables with the values of your environment:

Replace YOUR-RESOURCE-GROUP-NAME with the name of the resource group you want to use in this lab, for example, rg-eshoponweb-secure.
Set the value of the location variable to the name of the Azure region you want to deploy your resources, for example, centralus.
Replace YOUR-SUBSCRIPTION-ID with your Azure subscription id.
Replace YOUR-AZURE-SERVICE-CONNECTION-NAME with azure subs
Replace YOUR-WEB-APP-NAME with a globally unique name of the web app to be deployed, for example, the string eshoponweb-lab-multi-123456 followed by a random six-digit number.

9.Select Save and Run and choose to commit directly to the main branch.

10.Select Save and Run again.

11.Open the pipeline run. If you receive the message “This pipeline needs permission to access a resource before this run can continue to Deploy to WebApp”, select View, Permit and Permit again. This is needed to allow the pipeline to create the Azure App Service resource.

12.The deployment may take a few minutes to complete, wait for the pipeline to execute. The pipeline is triggered following the completion of the CI pipeline and it includes the following tasks:

AzureResourceManagerTemplateDeployment: Deploys the Azure App Service web app using bicep template.
AzureRmWebAppDeployment: Publishes the Web site to the Azure App Service web app.

Note: In case the deployment fails, navigate to the pipeline run page and select Rerun failed jobs to invoke another pipeline run.

Note: Your pipeline will take a name based on the project name. Let’s rename it for identifying the pipeline better.

13.Go to Pipelines > Pipelines and select the recently created pipeline. Select the ellipsis and then select Rename/move option.

14.Name it eshoponweb-cd-webapp-code and click on Save.

Now you should have two pipelines running in your eShopOnWeb project.

After troubleshooting, the pipeline finally succeeded.
NOTE: Check the Troubleshooting section below to see how I resolved the errors.

Task 3: Move the CD pipeline variables to a YAML template

In this task, you will create a YAML template to store the variables used in the CD pipeline. This will allow you to reuse the template in other pipelines.

1.Go to Repos and then Files.

2.Expand the .ado folder and select New file.

3.Name the file eshoponweb-secure-variables.yml and select Create.

4.Add the variables section used in the CD pipeline to the new file. The file should look like the following:

variables:
  resource-group: 'rg-eshoponweb-secure'
  location: 'southcentralus' #the name of the Azure region you want to deploy your resources
  templateFile: 'infra/webapp.bicep'
  subscriptionid: 'YOUR-SUBSCRIPTION-ID'
  azureserviceconnection: 'azure subs' #the name of the service connection to your Azure subscription
  webappname: 'eshoponweb-lab-secure-XXXXXX' #the globally unique name of the web app

Important: Replace the values of the variables with the values of your environment (resource group, location, subscription ID, Azure service connection, and web app name).

5.Select Commit, in the commit comment text box, enter [skip ci], and then select Commit.

Note: By adding the [skip ci] comment to the commit, you will prevent automatic pipeline execution, which, at this point, runs by default following every change to the repo.

6.From the list of files in the repo, open the eshoponweb-cd-webapp-code.yml pipeline definition, and replace the variables section with the following:

variables:
     - template: eshoponweb-secure-variables.yml

7.Select Commit, accept the default comment, and then select Commit to run the pipeline again.

8.Verify that the pipeline run completed successfully.

Now you have a YAML template with the variables used in the CD pipeline. You can reuse this template in other pipelines in scenarios where you need to deploy the same resources. Also, your operations team can control the resource group and location where the resources are deployed and other information in your template values and you don’t need to make any changes to your pipeline definition.

Task 4: Move the YAML templates to a separate repository and project

In this task, you will move the YAML templates to a separate repository and project.

1.In your eShopSecurity project, go to Repos > Files.

2.Create a new file named eshoponweb-secure-variables.yml.

3.Copy the content of the file .ado/eshoponweb-secure-variables.yml from the eShopOnWeb repository to the new file.

4.Commit the changes.

5.Open the eshoponweb-cd-webapp-code.yml pipeline definition in the eShopOnWeb repo.

6.Add the following to the resources section before the variables section in the pipeline definition:

   repositories:
     - repository: eShopSecurity
     type: git
     name: eShopSecurity/eShopSecurity #name of the project and repository

7.Replace the variables section with the following:

variables:
     - template: eshoponweb-secure-variables.yml@eShopSecurity #name of the template and repository

8.Select Commit, accept the default comment, and then select Commit to run the pipeline again.

9.Navigate to the pipeline run and verify that the pipeline is using the YAML file from the eShopSecurity repository.

Now you have the YAML file in a separate repository and project. You can reuse this file in other pipelines in scenarios where you need to deploy the same resources. Also, your operations team can control the resource group, location, security and where the resources are deployed and other information by modifying values in the YAML file and you don’t need to make any changes to your pipeline definition.

[!IMPORTANT] Remember to delete the resources created in the Azure portal to avoid unnecessary charges.

Review

In this lab, you learned how to configure and organize a secure project and repository structure in Azure DevOps. By managing permissions effectively, you can ensure that the right users have access to the resources they need while maintaining the security and integrity of your DevOps pipelines and processes.

Troubleshooting: Problems I Hit and How I Fixed Them

1. “Could not find any file matching the template file pattern”

Symptom: The “Deploy App Service Plan Bicep” step failed before reaching Azure.

Cause: I set templateFile: 'infra/webapp.bicep'. The task already builds the path as $(Pipeline.Workspace)/eshoponweb-ci/Bicep/$(templateFile), so it looked for .../Bicep/infra/webapp.bicep, which doesn’t exist. The CI pipeline publishes the file in a Bicep folder in the artifact, not in infra.

Fix: Set templateFile: 'webapp.bicep', with no folder prefix.

2. Quota error: `SubscriptionIsOverQuotaForSku`

Symptom:

Current Limit (S1 VMs): 0 Amount required for this deployment (S1 VMs): 1

Cause: The Bicep template defaults to the S1 SKU, and my subscription had a quota of 0 for it. This can happen even on pay-as-you-go subscriptions, because new subscriptions often start with no App Service quota for some tiers.

Fix attempt: I changed the SKU in infra/webapp.bicep to the Free tier, F1.

3. The change didn’t take effect, and F1 failed too

Symptom: The next run still reported S1 VMs.

Cause: The CD pipeline deploys the copy of the Bicep file inside the eshoponweb-ci build artifact, not the file in the repo. My edit only took effect after I re-ran eshoponweb-ci. After that, the error changed to Current Limit (F1 VMs): 0, so the subscription had no Free-tier quota in that region either.

Fix: Quota is per region, so I created a new resource group in West Europe and updated the resource-group and location variables.

4. 403 when checking the new resource group

Symptom:

Checking if the following resource group exists: RG. Failed to check the resource group status. Error: {"statusCode":403}

Cause: The Azure DevOps service connection signs in as a service principal whose role assignment only covered my original resource group.

Fix:

In Azure DevOps, Project settings → Service connections → Manage App registration gave me the service principal’s display name.
In the Azure portal, on the new resource group: Access control (IAM) → Add → Add role assignment.
I picked the plain Contributor role (not Web Plan Contributor or Website Contributor) and selected the service principal as the member. Check that the Role tab doesn’t say “None” before clicking Review + assign.
After waiting a few minutes, the deployment succeeded.

5. TF401027: You need the Git 'CreateBranch' permission

Symptom: Clicking Initialize on the empty eShopSecurity repo failed.

Cause: My account lacked the Create branch permission on that repository, even though I was the organization owner and in the Project Collection Administrators group. An explicit Deny or a repo-level permission issue overrides group membership.

Fix: I checked the repo’s Security tab for a Deny, then created a fresh repo instead, because the creator of a repo gets full rights on it. I reused the name eShopSecurity.

6. Pipeline run “Informational run” with a YAML error

Symptom: A run appeared with an error about YAML preventing Azure Pipelines from deciding whether to run, and clicking into it showed No plan found for identifier.

Cause: The YAML couldn’t be parsed, so the pipeline never started. There’s no execution plan to open or rerun.

Fix: Don’t rerun that entry. Use Edit → ⋮ → Validate on the pipeline to find the broken line, fix it, and start a new run.

7. The shared variables template

The lab moves variables into a template in a separate repo and references it from the pipeline:

resources:
  repositories:
    - repository: eShopSecurity
      type: git
      name: eShopSecurity/eShopSecurity

variables:
  - template: eshoponweb-secure-variables.yml@eShopSecurity

Things to check:

The alias after @ must match the repository: name in resources.
Put a space before any # comment, or YAML treats it as part of the value.
The template file holds the variable definitions (- name: / value:). It must not contain a template: line pointing to itself.
Because the pipeline no longer defines variables, the template must include every one it uses (resource-group, location, templateFile, subscriptionid, azureserviceconnection, webappname).
On the first run, Azure DevOps may ask you to permit access to the other repo.

Errors resolved while trying to view the App in a browser

8. Git: “Authentication failed” and PATs

Symptom: git pull and git push over HTTPS asked for a password and rejected it.

Cause: Azure Repos doesn’t accept your Microsoft account password. It needs a Personal Access Token with Code → Read & write, created for the right organization.

Fix: I created a new PAT (User settings → Personal access tokens), pasted it at the prompt (nothing shows as you type, and Shift+Insert or right-click pastes in WSL), and the push worked. I revoked the unused token afterwards.

After the push, I waited for the pipeline to run successfully.

9. Build warning NU1903 (known high severity vulnerability)

Symptom: The CI run and dotnet build both warned that Microsoft.AspNetCore.Components.WebAssembly.Server 8.0.15 had a known high severity vulnerability.

Fix:

dotnet list src/Web/Web.csproj package --outdated --highest-patch showed the newest 8.0.x patch (8.0.32).
Because the repo uses central package management, I changed the version in Directory.Packages.props.
Rebuilding with dotnet build eShopOnWeb.sln reported 0 warnings.

10. Running the app locally in WSL

MSB1011: more than one project or solution file: the repo root has both a .sln and a .dcproj. Name the target: dotnet build eShopOnWeb.sln and dotnet run --project src/Web.
LocalDB is not supported on this platform: LocalDB is Windows-only. Run with the in-memory database: UseOnlyInMemoryDatabase=true dotnet run --project src/Web, then open http://localhost:5001.

Key takeaways

A quota error is per SKU and per region, so changing either can clear it.
The CD pipeline deploys the CI artifact, so Bicep changes only apply after CI runs again.
A new resource group needs its own role assignment for the service connection’s service principal.
A failed YAML parse produces an informational run, so use Validate rather than rerunning it.
Azure Repos over HTTPS needs a PAT, not your account password.