DevOps & Cloud Interview Prep: Real Scenarios & Answers

Your Cassandra node is getting evicted at 2 a.m. or your PostgreSQL replica is sitting on 4× the memory it needs — this episode breaks down exactly which autoscaler to reach for and why VPA vs HPA is a staple senior SRE interview question.You'll learn:Why HPA's scale-out model breaks for StatefulSets (token rings, replication slots, unacknowledged messages) and when vertical scaling is the only safe leverVPA's three components — recommender, admission controller, updater — and why update mode auto is the one that pages you at 2 a.m. for stateful workloadsThe safe progression: start in Off mode for two weeks, apply recommendations manually, then consider Initial mode — and why Auto is rarely worth it for anything with persistent stateThe HPA + VPA feedback loop failure: both watching CPU on the same workload, pod count oscillating, resource allocation in chaosThe clean split that actually works: HPA on queue depth (custom metric), VPA managing memory requests — distinct signals, no interferenceKeywords: VPA vs HPA Kubernetes, vertical pod autoscaler stateful workloads, autoscaler SRE interview, HPA VPA conflict, Kubernetes StatefulSet autoscaling🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud▶ Daily 30-second interview drills: DevOps Interview Cloud on YouTubeTranscriptYour stateful app is getting evicted every few hours, or it's sitting on four times the memory it actually needs, and you're not sure which autoscaler to reach for. That's the exact scenario interviewers love, because most candidates only know one half of the answer.Let's set the stage. You have a Cassandra node, a RabbitMQ broker, or a PostgreSQL read replica running in Kubernetes. It's a StatefulSet. Traffic is not uniform. Some days it's busy, some days it's quiet. Someone on your team says "just add HPA" and someone else says "we need VPA." Who's right? That question shows up in senior SRE and platform engineer interviews constantly, because the answer is not obvious and the wrong choice causes real incidents.Interviewers ask this because autoscaling is one of those topics where surface-level knowledge falls apart fast. Saying "HPA scales pods out, VPA scales pods up" is correct but incomplete. The follow-up is always: okay, so which one do you use for a stateful workload, and why? And then: what happens if you use both at the same time? If you can't answer those, you signal that you've only worked with stateless services.Here's the mental model you need. The HPA, the Horizontal Pod Autoscaler, works by changing the number of pod replicas. It watches a metric, CPU or memory or a custom one, and when that metric crosses a threshold it adds or removes pods. That works beautifully for stateless services. Each replica is identical, sessions don't matter, and spinning up a new pod behind a load balancer is invisible to users.Stateful workloads break that assumption. A Cassandra node owns a subset of the token ring. A RabbitMQ broker may hold unacknowledged messages. A PostgreSQL replica has an open replication slot. Adding a new pod doesn't instantly help because the new pod has to join the cluster, sync data, or acquire state before it can carry load. Scaling out fast is often dangerous. Scaling in is even worse because you might be removing a pod that holds data not yet replicated elsewhere.So for stateful workloads, the more useful lever is usually vertical. Give the existing pod more CPU or more memory so it can handle the load without needing a new replica. That's where the VPA, the Vertical Pod Autoscaler, comes in.VPA has three components. The recommender watches historical resource usage and calculates what your requests and limits should be. The admission controller patches those values into new pods at scheduling time. And the updater, this is the dangerous one, can evict running pods so they restart with the new resource values. The key knob is the update mode, and this is a common interview question on its own.Update mode off means the VPA only calculates recommendations. It never touches your pods. You query the VPA object and apply changes yourself. This is the right starting point for any stateful workload. You get the data without the risk.Update mode initial means the admission controller sets resource values when a pod is first created, but the VPA never evicts a running pod. The pod keeps whatever resources it started with until it's naturally replaced, for example during a rollout. This is a reasonable middle ground for apps that tolerate restarts as part of deployments but not random evictions.Update mode auto, and this is the one that bites teams, allows the VPA updater to evict pods at any time to apply new recommendations. For a stateless deployment this is often fine. For a stateful pod it can be catastrophic. Imagine your primary Redis instance getting evicted at two in the morning because the VPA decided it needed fifty percent more memory. The pod comes back, but during those thirty seconds of restart your application is throwing errors.The fourth mode, recreate, behaves like auto but only triggers on pod creation events. In practice most teams treat auto and recreate as the same risk category for stateful workloads.So the concrete recommendation for stateful apps is: start with update mode off, run it for at least two weeks to cover your traffic patterns, look at the recommended requests in the VPA status, and then apply those values manually to your StatefulSet spec. After you've done that once and confirmed stability, you can consider bumping to initial mode. Staying in auto is rarely worth it for anything with persistent state.Now the tricky part that interviewers really test: can you run HPA and VPA at the same time?The short answer is yes, but only if they're watching different metrics. The classic failure pattern is enabling both on the same workload with both watching CPU. Here's what happens. Load increases. CPU goes up. HPA adds a replica. The CPU per pod drops because the load is now spread. VPA sees lower per-pod CPU and recommends lower requests. It either tells you to reduce requests or, in auto mode, evicts pods to resize them down. Meanwhile HPA is still watching the same CPU signal. You get a feedback loop where both controllers are fighting each other, your pod count oscillates, and your resource allocation is unstable.The safe combination is to give each controller a distinct domain. A common pattern for a message broker is to let HPA scale on queue depth, a custom metric from your metrics pipeline, while VPA manages the memory requests because memory usage on a broker tends to grow with configuration and message size rather than with load spikes. They're watching completely different signals so they don't interfere.Another safe pattern is to disable HPA entirely and rely only on VPA in initial mode, pairing that with a PodDisruptionBudget, the PDB, to prevent the cluster autoscaler from removing nodes that would orphan your pods. This is common for single-instance stateful workloads where horizontal scaling genuinely isn't possible.Let's talk real numbers briefly so you sound credible in the interview. A typical starting point for a moderately loaded RabbitMQ broker might be two hundred and fifty millicores CPU request and five hundred millicores limit, with one gigabyte memory request and two gigabytes limit. After running VPA in off mode for two weeks under normal traffic, you might see the recommender suggesting six hundred millicores CPU and one and a half gigabytes memory. That gap between what you provisioned and what you actually need is exactly the over-provisioning problem VPA is designed to solve. Without it, you're just guessing.On the eviction risk side, the VPA updater respects PodDisruptionBudgets. If your PDB says zero pods unavailable, the updater will not evict your pod even in auto mode. So if you are running VPA auto on a StatefulSet, always pair it with a PDB that reflects your actual availability requirements. This is a detail that separates strong answers from weak ones in interviews.There's also the resource policy inside the VPA spec itself. You can set min allowed and max allowed values per container so the recommender can't suggest something absurd. For example, you can tell it to never recommend less than one hundred millicores or more than four cores, keeping the recommendations inside bounds your infrastructure can actually handle. Always set these bounds for production workloads.One more wrinkle worth knowing. VPA and HPA cannot both target CPU or memory requests at the same time because HPA uses requests as the denominator for its utilization calculation. If VPA changes the request value, the HPA's target threshold shifts underneath it. The percentage-based math breaks. Kubernetes has a known limitation here, and the official guidance is to not use HPA on CPU or memory when VPA is also active on the same workload, unless you're using HPA with custom or external metrics only.Common wrong answers you want to avoid. The first is saying "just use HPA for everything, stateful or not." That ignores the data ownership and sync cost of scaling stateful replicas. The second is saying "VPA is dangerous so never use it." That's overcorrecting. VPA in off or initial mode is genuinely useful and low risk. The third is conflating pod autoscaling with cluster autoscaling. The cluster autoscaler adds nodes. VPA and HPA manage pods within existing nodes. They work at different layers and you need to be clear about which layer you're talking about. The fourth is not mentioning PodDisruptionBudgets when discussing VPA auto mode. Experienced interviewers will notice that gap immediately.Quick recap. For stateful workloads, your default should be VPA in off mode to gather recommendations, and then apply them manually. Use initial mode once you trust the recommendations. Avoid auto mode unless you've paired it with a tight PodDisruptionBudget and you understand the eviction risk. If you combine HPA and VPA, give them different metrics so they don't fight each other. Never point both at CPU at the same time. Set min and max resource policies in your VPA spec to bound the recommendations. And always think about what a pod eviction means for your specific stateful workload before enabling anything that can trigger one.If you want to drill questions exactly like this one, the DevOps Interview Cloud channel on YouTube posts a thirty-second interview drill every single day, short sharp questions you can work through on your commute. And if you want the deeper reference material, practice scenarios, and answer frameworks, head over to devopsinterview dot cloud.

Learn how to write a Kubernetes scheduler extender webhook that restricts GPU pods to nodes with NVLink interconnects. This episode covers the extender contract, KubeSchedulerConfiguration registration, filter and prioritize endpoints, and the latency tradeoffs interviewers probe in senior SRE and platform engineering interviews.Full interview prep guides and scenario walkthroughs: DevOpsInterview.Cloud

What really happens when you run kubectl apply?
In Part 1 of this Kubernetes masterclass, we go far beyond basic definitions and trace how Kubernetes works as a distributed, API-driven control system.
You will learn how a YAML manifest moves through kubectl, the API server, authentication, authorization, admission, etcd, controllers, the scheduler, kubelet and the container runtime before finally becoming a running Pod.
This episode also explains the deeper ideas that make Kubernetes work:
Desired state versus observed state
Reconciliation loops
spec versus status
Watches and events
Labels and selectors
ReplicaSets and Deployments
Scheduling decisions
Pod lifecycle
Owner references, finalizers and garbage collection
Server-side apply and field ownership
By the end of this episode, you will be able to mentally replay the complete journey from user intent to a healthy running workload—and understand which component is responsible at every step.
Mental model:
Intent → Store → Observe → Reconcile
Full interview prep guides and scenario walkthroughs: DevOpsInterview.Cloud

Most engineers assume Karpenter is always the right answer for Kubernetes node autoscaling, but at 500 nodes the tradeoffs around ASG lock-in, provisioner complexity, and migration risk get serious. This episode breaks down when to keep Cluster Autoscaler, when Karpenter wins, and how to articulate both sides clearly in a senior DevOps or SRE interview. Covers real configuration details, scaling latency numbers, and common wrong answers interviewers flag.Full interview prep guides and scenario walkthroughs: DevOpsInterview.Cloud

A Java service keeps getting OOMKilled in Kubernetes even though memory requests look fine on paper. This episode explains why JVM heap defaults ignore container limits, how to set maximum heap size correctly, and what interviewers expect when they probe your understanding of Java memory in containerized environments. Covers Xmx flags, UseContainerSupport, native memory overhead, and the tradeoffs between requests and limits.Full interview prep guides and scenario walkthroughs: DevOpsInterview.Cloud

When AWS fires the 2-minute Spot reclaim notice, Karpenter's interruption queue is the difference between a blip and a batch job disaster — here's exactly how to configure it.You'll learn:How to set karpenter.sh/capacity-type in a NodePool to prefer Spot with automatic On-Demand fallbackThe full interruption flow: SQS queue → cordon → graceful drain → pod rescheduling, all within the 2-minute windowWhy the order of values in the capacity-type array doesn't control selection — Karpenter uses price-capacity optimizationWhen to use strict values: ['spot'] and what happens when capacity dries upWhy Pod Disruption Budgets and gracefulTerminationPeriod are non-negotiable for fault-tolerant batch workloadsKeywords: Karpenter Spot interruption handling, Spot instance fallback on-demand, NodePool capacity type configuration, Kubernetes batch workload cost optimization, Spot 2-minute warning drain🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud

Automated canary analysis for a Flink-based streaming app is a common senior SRE interview scenario — here's how to wire Prometheus, Loki, and Pyroscope into a production-grade rollout strategy.You'll learn:How to define canary success criteria using Prometheus metrics like consumer lag, throughput, and error rate on Flink jobsUsing Loki log queries to surface structured errors in canary vs. baseline deployments side-by-sideContinuous profiling with Pyroscope to catch CPU or memory regressions in the new Flink version before full rolloutHow automated analysis gates work — failing fast vs. baking time — and how to articulate the tradeoff in an interviewStitching observability signals into a single canary decision: pass, fail, or inconclusiveKeywords: canary deployment Flink, automated canary analysis SRE, Prometheus Loki Pyroscope, streaming app observability, DevOps interview questions🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud

Grafana Mimir storage at 10TB/day scale forces real trade-offs — here's how to configure tiered storage to S3 without bleeding cost or tanking query performance.You'll learn:How Mimir's store-gateway and compactor interact with S3-backed object storage at high ingest volumeConfiguring blocks_storage with tiered retention — keeping hot blocks in fast storage while offloading cold blocks to S3 Glacier-compatible tiersTuning compaction schedules and chunk caching (memcached) to reduce S3 GET costs under sustained 10TB/day ingestCommon pitfalls: misconfigured bucket lifecycle policies, compactor overlap errors, and index cache misses killing query latencySizing ruler and alertmanager storage separately so they don't contend with block storage I/OKeywords: Grafana Mimir S3 storage, Mimir tiered storage config, Mimir compactor tuning, metrics storage at scale, Mimir blocks_storage🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud

If your service has a 99.99% SLO and Azure drops a zone for 15 minutes, here's exactly how to calculate the error budget burn rate before your next SRE interview.You'll learn:How to derive total monthly error budget from a 99.99% SLO (~4.38 minutes/month)Why a 15-minute outage consumes roughly 3.4x your entire monthly budget — and how to show that mathThe burn rate formula interviewers expect: burn rate = error rate / (1 − SLO target)How fast vs. slow burn rates map to alerting windows in Google's SRE workbook approachCommon gotchas: partial zone failures, dependency blame, and how to frame mitigation in your answerKeywords: SLO error budget burn rate, Azure availability zone outage, SRE interview questions, error budget calculation, 99.99 SLO math🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud

Designing a PCI-DSS compliant serverless payments architecture on GCP means getting Confidential VMs, Cloud External Key Manager, and Binary Authorization working together — here's how to answer that in a senior interview.You'll learn:How Confidential VMs provide hardware-level memory encryption to satisfy PCI-DSS data-in-use requirementsWhy Cloud External Key Manager (CEKM) lets you hold encryption keys outside GCP's control — and what that means for scope reductionHow Binary Authorization enforces cryptographic attestation so only verified container images reach your payment workloadsThe serverless boundary decisions (Cloud Run vs bare GKE) that affect your Cardholder Data Environment scopeCommon interview gotchas around shared responsibility, audit logging with Cloud Audit Logs, and VPC Service Controls for perimeter defenceKeywords: PCI-DSS GCP architecture, Confidential VMs interview, Cloud External Key Manager, Binary Authorization Cloud Run, serverless payments compliance🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud