How Do I Decide If Network or IO Will Bottleneck After Moving to Shared CPU?

When transitioning workloads to shared CPU instances in cloud environments, it’s tempting to focus primarily on CPU allocation and vCPU counts — after all, more cores seem like more power. But experienced cloud engineers know this is only part of the story. Network limits and IO demand often become the *actual* bottlenecks, and these constraints vary in subtle but important ways depending on the cloud provider and instance family.

In this post, I’ll share engineering-proven guidance to help you determine reliably whether network or IO will throttle your performance after moving to shared CPU instances. Along the way, I’ll reference tools like AWS Compute Optimizer and Azure Advisor, and emphasize why measuring peaks with the right observation window and percentiles—rather than averages—is critical.

Understanding the Context: What Does "Shared CPU" Even Mean?

Before diving into thresholds and performance metrics, let’s clarify a common https://computingforgeeks.com/shared-cpu-cloud-waste-migration-guide/ source of confusion: the term “shared CPU” varies by cloud provider and instance type.

• AWS shared CPU instances like T3, T4g provide “burstable” CPU credits allowing temporary surges beyond baseline CPU allocation. However, the underlying hardware might be shared with noisy neighbors.
• Azure B-series VMs similarly offer credits, but the implementation details (e.g., hypervisor scheduling, CPU steal time characteristics) differ from AWS.
• Google Cloud offers "shared-core" machines, where fractions of a physical core are allocated to your VM.

The takeaway? Never assume that a “shared CPU” instance from one provider behaves identically to another. This matters because the performance impact extends beyond CPU — network and storage IO limits may be more constrained or bursty depending on the specific shared environment.

Why Your Average CPU and Network Metrics Lie

One of my biggest frustrations: teams make decisions based solely on average CPU utilization or average network throughput. This approach invariably misses the real problem — bottlenecks appear during peaks, not averages.

Imagine your service has mostly low CPU usage but experiences 100ms spikes every minute that cause delays. The average CPU might be 20%, leading you to believe an instance downgrade would be safe. In reality, those spikes could suffer from CPU throttling or network saturation in a shared CPU environment.

This is why I always ask: what do the P95 and P99 percentiles look like? What’s the spike duration? Those metrics reveal hidden problems masked by averages.

Step 1: Define Your Observation Window Appropriately

Choosing the right timeframe to capture peaks is crucial. Too short, and you get noisy data that’s hard to interpret. Too long, and you smooth out spikes into harmless-looking averages.

For most “always-on small services,” I recommend gathering sampling data over at least a week and slicing it into:

• 5-minute intervals: To preserve spikes without overwhelming noise
• Hourly windows: To observe diurnal and day-of-week patterns

This contrasts with one-minute data, which might be too noisy, or daily aggregates, which hide spikes entirely.

Step 2: Capture and Interpret Peak Metrics – Use P95 and P99 Percentiles, Not Averages

For both network and IO performance, look at the following metrics at P95 and P99:

• Network throughput (Mbps or Gbps): Understand outbound and inbound transfer rates during worst-case intervals.
• IOPS and bytes per second to storage: Distinguish between read/write patterns and the distribution of IO sizes.
• Latency percentiles for IO calls: 95th or 99th percentile latency spikes reveal real performance degradation often masked by average latency.

The goal: identify the highest sustained demand levels and the duration of those spikes. Short-lived spikes might be acceptable depending on tolerance; sustained bottlenecks are not.

Step 3: Correlate CPU Sharing Behavior with Network Limits and IO Demand

When moving to shared CPU instances, CPU steal time (time your process is ready but waiting for a physical core) can cause cascading effects:

1. CPU throttling can delay network packet processing, making network limits feel like a bottleneck.
2. Disk/FS IO commands can queue longer, increasing latency and reducing throughput.

Use tools like AWS Compute Optimizer and Azure Advisor to get tailored recommendations based on your actual usage patterns. Compute Optimizer, for example, analyzes CPU, memory, and network usage patterns and will flag when your instance type is under-provisioned for burst demand or sustained peaks.

Quick Tips on Using AWS Compute Optimizer and Azure Advisor

• AWS Compute Optimizer: Use the “Utilization Metrics” report to view CPU, network, and disk IO at high percentiles. Look for EC2 instance recommendations that consider burst credit exhaustion and network IO limits.
• Azure Advisor: Pay attention to recommendations on VM resizing related to “performance” and “resource utilization.” The Advisor also gives IO performance guidance; check the storage account metrics recommended for your workload.

Step 4: Validate Storage Performance Constraints Separately

Storage performance is often overlooked but can be a critical bottleneck when CPU is shared. Two common issues:

• Throughput limits: e.g., EBS volume throughput caps or Azure managed disk limits
• IOPS limits: high IO operations per second demand can saturate provisioned IOPS volumes

Examples: An application with low average IO but short high-volume spike periods can get throttled if volume limits are reached.

Validate your storage path by looking at:

• Disk queue length and latency at 95th/99th percentile during peak windows
• Volume bandwidth vs provisioned maximums
• Read/write cache hit ratios

If you detect regular saturation, consider either:

• Provisioning higher tier volumes (more IOPS or bandwidth)
• Restructuring workload to smooth out spikes
• Moving to local NVMe/NVMe SSD where supported

Step 5: Putting It All Together—When Is Network or IO Actually the Bottleneck?

Symptom Likely Bottleneck Measurement Mitigation High latency spikes during bursts despite low average CPU CPU steal / shared CPU contention leading to network packet delays CPU steal time, network packet delay at P99 Move to dedicated CPU instances, redesign service for less bursty usage Network throughput saturates at reported instance limits during peak windows Network limits Network bytes per second at P95/P99, monitor bursting credits if available Upgrade instance family or use enhanced networking features High IO latency or disk queue length during burst writes/reads Storage IO limits (IOPS or throughput) Disk IOPS and latency percentiles, queue length during bursts Upgrade storage volume type; redesign workload for smoother IO patterns

Common Pitfalls to Avoid

• Don’t treat vCPU count as a performance guarantee. More virtual cores don’t equal linear performance in bursty or shared CPU setups.
• Always measure and validate with proper percentiles and observation windows. Averages alone hide headache-inducing bursts.
• Don’t overlook storage and egress costs in your performance matching. Optimizing CPU but ignoring storage demand leads to unpredictable slowdowns and cost surprises.
• Avoid assumptions that shared CPU equals bad uptime. Many modern shared CPU offerings manage contention well, but understanding your workload’s burstiness is key.

Final Words: Metrics, Margins, and Monitoring Matter

Deciding if network or IO will bottleneck after moving to shared CPU requires a granular and nuanced approach:

1. Know your cloud provider’s shared CPU model and how network/IO limits tie into it.
2. Use AWS Compute Optimizer and Azure Advisor insights but don’t treat them as gospel—validate frequently.
3. Measure P95 and P99 metrics over meaningful intervals to capture spikes and their duration.
4. Analyze CPU steal, network throughput, and storage IOPS together to correlate bottlenecks.
5. Prepare rollback criteria and pilot changes carefully—never guess based on averages.

By focusing on these engineering best practices and treating network limits and storage performance as first-class citizens in your capacity planning, you’ll avoid hidden bottlenecks that plague many shared CPU migrations.

Happy optimizing!