CPU Scheduling Algorithms Overview: Operating System Guide

1. Introduction and Core Concepts

CPU scheduling is the fundamental operating system mechanism that determines which process among many ready processes gets access to the central processing unit at any given moment. In a multiprogramming environment where dozens or hundreds of processes compete for limited CPU time, the scheduler acts as the arbiter of fairness, efficiency, and responsiveness. This cpu scheduling algorithms overview provides a systematic examination of the strategies operating systems use to maximize throughput while minimizing waiting time and ensuring no process is indefinitely starved.

At its core, CPU scheduling addresses the problem of multiplexing a single physical resource — the CPU — across multiple logical tasks. When a process enters the ready state, it joins the ready queue, a data structure managed by the short-term scheduler (also called the CPU scheduler). The scheduler selects one process from this queue according to a defined policy, dispatches it to the CPU, and later preempts or deallocates it when its time slice expires, it blocks for I/O, or it terminates. The quality of this decision directly impacts key performance metrics: CPU utilization, throughput (processes completed per unit time), turnaround time (total time from submission to completion), waiting time (time spent in the ready queue), response time (time from submission until first response), and fairness.

Understanding CPU scheduling is essential not only for operating system designers but also for application developers, system administrators, and performance engineers. Poor scheduling choices can lead to convoy effects, starvation, excessive context switching overhead, or unacceptable latency for interactive users. Modern systems ranging from embedded real-time controllers to cloud-scale data centers all rely on variations of these classic algorithms, often combining them into hierarchical or adaptive frameworks. This overview establishes the vocabulary and intuition needed to evaluate, implement, and tune scheduling policies for diverse workloads.

Key Terminology and Scheduling Criteria

Before diving into specific algorithms, it is important to define the criteria used to compare them. CPU burst refers to the period a process requires continuous CPU execution before it blocks for I/O, while I/O burst is the time spent waiting for input/output. Processes are generally classified as CPU-bound (long CPU bursts, infrequent I/O) or I/O-bound (short CPU bursts, frequent I/O). An optimal scheduler favors I/O-bound processes briefly to keep devices busy while still making progress on CPU-bound work.

Schedulers are also characterized as preemptive or non-preemptive. A non-preemptive scheduler allows a running process to hold the CPU until it voluntarily releases it through termination or blocking. A preemptive scheduler can forcibly interrupt a running process to allocate the CPU elsewhere, enabling better responsiveness but incurring context-switch overhead. The choice between these models influences every algorithm's behavior, complexity, and suitability for interactive versus batch environments.

2. How It Works

CPU scheduling operates within the broader process lifecycle managed by the operating system kernel. When the system boots, the long-term scheduler admits processes into memory and the medium-term scheduler handles swapping. The short-term scheduler, which implements the CPU scheduling algorithm, is invoked at critical dispatch points: when a process switches from running to waiting (e.g., I/O request), from running to ready (e.g., interrupt or time quantum expiration), from waiting to ready (e.g., I/O completion), and upon termination. The first and fourth events allow only non-preemptive decisions, while all four events permit preemptive scheduling.

The dispatch mechanism itself involves three components: the ready queue, the scheduling policy, and the dispatcher. The ready queue is typically implemented as a FIFO queue, priority queue, or multilevel feedback structure depending on the algorithm. The policy defines the selection logic — for example, shortest job first selects the process with the smallest next CPU burst, while round robin cycles through processes in arrival order. The dispatcher is the low-level module that performs the context switch: saving the current process's registers and program counter, loading the next process's state, and jumping to its execution point. Dispatch latency, the time taken to stop one process and start another, must be minimized because it is pure overhead.

Most modern schedulers use a combination of priority, burst time estimation, and aging to make decisions. Since the exact length of the next CPU burst is unknown, the OS predicts it using exponential averaging: predicted_next = α * actual_last + (1 - α) * predicted_last, where α is typically 0.5. This prediction allows algorithms like Shortest Job First to be approximated in practice as Shortest Remaining Time First. Priority values may be static (assigned by users or administrators) or dynamic (adjusted by the OS based on waiting time, I/O behavior, or system load) to prevent starvation and adapt to changing conditions.

The Role of Context Switching and Time Quanta

Context switching is the price of multiprogramming. Each switch requires saving and restoring CPU registers, memory maps, and kernel bookkeeping structures, and it invalidates processor caches. If the time quantum in a round-robin system is too small, the system spends more time switching than executing; if it is too large, the system degenerates to first-come, first-served behavior. Finding the balance — often 10 to 100 milliseconds on general-purpose systems — is crucial for achieving both responsiveness and efficiency.

Preemptive schedulers rely on hardware timer interrupts to enforce quanta. The timer is programmed to interrupt the CPU after a fixed interval, transferring control to the scheduler. Non-preemptive schedulers, in contrast, depend on cooperative yielding, which is simpler but vulnerable to a single misbehaving process monopolizing the CPU. This distinction explains why desktop and server operating systems universally employ preemptive scheduling, while some embedded or cooperative threading libraries use non-preemptive models for simplicity.

3. Implementation

Implementing CPU scheduling algorithms in Python allows us to simulate their behavior, calculate performance metrics, and compare policies without kernel-level programming. The following examples model processes as objects with arrival time, burst time, and priority, and simulate execution over discrete time units. While simplified, these implementations capture the essential logic used in real operating system kernels.

The first implementation demonstrates First-Come, First-Served (FCFS), the simplest non-preemptive algorithm. Processes are executed strictly in order of arrival, making it easy to implement but prone to the convoy effect where a long process delays all others.

# FCFS - First Come First Served (Non-Preemptive)
class Process:
    def __init__(self, pid, arrival, burst):
        self.pid = pid
        self.arrival = arrival
        self.burst = burst
        self.waiting_time = 0
        self.turnaround_time = 0

def fcfs_scheduling(processes):
    # Sort by arrival time - FCFS serves in arrival order
    processes.sort(key=lambda p: p.arrival)
    current_time = 0
    total_waiting = 0
    
    for p in processes:
        # CPU idle if next process hasn't arrived yet
        if current_time < p.arrival:
            current_time = p.arrival
        # Waiting time = start time - arrival time
        p.waiting_time = current_time - p.arrival
        # Execute process to completion (non-preemptive)
        current_time += p.burst
        p.turnaround_time = current_time - p.arrival
        total_waiting += p.waiting_time
        print(f"Process {p.pid}: Waiting={p.waiting_time}, Turnaround={p.turnaround_time}")
    
    avg_waiting = total_waiting / len(processes)
    print(f"Average Waiting Time: {avg_waiting:.2f}")
    return processes

# Example usage
procs = [Process("P1", 0, 5), Process("P2", 1, 3), Process("P3", 2, 8)]
fcfs_scheduling(procs)

The second implementation shows Shortest Job First (SJF), which is optimal for minimizing average waiting time when all jobs are available simultaneously. This non-preemptive version selects the shortest burst among arrived processes. The exponential averaging predictor mentioned earlier would be added here in a production system to estimate burst lengths.

# SJF - Shortest Job First (Non-Preemptive)
def sjf_scheduling(processes):
    # Sort by arrival to process in time order
    processes.sort(key=lambda p: p.arrival)
    n = len(processes)
    completed = []
    current_time = 0
    idx = 0
    ready_queue = []
    
    while len(completed) < n:
        # Add all processes that have arrived by current_time
        while idx < n and processes[idx].arrival <= current_time:
            ready_queue.append(processes[idx])
            idx += 1
        
        if not ready_queue:
            # No process ready, jump to next arrival
            current_time = processes[idx].arrival
            continue
        
        # Select process with shortest burst time
        ready_queue.sort(key=lambda p: p.burst)
        current = ready_queue.pop(0)
        
        current.waiting_time = current_time - current.arrival
        # Ensure waiting time is non-negative
        if current.waiting_time < 0:
            current.waiting_time = 0
        current_time += current.burst
        current.turnaround_time = current_time - current.arrival
        completed.append(current)
        print(f"Process {current.pid}: Burst={current.burst}, Waiting={current.waiting_time}")
    
    return completed

# SJF minimizes average waiting time but may cause starvation
sjf_scheduling([Process("P1", 0, 6), Process("P2", 0, 2), Process("P3", 0, 8)])

Round Robin (RR) is the classic preemptive algorithm for time-sharing systems. Each process receives a fixed time quantum in cyclic order, ensuring fair CPU distribution and bounded response time. The implementation below simulates quantum-based preemption and tracks remaining burst time.

# Round Robin - Preemptive with Time Quantum
from collections import deque

def round_robin_scheduling(processes, quantum=2):
    # Sort by arrival and initialize remaining time
    processes.sort(key=lambda p: p.arrival)
    for p in processes:
        p.remaining = p.burst
        p.completion_time = 0
    
    queue = deque()
    current_time = 0
    idx = 0
    n = len(processes)
    completed = 0
    
    # Add first arriving processes
    queue.append(processes[0])
    idx = 1
    
    while completed < n:
        if not queue:
            # CPU idle, jump to next arrival
            queue.append(processes[idx])
            current_time = processes[idx].arrival
            idx += 1
            continue
            
        current = queue.popleft()
        
        # Execute for quantum or remaining time, whichever is smaller
        exec_time = min(quantum, current.remaining)
        current.remaining -= exec_time
        current_time += exec_time
        
        # Add newly arrived processes during execution
        while idx < n and processes[idx].arrival <= current_time:
            queue.append(processes[idx])
            idx += 1
        
        if current.remaining == 0:
            # Process completed
            completed += 1
            current.completion_time = current_time
            current.turnaround_time = current.completion_time - current.arrival
            current.waiting_time = current.turnaround_time - current.burst
            print(f"Process {current.pid} completed at {current_time}, Waiting={current.waiting_time}")
        else:
            # Not finished, requeue for next round
            queue.append(current)
    
    return processes

round_robin_scheduling([Process("P1", 0, 5), Process("P2", 1, 3), Process("P3", 2, 8)], quantum=2)

Priority scheduling assigns each process a priority value and selects the highest-priority ready process. It can be preemptive or non-preemptive and is the basis for many real-world schedulers. To prevent starvation of low-priority processes, aging gradually increases the priority of waiting processes over time.

# Priority Scheduling (Non-Preemptive) with Aging Concept
class PriorityProcess:
    def __init__(self, pid, arrival, burst, priority):
        self.pid = pid
        self.arrival = arrival
        self.burst = burst
        self.priority = priority  # Lower number = higher priority
        self.waiting_time = 0

def priority_scheduling(processes):
    # Sort by arrival time initially
    processes.sort(key=lambda p: p.arrival)
    n = len(processes)
    current_time = 0
    idx = 0
    ready_queue = []
    completed = []
    
    while len(completed) < n:
        # Admit arrived processes
        while idx < n and processes[idx].arrival <= current_time:
            ready_queue.append(processes[idx])
            idx += 1
        
        if not ready_queue:
            current_time = processes[idx].arrival
            continue
        
        # Aging: increase priority (decrease number) for long-waiting processes
        for p in ready_queue:
            wait = current_time - p.arrival
            if wait > 5:
                # Boost priority to prevent starvation
                p.priority = max(1, p.priority - 1)
        
        # Select highest priority (lowest number), tie-break by arrival
        ready_queue.sort(key=lambda p: (p.priority, p.arrival))
        current = ready_queue.pop(0)
        
        current.waiting_time = current_time - current.arrival
        current_time += current.burst
        current.turnaround_time = current_time - current.arrival
        completed.append(current)
        print(f"Process {current.pid}: Priority={current.priority}, Waiting={current.waiting_time}")
    
    return completed

# Lower priority number means higher urgency
priority_scheduling([
    PriorityProcess("P1", 0, 4, 3),
    PriorityProcess("P2", 0, 2, 1),
    PriorityProcess("P3", 1, 6, 2)
])

4. Step-by-Step Example

To make the behavior of different algorithms concrete, consider a workload of four processes arriving at different times with varying CPU bursts. This example walks through FCFS, SJF (non-preemptive), and Round Robin (quantum = 2) side by side to highlight how scheduling decisions affect waiting and turnaround times.

Workload definition: P1 arrives at time 0 with burst 5, P2 arrives at time 1 with burst 3, P3 arrives at time 2 with burst 8, and P4 arrives at time 3 with burst 6. All times are in milliseconds. We will compute the Gantt chart, waiting time, and turnaround time for each policy.

Under FCFS, execution order is strictly P1, P2, P3, P4. P1 runs from 0 to 5, P2 from 5 to 8, P3 from 8 to 16, and P4 from 16 to 22. Waiting times are: P1 = 0 (starts immediately), P2 = 4 (arrived at 1, started at 5), P3 = 6 (arrived at 2, started at 8), P4 = 13 (arrived at 3, started at 16). Average waiting time is (0+4+6+13)/4 = 5.75 ms. Turnaround times are 5, 7, 14, and 19 respectively, averaging 11.25 ms. The convoy effect is visible: the relatively long P3 delays P4 significantly.

Comparing SJF and Round Robin on the Same Workload

With non-preemptive SJF, the scheduler always picks the shortest burst among available processes. At time 0, only P1 is available, so it runs from 0 to 5. At time 5, the ready queue contains P2 (burst 3), P3 (burst 8), and P4 (burst 6). The shortest is P2, which runs from 5 to 8. Next, between P3 (8) and P4 (6), P4 is shorter and runs from 8 to 14, leaving P3 to run from 14 to 22. Waiting times become: P1=0, P2=4, P4=5, P3=12, averaging 5.25 ms — slightly better than FCFS. Turnaround times are 5, 7, 11, and 20, averaging 10.75 ms.

Round Robin with quantum 2 interleaves execution to improve response time. The Gantt chart is: P1(0-2), P2(2-4), P3(4-6), P1(6-8), P4(8-10), P2(10-11), P3(11-13), P1(13-14), P4(14-16), P3(16-18), P4(18-20), P3(20-22). Response times (first run minus arrival) are: P1=0, P2=1, P3=2, P4=5, averaging just 2 ms, far better for interactive users. However, average waiting time increases to 8.5 ms due to frequent preemptions, and total context switches rise to 11 versus 3 for non-preemptive algorithms. This trade-off illustrates why time-sharing systems favor Round Robin despite its higher overhead.

This walkthrough reveals a critical insight: no single metric tells the full story. SJF optimizes average waiting time but penalizes long jobs and requires burst prediction. FCFS is simple and starvation-free but suffers from convoy effects. Round Robin provides excellent responsiveness and fairness at the cost of higher waiting times and context-switch overhead. Real schedulers therefore blend these ideas, as seen in multilevel feedback queues.

5. Complexity Analysis

Analyzing CPU scheduling algorithms requires evaluating both the computational complexity of the scheduler itself and the performance metrics it optimizes for the workload. The scheduler's overhead directly impacts system scalability: an O(n log n) selection step may be acceptable for tens of processes but becomes a bottleneck with thousands of threads in a data center.

For non-preemptive algorithms like FCFS and SJF, the main cost is sorting or selecting the next process. FCFS requires only a FIFO queue, giving O(1) enqueue and dequeue operations. SJF needs to find the minimum burst among ready processes, which can be done in O(n) with a linear scan or O(log n) with a min-heap, but sorting the ready queue each dispatch costs O(n log n). Preemptive algorithms like Shortest Remaining Time First (SRTF) and Round Robin add the cost of timer interrupts and requeueing, typically O(1) per quantum with a circular queue, but the number of dispatches grows with the number of quanta, potentially O(total_burst / quantum).

Space complexity is generally O(n) for storing the process control blocks and ready queue, regardless of algorithm. However, multilevel feedback queues require multiple queues and additional state for aging and priority adjustment, increasing constant factors but not asymptotic space. The dispatcher itself uses O(1) additional space per context switch.

Beyond scheduler overhead, the algorithms are evaluated on waiting time, turnaround time, and response time. SJF is provably optimal for minimizing average waiting time when all jobs are available at time zero, with an average waiting time that can be O(n) better than FCFS in skewed workloads. Round Robin's average waiting time is typically higher than SJF's but its response time is O(quantum), providing strong latency guarantees. Priority scheduling's performance depends entirely on the priority assignment; with random priorities it behaves like FCFS, while with well-tuned priorities it can approximate SJF or deadline-driven scheduling.

Context switch overhead deserves special attention. Each preemption costs 1 to 100 microseconds depending on hardware and OS, and excessive preemptions can waste significant CPU cycles. If the quantum is set to 10 ms and context switch costs 10 microseconds, overhead is just 0.1%, but with a 1 ms quantum overhead rises to 1%. This is why most systems avoid quanta smaller than 10 ms and why non-preemptive algorithms remain attractive for batch workloads where throughput matters more than responsiveness.

6. Variations and Optimizations

Classic scheduling algorithms are rarely used in isolation in production systems. Instead, they serve as building blocks for more sophisticated, adaptive policies that address their individual weaknesses such as starvation, convoy effects, and poor handling of mixed workloads.

One major optimization is aging, which gradually increases the priority of processes that have waited a long time. In priority scheduling and multilevel queues, a process that waits beyond a threshold has its priority boosted, guaranteeing eventual execution and bounding maximum waiting time. Without aging, a steady stream of high-priority or short jobs could starve long or low-priority jobs indefinitely. Linux's Completely Fair Scheduler (CFS) implements a form of aging through virtual runtime, ensuring each process receives a fair share proportional to its weight.

Multilevel Queue Scheduling partitions the ready queue into several separate queues, each with its own algorithm and priority. For example, system processes might use FCFS in the top queue, interactive processes use Round Robin in the middle queue, and batch jobs use FCFS in the lowest queue. Processes are permanently assigned to a queue, and CPU time is divided among queues using fixed priority or time slicing. This structure isolates workload classes but can still starve lower queues if higher queues are always busy.

Multilevel Feedback Queue and Lottery Scheduling

Multilevel Feedback Queue (MLFQ) extends the multilevel model by allowing processes to move between queues based on observed behavior. A new process starts in the highest-priority queue with a small quantum. If it exhausts its quantum, it is demoted to the next lower queue with a larger quantum, favoring I/O-bound and interactive jobs that block quickly while penalizing CPU-bound jobs that consume full quanta. If a process waits too long in a lower queue, it is aged and promoted upward. MLFQ is the basis for schedulers in Windows, macOS, and many Unix variants because it learns workload characteristics without prior knowledge and approximates SJF for short jobs while remaining fair.

Lottery scheduling and fair-share scheduling represent alternative paradigms. In lottery scheduling, each process holds tickets, and the scheduler randomly draws a ticket to select the next process. Processes with more tickets have proportionally higher chances, enabling flexible, probabilistic fair sharing and straightforward handling of ticket transfers for resource management. Fair-share scheduling extends fairness to users or groups, ensuring each user receives a guaranteed CPU share regardless of how many processes they create, which is critical in multi-tenant cloud environments.

Other notable variants include Earliest Deadline First (EDF) and Rate Monotonic Scheduling for real-time systems, where meeting deadlines is more important than average waiting time, and the Completely Fair Scheduler, which models CPU time as a fluid resource and tracks each process's virtual runtime in a red-black tree to achieve O(log n) selection with near-perfect fairness. These optimizations demonstrate how the simple ideas behind FCFS, SJF, and Round Robin evolve into production-grade schedulers.

7. Real-World Applications

CPU scheduling algorithms are not academic curiosities; they directly influence the performance, responsiveness, and reliability of every computing system you interact with, from smartphones to global cloud platforms. The choice of scheduler shapes user experience, energy consumption, and infrastructure cost.

In general-purpose desktop and mobile operating systems, the scheduler must balance responsiveness for interactive applications with throughput for background tasks. Windows uses a multilevel feedback queue with 32 priority levels, boosting the priority of threads that complete I/O or wait on user input to keep the UI fluid. macOS and iOS use a similar Mach-based multilevel feedback system with additional QoS (Quality of Service) classes that let developers hint whether work is user-interactive, user-initiated, utility, or background. Linux's CFS, introduced in kernel 2.6.23, abandons traditional quanta in favor of virtual runtime accounting, giving each process a weighted fair share and achieving excellent interactivity without manual tuning.

Web servers and application runtimes implement user-level scheduling that mirrors OS principles. Node.js and Nginx use event loops that are essentially non-preemptive, cooperative schedulers, dispatching callbacks in FIFO order but yielding quickly to avoid blocking. Java's ForkJoinPool and Go's goroutine scheduler use work-stealing queues, a variant of multilevel feedback, to balance load across cores. Database engines like PostgreSQL and MySQL schedule queries using priority and shortest-job approximations, favoring short OLTP transactions over long analytical queries to maintain low latency for interactive users.

Network packet scheduling applies identical concepts to bandwidth allocation. Routers use Weighted Fair Queuing (WFQ), Deficit Round Robin, and priority queuing to schedule packets, directly analogous to CPU scheduling. Content delivery networks and load balancers schedule requests across backend servers using shortest-queue-first and round-robin policies to minimize tail latency. Even in machine learning training, GPU schedulers like those in Kubernetes and Slurm use SJF-inspired backfilling to pack jobs efficiently and reduce average job completion time.

Understanding these mappings helps engineers make informed decisions. For instance, setting a Kubernetes CPU limit too low can throttle a latency-sensitive service because the CFS quota enforcement acts like a very small Round Robin quantum, increasing context switches. Recognizing that a database's query scheduler is starving long-running reports may prompt adding aging or separate queues for OLAP workloads. The principles in this overview therefore translate directly into debugging and tuning real systems.

8. Comparison with Alternatives

Choosing a CPU scheduling algorithm involves trading off competing objectives: fairness versus efficiency, responsiveness versus throughput, simplicity versus adaptability. No single algorithm dominates across all criteria, which is why modern systems combine them. The table below provides a structured comparison to guide selection.

FCFS stands out for its simplicity and starvation freedom. Its FIFO queue requires no sorting or priority management, making it trivial to implement and reason about. However, the convoy effect — where a single long CPU burst blocks all subsequent short jobs — leads to poor average waiting time and responsiveness, making it unsuitable for interactive systems. It remains useful for batch queues, print spoolers, and as a building block within lower-priority levels of multilevel systems.

SJF and its preemptive variant SRTF offer the best average waiting time, a mathematically proven optimum. The catch is that they require knowledge of future burst lengths, which must be estimated, and they can starve long jobs if short jobs arrive continuously. SRTF improves responsiveness by preempting a running long job when a shorter one arrives, but it increases context switches. These algorithms are ideal when job lengths are known in advance, such as in HPC batch schedulers with user-provided wall times, or when approximating with exponential averaging yields accurate predictions.

Round Robin solves the responsiveness problem by guaranteeing each process a quantum within n*quantum time, where n is the number of ready processes. This makes it the default for time-sharing, but the frequent preemptions increase overhead and average waiting time compared to SJF. Priority scheduling provides the flexibility to favor important tasks, essential for real-time and differentiated services, but it must be paired with aging to avoid starvation. MLFQ synthesizes the strengths of all alternatives: it behaves like SJF for short jobs, like Round Robin for interactive jobs, and like FCFS for CPU-bound jobs, while aging ensures fairness. Its complexity is justified by its superior general-purpose performance, which is why it underpins most modern operating systems.

9. Common Mistakes and Best Practices

Even experienced engineers make subtle errors when implementing or tuning CPU schedulers, often leading to performance anomalies that are difficult to diagnose. Recognizing these pitfalls and following proven practices can prevent hours of debugging and significant production impact.

A frequent mistake is choosing an inappropriate time quantum for Round Robin or MLFQ. A quantum that is too small causes excessive context switching, cache thrashing, and wasted CPU cycles, while a quantum that is too large makes Round Robin behave like FCFS, destroying interactivity. A good rule of thumb is to set the quantum so that 80% of CPU bursts are shorter than the quantum, typically 10-100 ms for general-purpose systems, and to measure context switch rate and CPU utilization after any change. Another common error is ignoring dispatch latency and assuming scheduling is free; in reality, every preemption has a cost that must be amortized over useful work.

Starvation is another classic failure, especially with SJF and priority scheduling without aging. Developers often assign static priorities and forget that a continuous stream of high-priority work can indefinitely delay low-priority background tasks like garbage collection, logging, or backups, eventually causing memory pressure or data loss. The best practice is to always implement aging or priority inheritance, and to monitor the maximum waiting time and 99th percentile latency, not just the average, to detect starvation early. Similarly, priority inversion — where a low-priority process holds a lock needed by a high-priority process — can cause unbounded blocking; protocols like priority inheritance or priority ceiling should be used whenever locks are involved.

Best Practices for Design and Tuning

• Measure the right metrics: Track CPU utilization, throughput, average and p99 waiting/turnaround/response times, and context switch rate. Optimizing only one metric often degrades others.
• Use exponential averaging for burst prediction: Tune the α parameter (0.3-0.7) based on workload stability; higher α reacts faster to changes but is noisier.
• Isolate workload classes: Use multilevel queues or cgroups to separate latency-sensitive, batch, and background work so they do not interfere.
• Validate with realistic traces: Test schedulers with production burst distributions, not just synthetic uniform workloads; real workloads are often heavy-tailed.
• Account for multicore effects: On multiprocessors, consider affinity, load balancing, and NUMA locality; a single global queue may not scale, while per-core queues need work stealing.

Finally, avoid over-optimizing for average case at the expense of tail latency and fairness. An SJF scheduler that minimizes average waiting time may still be unacceptable if it causes a 10x increase in p99 latency for long jobs. Production systems should enforce fairness guarantees, such as lottery shares or CFS virtual runtime bounds, and use admission control to prevent overload. Documenting scheduling policies, quanta, and priority assignments is also crucial for operability, as these parameters are often the first place to look when latency spikes occur.

10. Interview Questions

CPU scheduling is a staple of operating system interviews for software engineering, systems, and SRE roles. Interviewers use it to assess understanding of concurrency, performance trade-offs, and practical system design. Below are common questions with guidance on what constitutes a strong answer.

1. Why is Shortest Job First optimal for average waiting time, and what are its practical limitations? A strong answer should sketch the proof: swapping any two jobs where the longer job runs first increases total waiting time, so sorting by burst length minimizes the sum of waiting times. Then discuss limitations: it requires knowing future burst lengths (solved via exponential averaging), it can cause starvation of long jobs, and it is non-preemptive in its basic form, so a newly arrived short job must wait. Mention SRTF as the preemptive variant and aging as a mitigation for starvation.

2. How do you choose a time quantum for Round Robin? What happens if it is too small or too large? Explain the trade-off: quantum too small leads to high context switch overhead, cache pollution, and lower throughput, while quantum too large degenerates to FCFS with poor response time. Describe the heuristic that 80% of bursts should be < quantum, typical values of 10-100 ms, and how to measure the impact via CPU utilization and response time. For bonus points, discuss how MLFQ uses multiple quanta (small at high priority, large at low priority) to get the best of both worlds.

3. What is starvation and how do aging and multilevel feedback queues prevent it? Define starvation as indefinite postponement due to continuous arrival of higher-priority or shorter jobs. Explain aging as gradually increasing the priority of waiting processes, guaranteeing a bound on waiting time. Describe MLFQ's mechanism: new jobs start at top priority with small quantum, demoted if they use full quantum, promoted if they wait too long, thus learning whether a job is interactive or CPU-bound without prior knowledge. Contrast with static priority scheduling without aging, which is vulnerable to starvation.

4. Compare preemptive versus non-preemptive scheduling. When would you use each? A good answer contrasts responsiveness and overhead: preemptive scheduling enables bounded response time and fair sharing but incurs context switch cost and requires timer interrupts and synchronization; non-preemptive scheduling is simpler, has lower overhead, and avoids race conditions inside kernel critical sections but risks monopolization and poor interactivity. Give examples: preemptive for desktop, mobile, and server OS (CFS, MLFQ), non-preemptive for cooperative threading, some embedded systems, and batch schedulers where jobs run to completion. Mention that modern kernels are preemptive even in kernel mode for better latency.

5. How does Linux's Completely Fair Scheduler (CFS) differ from traditional Round Robin and Priority schedulers? Explain that CFS models CPU as a fluid resource, tracks each task's virtual runtime (vruntime = actual_runtime * weight), and always picks the task with smallest vruntime using a red-black tree (O(log n)). Unlike fixed quanta and static priorities, CFS provides weighted fair sharing proportional to nice values, has no fixed time slice (slices are dynamic based on number of tasks), and achieves excellent fairness and interactivity without manual quantum tuning. Mention cgroups for hierarchical fair sharing in containers, which traditional schedulers do not natively support.