Java Virtual Threads: Solving the Thread-per-Request Problem
Introduction
For years, Java developers have relied on the traditional thread-per-request model to build scalable backend applications. Every incoming HTTP request gets assigned a thread.
That thread handles the request, executes business logic, communicates with databases, calls downstream services, and finally sends the response. This model works well for many applications.
But what happens when your application receives thousands of concurrent requests? Traditional platform threads are relatively expensive resources. Creating and managing thousands of threads consumes memory, increases context switching, and can eventually limit application scalability.
This is where Java Virtual Threads, introduced as a preview feature in Java 19 and finalized in Java 21 (JEP 444), change the game. Virtual Threads allow Java applications to handle large numbers of concurrent tasks with significantly lower thread-management overhead.
Let's understand how they work and why they matter for modern microservices.
1. The Problem with Traditional Platform Threads
Traditionally, Java applications use platform threads. A platform thread is backed by an operating-system thread.
Consider a Spring Boot REST API handling customer account requests.
@RestController
@RequestMapping("/accounts")
public class AccountController {
@GetMapping("/{id}")
public Account getAccount(@PathVariable Long id) {
return accountService.getAccount(id);
}
}
Under a traditional thread-per-request model:
- A request arrives.
- The server assigns a platform thread.
- The thread executes business logic.
- It queries the database.
- It waits for the database response.
- It returns the response.
The biggest problem is that the thread may spend most of its time waiting for I/O operations. For example:
- Database queries
- REST API calls
- Kafka-related blocking operations
- External payment gateway calls
- File operations
While waiting, the platform thread remains occupied.
What happens under high concurrency? Imagine:
- 10,000 concurrent HTTP requests.
- Each request takes 500 ms.
- 400 ms is spent waiting for external I/O.
- 100 ms is spent executing business logic.
A large number of platform threads may remain occupied while waiting for I/O. Increasing the thread pool size is not an unlimited solution. It introduces additional overhead:
- Higher memory consumption.
- More operating-system thread scheduling.
- Increased context switching.
- Thread pool saturation.
- Reduced application throughput under heavy load.
The real challenge is not necessarily CPU processing. It is managing concurrency efficiently when requests spend significant time waiting.
2. What Are Java Virtual Threads?
Virtual Threads are lightweight threads managed by the JVM rather than being directly mapped one-to-one to operating-system threads. They were finalized in Java 21 through JEP 444.
Unlike platform threads, millions of virtual threads can potentially be created without requiring millions of operating-system threads. The JVM schedules virtual threads onto a smaller number of platform threads called carrier threads.
Think of it this way:
- Virtual Thread: Represents an application task.
- Carrier Thread: Platform thread that executes a virtual thread.
- JVM Scheduler: Coordinates scheduling of virtual threads.
When a virtual thread performs a supported blocking operation, it can be suspended, allowing its carrier thread to execute another virtual thread. When the operation completes, the suspended virtual thread becomes eligible to resume.
This makes blocking-style code much more scalable for I/O-intensive workloads.
3. Platform Threads vs Virtual Threads
Feature Platform Threads Virtual Threads Managed by OS/JVM JVM OS thread mapping Typically 1:1 M:N scheduling Creation cost Relatively high Very low Memory overhead Higher Much lower Large concurrency Resource-intensive More efficient Best suited for CPU-intensive tasks I/O-intensive tasks Programming model Blocking or asynchronous Blocking-style code
Important: Virtual Threads do not make CPU-intensive operations faster. They primarily improve scalability when many tasks spend time waiting.
4. Creating Virtual Threads in Java 21
Java 21 provides simple APIs for creating virtual threads.
Approach 1: Thread.ofVirtual()
public class VirtualThreadExample {
public static void main(String[] args) throws Exception {
Thread thread = Thread.ofVirtual().start(() -> {
System.out.println("Running on: " + Thread.currentThread());
});
thread.join();
}
}
Approach 2: Thread.startVirtualThread()
public class VirtualThreadExample {
public static void main(String[] args) {
Thread.startVirtualThread(() -> {
System.out.println("Hello from Virtual Thread");
});
}
}
Approach 3: Virtual Thread Executor
For application-level task execution, Java provides a dedicated executor.
import java.util.concurrent.*;
public class VirtualThreadExecutorExample {
public static void main(String[] args) throws Exception {
try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
for (int i = 0; i < 1000; i++) {
executor.submit(() -> {
System.out.println(Thread.currentThread());
});
}
}
}
}
Each submitted task gets its own virtual thread. Unlike a fixed-size thread pool, this executor is designed around creating a new virtual thread per task.
5. How Virtual Threads Work Internally
Understanding the internal architecture is important for experienced Java developers. Virtual Threads use a scheduling mechanism involving carrier threads.
Consider this execution flow:
- A virtual thread is created.
- The JVM schedules it onto a carrier platform thread.
- The virtual thread starts executing application code.
- It reaches a supported blocking I/O operation.
- The virtual thread is suspended.
- The carrier thread becomes available for another virtual thread.
- Once the I/O operation completes, the virtual thread becomes runnable again.
- The scheduler assigns it to a carrier thread to continue execution.
This process is called unmounting and mounting.
- Mounting: A virtual thread runs on a carrier thread.
- Unmounting: A virtual thread temporarily releases its carrier thread.
This is one of the fundamental mechanisms that makes virtual threads efficient.
Important internal concepts
Carrier Threads
These are ordinary platform threads that execute virtual threads.
Continuation
A continuation represents the execution state of a virtual thread, allowing it to suspend and resume.
ForkJoinPool Scheduler
The default virtual thread scheduler uses a ForkJoinPool implementation to schedule virtual threads onto carrier threads.
6. Using Virtual Threads in Spring Boot 3.2+
Spring Boot 3.2 introduced built-in support for virtual threads. With Java 21 and Spring Boot 3.2+, enabling virtual threads is straightforward.
Add the following configuration to application.properties:
spring.threads.virtual.enabled = true
That's it! Spring Boot can configure supported task execution and scheduling infrastructure to use virtual threads. For Spring MVC applications using embedded Tomcat or Jetty, supported request-processing infrastructure can also use virtual threads.
Example: Spring Boot REST Controller
@RestController
@RequestMapping("/payments")
public class PaymentController {
private final PaymentService paymentService;
public PaymentController(PaymentService paymentService) {
this.paymentService = paymentService;
}
@PostMapping
public ResponseEntity<String> processPayment(@RequestBody PaymentRequest request) {
paymentService.processPayment(request);
return ResponseEntity.ok("Payment processed successfully");
}
}
The controller can continue using a familiar synchronous programming model. Virtual threads help the application manage concurrent blocking requests more efficiently.
Note: Enabling virtual threads does not automatically make every component of an application virtual-thread-aware. Verify the behavior of your server, task executors, libraries, and integrations.
7. Real-World Fintech Example: Payment Processing
Consider a banking application processing payment requests. A single payment request may involve:
- Customer authentication.
- Account balance validation.
- Fraud detection service invocation.
- Database transaction.
- External payment gateway integration.
- Transaction status update.
- Response generation.
Several operations involve waiting for external systems.
Traditional platform thread model Incoming Payment Request | v Platform Thread | v Account Validation | v Fraud Detection API | v Database Operation | v Payment Gateway | v Response During blocking operations, the platform thread may remain occupied.
Virtual thread model Incoming Payment Request | v Virtual Thread | v Account Validation | v Fraud Detection API | | Waiting v Virtual Thread Suspended | Carrier Released | v I/O Operation Completes | v Virtual Thread Resumes | v Database + Payment Gateway | v Response
The application can support more concurrent requests without requiring a corresponding number of platform threads. However, database connection pools, payment gateway rate limits, and downstream service capacity still impose limits.
Virtual Threads improve concurrency management, not the capacity of external dependencies.
8. Virtual Threads vs CompletableFuture vs Reactive Programming
This is a common interview discussion.
Feature Virtual Threads CompletableFuture Reactive Programming Programming style Synchronous Asynchronous Non-blocking reactive Code readability Simple Can become complex Requires reactive concepts Blocking operations Supported Depends on execution strategy Generally avoided Thread management JVM-managed Executor-managed Event-loop based Best use case Blocking I/O workloads Async task composition End-to-end non-blocking systems
When should you choose Virtual Threads?
Choose virtual threads when:
- Your application uses blocking JDBC.
- You have many synchronous REST API calls.
- Your existing codebase uses imperative Spring MVC.
- You want to improve concurrency without rewriting everything into reactive programming.
When should you consider Reactive Programming?
Consider reactive programming when:
- You need end-to-end non-blocking I/O.
- You are using reactive database drivers.
- You have complex streaming pipelines.
- Backpressure is an important requirement.
Virtual Threads and Reactive Programming solve related but different problems. They are not direct replacements for every other concurrency model.
9. Important Limitations and Best Practices
Virtual Threads are powerful, but they are not a magic solution.
- CPU-intensive workloads
Virtual Threads do not increase CPU capacity. If your application performs heavy encryption, image processing, or complex calculations, CPU cores remain the primary limitation.
- Database Connection Pool Limits
Suppose your application creates 10,000 virtual threads but has only 50 database connections. Only 50 operations can actively use those connections at a time. The remaining operations may wait for connection availability.
Tune database connection pools according to database capacity and workload.
- Avoid Thread Pooling Virtual Threads
Virtual Threads are designed to be created per task. Avoid treating them like scarce platform threads that need to be reused in a fixed-size pool.
- Avoid Excessive ThreadLocal Usage
Virtual Threads support ThreadLocal variables, but creating millions of virtual threads with large ThreadLocal values can increase memory consumption. Review existing ThreadLocal usage before migration.
- Understand Pinning
Pinning happens when a virtual thread cannot unmount from its carrier thread during certain operations. In Java 21, synchronized blocks and methods can cause pinning when a virtual thread blocks while holding a monitor. Native or foreign function calls can also pin virtual threads.
Review long-running blocking operations inside synchronized sections. Do not blindly replace every synchronized block with another synchronization mechanism.
- Virtual Threads Do Not Replace Rate Limiting
If a downstream payment gateway supports only 500 concurrent requests, creating 10,000 virtual threads will not increase its capacity. Use:
- Rate limiting.
- Bulkheads.
- Timeouts.
- Connection pooling.
- Circuit breakers.
Concurrency management must still respect downstream system limits.
10. Monitoring Virtual Threads in Production
Before introducing virtual threads into production, monitor the application carefully.
Important metrics include:
- Active virtual thread count.
- Request throughput.
- API response time (P95/P99).
- Database connection pool utilization.
- CPU utilization.
- Heap memory consumption.
- GC frequency and pause times.
- Downstream API latency.
- Thread pinning events.
Useful tools:
- Java Flight Recorder (JFR).
- Java Mission Control (JMC).
- Thread dumps.
- Spring Boot Actuator.
- Micrometer and Prometheus.
- Grafana.
For Java 21, JFR includes virtual-thread-related events that help investigate pinning and scheduling behavior.
11. When Should Enterprises Adopt Virtual Threads?
For enterprise Java applications, virtual threads are particularly useful when workloads are I/O-bound and use a blocking programming model.
Examples include:
- Banking transaction APIs.
- Payment processing systems.
- Customer account services.
- Loan processing applications.
- Insurance policy services.
- Enterprise integration services.
Before migration, evaluate:
- Current thread pool saturation.
- Request concurrency.
- Blocking I/O patterns.
- Database connection pool limits.
- Downstream API capacity.
- Performance under realistic load.
Run controlled load tests comparing platform threads and virtual threads. Measure throughput, latency, memory consumption, and resource utilization rather than assuming virtual threads will always improve performance.
Conclusion
Java Virtual Threads represent one of the most significant improvements to Java concurrency in recent years. They make it possible to write simple, blocking-style code while efficiently managing large
Comments
No comments yet. Start the discussion.