Java Thread Pools and Executors

In the last lesson you learned about Java deadlock. Creating threads by hand with new Thread() works for a few tasks. But with hundreds of tasks, a thread for each one is wasteful and can crash your program. Java’s answer is the executor framework, which manages a pool of reusable threads for you.

🤔 Why not just create threads yourself?

Picture a shop that hires a new worker for each customer, then fires them. Hiring and firing takes more effort than the work. A real shop keeps a small team who serve customer after customer all day.

Creating a thread per task is that silly hiring loop. A thread is not free:

• new Thread() makes the operating system set aside memory for the thread’s stack, register it, and schedule it, then tear it all down again.
• For one or two tasks you never notice. For a thousand, it hurts.

Suppose a web server gets a thousand requests at once and you start a thread for each:

• Each thread reserves its own memory, so a thousand can use up all your memory and crash with OutOfMemoryError.
• Starting and stopping threads over and over is slow, so time goes into setup instead of real work.
• You now have a thousand threads to track by hand, which becomes impossible to manage.

The fix is to keep a small team and let them reuse themselves. That is what a thread pool does.

🍽️ The restaurant analogy

A restaurant with three waiters serves a hundred customers all evening. It does not hire a hundred waiters. That is a thread pool in one picture:

• The waiters are the worker threads. There is a small, fixed number.
• The customers are the tasks. There can be far more tasks than waiters.
• The line of waiting customers is the task queue. Tasks sit there until a waiter is free.

Three workers handle a hundred jobs by reusing themselves. That is why a pool is faster and safer than one thread per task.

🧩 What is the executor framework?

The executor framework is the set of classes Java gives you to run tasks on a pool of threads. It separates two things that used to be tangled:

• What you want to run, written as a Runnable or a lambda.
• How it runs, which is the thread pool the framework manages.

You only write the task and hand it over. The framework owns the threads. Three pieces you will use:

• A thread pool is the set of reusable worker threads. You rarely touch it directly.
• An ExecutorService is the manager you talk to. You submit tasks and it hands them to free threads.
• The Executors class is a factory with static methods that build common pools for you.

You say “run this task.” The ExecutorService finds a free thread, or the task waits in the queue until one is free.

🏭 The factory methods in Executors

You do not build a pool with new. You call a factory method on the Executors class, which returns a ready-made ExecutorService. Each method gives a pool with a different personality.

This snippet creates one of each common pool type so you can see them side by side.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

// exactly n threads, always
ExecutorService fixed = Executors.newFixedThreadPool(4);

// just one thread, tasks run one after another in order
ExecutorService single = Executors.newSingleThreadExecutor();

// grows and shrinks with demand, reuses idle threads
ExecutorService cached = Executors.newCachedThreadPool();

Here is when to reach for each one.

• newFixedThreadPool(n) keeps exactly n threads alive for the whole life of the pool. Extra tasks wait in the queue. This is the safe default. You decide the limit, so you can never accidentally start ten thousand threads.
• newSingleThreadExecutor() is a pool of one. Tasks run strictly one at a time, in the order you submitted them. Use it when order matters or when the tasks must not overlap.
• newCachedThreadPool() makes new threads when all current ones are busy, and quietly retires threads that sit idle for a while. It is great for lots of short tasks. The catch is there is no built-in cap, so a sudden flood of slow tasks can create a lot of threads. Use it only when tasks are short.

For modern Java, Java 21 added Executors.newVirtualThreadPerTaskExecutor(), which runs each task on its own virtual thread. Virtual threads are so cheap that one per task is fine, even for thousands. Good to know it exists.

🛠️ Creating a thread pool and submitting tasks

We make a pool with Executors.newFixedThreadPool(n) and hand it work. There are two ways to hand over a task:

• execute(Runnable) fires off a task and gives nothing back. Use it for fire-and-forget work.
• submit(...) accepts a Runnable or Callable and returns a Future, a handle to check if the task finished or grab its result. We meet Future in the next lesson.

Let’s run five tasks on a pool of two threads.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
    public static void main(String[] args) {
        // a pool of 2 reusable threads
        ExecutorService pool = Executors.newFixedThreadPool(2);

        for (int i = 1; i <= 5; i++) {
            int taskId = i;
            pool.submit(() -> {
                System.out.println("Running task " + taskId
                    + " on " + Thread.currentThread().getName());
            });
        }

        pool.shutdown();   // no new tasks; finish the queued ones
    }
}

What happens:

• We create a pool of two threads, then submit five tasks, each a lambda that prints its id and thread.
• Two threads pick up the first two tasks. The other three wait in the queue.
• As soon as a thread finishes, it grabs the next waiting task. Two workers handle all five jobs.
• shutdown() tells the pool we are done, and it stops once the queue is empty.

The output shows tasks sharing just two thread names, like pool-1-thread-1. That is the reuse in action. The order changes from run to run, because the two threads race to grab the next task.

Running task 1 on pool-1-thread-1
Running task 2 on pool-1-thread-2
Running task 3 on pool-1-thread-1
Running task 4 on pool-1-thread-2
Running task 5 on pool-1-thread-1

Notice there is no new Thread() anywhere. You never start, name, or clean up a thread. You only describe the work. The framework owns the threads.

🛑 Shutting down the pool

Skipping this is the most common executor bug. You must shut down the pool, or your program may never end.

Remember the restaurant. Even with no customers, the waiters stay, waiting for the next one. Pool threads do the same. After your last task finishes, the threads stay alive watching the queue. As long as they are alive, the program counts as still running, so it refuses to exit and just hangs.

You end a pool with one of two methods.

pool.shutdown();              // finish queued tasks, then stop
// pool.shutdownNow();        // try to stop everything immediately

Here is the difference between them.

• shutdown() is the polite one. It stops accepting new tasks, lets everything already submitted finish, and then ends the threads. This is what you want almost every time.
• shutdownNow() is the firm one. It tries to stop right away. Queued tasks that have not started are dropped, and running tasks get interrupted. Reach for this only when you truly need to stop fast.

One more method. After shutdown(), call awaitTermination(...) to block until the pool finishes all tasks, or a time limit runs out. Handy when the main thread must wait for the work before moving on.

This example shuts the pool down, then waits up to a minute for it to drain.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

ExecutorService pool = Executors.newFixedThreadPool(3);
// ... submit tasks here ...

pool.shutdown();   // stop taking new work, finish what is queued
try {
    // wait up to 60 seconds for all tasks to finish
    if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
        pool.shutdownNow();   // still not done? force it
    }
} catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt();
}

This is the safe shutdown pattern. Ask politely with shutdown(), wait with awaitTermination(), and only force a stop with shutdownNow() if the wait runs out.

🎯 Choosing the right pool

Now let’s tie the choice to real situations, because picking the wrong pool causes real trouble.

This single-thread example shows the simplest case, where tasks must run in order.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

ExecutorService single = Executors.newSingleThreadExecutor();
single.submit(() -> System.out.println("Runs alone, in order"));
single.submit(() -> System.out.println("Then me, after the first one"));
single.shutdown();

A single-thread executor runs tasks one at a time, in submit order. That is what you want when one task depends on the one before it, or when two tasks must never touch the same data at once.

A quick way to decide:

• Steady, predictable workload where you want a firm limit on threads? Use newFixedThreadPool(n).
• Tasks that must run one after another, in order? Use newSingleThreadExecutor().
• A burst of many small, quick tasks? Use newCachedThreadPool().
• On Java 21 or newer with lots of tasks that mostly wait on input or output? Look at newVirtualThreadPerTaskExecutor().

For sizing a fixed pool: tasks that mostly use the processor want a size near the number of cores. Tasks that mostly wait, like a network call, can go higher, because the threads spend most of their time idle.

⚠️ Common Mistakes

A few executor slip-ups trip up almost everyone at least once.

Forgetting to shut down. The classic one. Without shutdown(), the pool’s threads stay alive and the program hangs forever.

// ❌ no shutdown, the program never exits
ExecutorService pool = Executors.newFixedThreadPool(2);
pool.submit(() -> System.out.println("done"));

// ✅ shut it down when finished
ExecutorService pool = Executors.newFixedThreadPool(2);
pool.submit(() -> System.out.println("done"));
pool.shutdown();

Sizing the pool wrong. A pool of thousands of threads throws away the whole benefit, while a pool of one when you wanted parallel work makes everything slow and serial.

// ❌ a giant pool can swamp the machine
ExecutorService pool = Executors.newFixedThreadPool(5000);

// ✅ size it to the work and the hardware
int cores = Runtime.getRuntime().availableProcessors();
ExecutorService pool = Executors.newFixedThreadPool(cores);

Swallowing exceptions inside tasks. If a task throws and you catch nothing, the error can vanish silently with submit, because the exception is stored in the Future instead of being printed. So a task can fail and you never find out.

// ❌ if this blows up, you may never see the error
pool.submit(() -> {
    riskyWork();   // throws, but the exception hides in the Future
});

// ✅ handle problems inside the task so they are visible
pool.submit(() -> {
    try {
        riskyWork();
    } catch (Exception e) {
        System.out.println("Task failed: " + e.getMessage());
    }
});

✅ Best Practices

Habits that keep executor code clean and safe.

• Use a thread pool for many tasks. Let the framework reuse threads instead of creating one per task with new Thread().
• Always call shutdown(). Clean up the pool when you are done submitting work, and use awaitTermination() if you need to wait for it to finish.
• Choose the right pool type. Fixed for steady limits, single for ordered tasks, cached for many short ones.
• Size fixed pools deliberately. Tie the number of threads to your cores and the kind of work, not to a guess.
• Handle exceptions inside tasks. Wrap risky work in try/catch so failures do not disappear silently.
• Keep tasks small and focused. Submit Runnables or short lambdas. One task should do one clear job.

🧩 What You’ve Learned

Nicely done. Let’s recap the executor framework.

• ✅ The executor framework runs many tasks on a pool of reusable threads, instead of one thread per task.
• ✅ An ExecutorService is the manager you submit tasks to; the Executors class has factory methods that build pools.
• ✅ Common pools: fixed, single-thread, and cached, each for a different need, plus virtual threads on modern Java.
• ✅ Use execute for fire-and-forget work and submit when you want a Future back.
• ✅ Always call shutdown() when done, optionally with awaitTermination(), or the program may never end.

🚀 What’s Next?

You have met the common pools at a glance. Next we go deeper into ExecutorService and each pool type, so you know exactly which one to reach for and how to tune it.

Java ExecutorService and Thread Pool Types