Parallel Programming

By the Syncfusion DevTutorials team · Last updated October 20, 2026

Use every CPU core: Parallel.For, ForEach and Invoke, thread-local data, PLINQ, degree of parallelism, stopping loops, errors, performance and concurrency.

Introduction to Parallel Programming

Think about marking 400 exam papers.

One teacher marking alone takes all weekend. Four teachers who split the pile, 100 papers each, finish in an afternoon. They each do the same kind of work, just on different papers, at the same time.

That is parallel programming: splitting one big job into pieces and running the pieces at the same time on different CPU cores.

In simple words: parallel programming uses all of your computer's cores to finish heavy calculations faster.

Modern computers have 4, 8, 16, or more cores. A normal for loop uses only one of them. The rest sit idle. The Parallel class and PLINQ let you use all of them with just a few lines of code.

Easy Way to Remember: a for loop is one teacher. Parallel.For is a team of teachers sharing the pile.

What You Will Learn

1. The Parallel class: Parallel.For, Parallel.ForEach, and Parallel.Invoke.
2. Thread-local data, and how to combine parallel results.
3. PLINQ: parallel LINQ with AsParallel().
4. How to control the degree of parallelism.
5. How to stop or break out of a parallel loop.
6. How exceptions work in parallel code.
7. When parallel code is faster, and when it is not.
8. The difference between parallel programming and concurrency.

Start With a Working Example

Press Run.

Check 8 images, using every core

using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading.Tasks;

string[] images = { "a.jpg", "b.jpg", "c.jpg", "d.jpg", "e.jpg", "f.jpg", "g.jpg", "h.jpg" };
var sizes = new ConcurrentDictionary<string, int>();

Parallel.ForEach(images, image =>
{
    sizes[image] = FakeResize(image);   // heavy work on one image
});

foreach (var pair in sizes.OrderBy(p => p.Key))
    Console.WriteLine($"{pair.Key}: {pair.Value} KB");

static int FakeResize(string name)
{
    int work = 0;
    for (int i = 0; i < 200_000; i++) work += i % 7;
    return 100 + name[0] - 'a';
}

Parallel.ForEach distributes the images among several workers. On an 8-core computer, all eight can be processed at the same time. The results are stored in a ConcurrentDictionary because several workers write to it simultaneously.

Try This

Add "i.jpg" to the array. A ninth line appears: i.jpg: 108 KB.

Parallel Class

The Parallel class (in System.Threading.Tasks) runs work on many cores at once. It has three main methods:

Three important differences from normal loops:

• The iterations run in no fixed order. Different iterations may finish at different times.
• The call blocks until every iteration has finished. When it returns, all the work is done.
• The body must be safe to run at the same time as itself. Do not change shared variables unless you protect them.

Parallel.For

Parallel.For(fromInclusive, toExclusive, i => { ... }) works like for (int i = from; i < to; i++), but spreads the iterations across multiple cores.

Fill an array of results in parallel

using System;
using System.Threading.Tasks;

double[] roots = new double[10];

Parallel.For(0, roots.Length, i =>
{
    roots[i] = Math.Sqrt(i);   // each i writes its own slot: safe
});

for (int i = 0; i < roots.Length; i++)
    Console.WriteLine($"sqrt({i}) = {roots[i]:0.000}");

Writing to roots[i] is safe because each iteration writes to a different slot. The to value is exclusive, just like i < roots.Length in a normal loop.

Parallel.ForEach

Parallel.ForEach(items, item => { ... }) works like foreach, but spreads the work across cores. It works with any collection: arrays, lists, dictionaries, or lines from a file.

Score every student in parallel

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

var students = new List<(string Name, int[] Marks)>
{
    ("Asha",  new[] { 78, 85, 92 }),
    ("Ravi",  new[] { 65, 70, 58 }),
    ("Meena", new[] { 90, 95, 88 }),
    ("Kiran", new[] { 40, 55, 61 }),
};

var report = new ConcurrentBag<string>();

Parallel.ForEach(students, student =>
{
    double average = student.Marks.Average();
    string grade = average >= 85 ? "A" : average >= 65 ? "B" : "C";
    report.Add($"{student.Name,-6} {average,5:0.0}  {grade}");
});

foreach (string line in report.OrderBy(l => l))
    Console.WriteLine(line);

Because iterations finish in any order, the results go into a ConcurrentBag and are sorted before printing.

Parallel.Invoke

Parallel.Invoke runs several different pieces of work at the same time, and waits for all of them to finish.

Build three parts of a report at once

using System;
using System.Threading.Tasks;

string summary = "", chart = "", table = "";

Parallel.Invoke(
    () => summary = BuildSummary(),
    () => chart = BuildChart(),
    () => table = BuildTable());

Console.WriteLine(summary);
Console.WriteLine(chart);
Console.WriteLine(table);

static string BuildSummary() => "Summary: sales up 12%";
static string BuildChart() => "Chart: 12 bars drawn";
static string BuildTable() => "Table: 340 rows";

Each action writes its own variable, so there is no shared data to protect. When Parallel.Invoke returns, all three pieces of work are complete.

Thread-Local Data

Imagine summing a million numbers with Parallel.For. If every iteration adds to one shared total variable using a lock, the workers spend most of their time waiting for the lock instead of doing work.

The better way is thread-local data: each worker keeps its own running total. At the end, each worker's total is combined into the shared result, using a lock just once per worker instead of a million times.

Parallel.For has a special overload for this, with three steps:

Each worker keeps its own subtotal

using System;
using System.Threading;
using System.Threading.Tasks;

long total = 0;

Parallel.For(
    1, 1_000_001,
    () => 0L,                                    // localInit: my own subtotal
    (i, state, subtotal) => subtotal + i,        // body: add to my subtotal
    subtotal => Interlocked.Add(ref total, subtotal)); // localFinally: combine once

Console.WriteLine($"Sum of 1..1,000,000 = {total}");

The shared total is touched only once per worker, not a million times, so there is almost no waiting for locks.

There is also a ThreadLocal<T> class, which gives every thread its own private copy of a value. This is useful for objects that are not thread-safe, such as Random in older .NET versions:

var random = new ThreadLocal<Random>(() => new Random());
Parallel.For(0, 100, i => Console.WriteLine(random.Value!.Next(1, 7)));

Aggregating Parallel Results

Aggregating means combining many separate results into one final result: a total, a count, a maximum, or a list.

Count, max and grouped totals from one parallel loop

using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

int[] temperatures = { 31, 35, 29, 38, 33, 41, 36, 30, 28, 39 };

int hotDays = 0;
int maxTemp = int.MinValue;
object gate = new object();
var byBand = new ConcurrentDictionary<string, int>();

Parallel.ForEach(temperatures, t =>
{
    if (t >= 35) Interlocked.Increment(ref hotDays);
    lock (gate) { if (t > maxTemp) maxTemp = t; }
    string band = t >= 35 ? "hot" : t >= 30 ? "warm" : "mild";
    byBand.AddOrUpdate(band, 1, (_, n) => n + 1);
});

Console.WriteLine($"Hot days: {hotDays}");
Console.WriteLine($"Highest: {maxTemp}");
foreach (var b in byBand.OrderBy(b => b.Key))
    Console.WriteLine($"{b.Key}: {b.Value}");

PLINQ

PLINQ (Parallel LINQ) runs a LINQ query on several cores at the same time. You add one method call, .AsParallel(), and the rest of the query stays the same.

LINQ, then the same query with AsParallel

using System;
using System.Linq;

int[] numbers = Enumerable.Range(1, 100_000).ToArray();

int normal = numbers.Where(IsPrime).Count();
int parallel = numbers.AsParallel().Where(IsPrime).Count();
Console.WriteLine($"Primes (LINQ):  {normal}");
Console.WriteLine($"Primes (PLINQ): {parallel}");

int[] firstFive = numbers.AsParallel()
                         .AsOrdered()          // keep the original order
                         .Where(IsPrime)
                         .Take(5)
                         .ToArray();
Console.WriteLine($"First five primes: {string.Join(", ", firstFive)}");

long sumOfSquares = numbers.AsParallel().Sum(n => (long)n * n);
Console.WriteLine($"Sum of squares: {sumOfSquares}");

static bool IsPrime(int n)
{
    if (n < 2) return false;
    for (int d = 2; d * d <= n; d++)
        if (n % d == 0) return false;
    return true;
}

By default, PLINQ results come back in any order. Add AsOrdered() if order is important, or sort the results at the end.

Degree of Parallelism

The degree of parallelism is the number of iterations that may run at the same time. By default, .NET decides automatically, usually about one iteration per core.

You can set a limit with ParallelOptions.MaxDegreeOfParallelism, or WithDegreeOfParallelism in PLINQ. Reasons to limit it:

• Leave some cores free for other work, such as handling a web server's requests.
• Protect a shared resource, such as a disk or database that cannot handle many connections at once.
• Make testing and debugging easier.

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

int running = 0, maxRunning = 0;
object gate = new object();
var options = new ParallelOptions { MaxDegreeOfParallelism = 2 };

Parallel.For(0, 20, options, i =>
{
    lock (gate) { running++; maxRunning = Math.Max(maxRunning, running); }
    int work = 0;
    for (int k = 0; k < 100_000; k++) work += k % 3;
    lock (gate) { running--; }
});

Console.WriteLine($"Never more than 2 at once: {maxRunning <= 2}");

int[] evens = Enumerable.Range(1, 10).AsParallel()
                        .WithDegreeOfParallelism(2)
                        .Where(n => n % 2 == 0)
                        .OrderBy(n => n)
                        .ToArray();
Console.WriteLine($"Evens: {string.Join(", ", evens)}");

This example needs real worker threads, so it is shown as plain code. In the single-threaded browser it would wait forever for a second worker that never comes. Run it in a console app on your own computer.

Setting the degree higher than the number of cores available does not make CPU work faster. The extra workers just take turns using the available cores.

Stopping and Breaking Parallel Loops

A normal loop uses break to stop early. A parallel loop cannot use break because several iterations are running at the same time. Instead, the body can take a ParallelLoopState parameter and call:

The loop returns a result that tells you how it ended:

Stop as soon as a match is found

using System;
using System.Threading.Tasks;

int[] codes = { 12, 47, 33, 99, 15, 99, 64 };
bool found = false;

ParallelLoopResult result = Parallel.For(0, codes.Length, (i, state) =>
{
    if (codes[i] == 99)
    {
        found = true;
        state.Stop();     // no need to look any further
    }
});

Console.WriteLine($"Found 99: {found}");
Console.WriteLine($"Loop ran to the end: {result.IsCompleted}");

Break: everything before the first big number

using System;
using System.Threading.Tasks;

int[] readings = { 3, 5, 8, 120, 6, 4 };

ParallelLoopResult result = Parallel.For(0, readings.Length, (i, state) =>
{
    if (readings[i] > 100) state.Break();
});

Console.WriteLine($"First bad reading at index: {result.LowestBreakIteration}");
Console.WriteLine($"Every reading before it was processed: {result.LowestBreakIteration == 3}");

To stop a parallel loop from outside (for example, when the user clicks a Cancel button), pass a CancellationToken in ParallelOptions. The loop then throws OperationCanceledException:

var options = new ParallelOptions { CancellationToken = cts.Token };
try
{
    Parallel.ForEach(files, options, file => Process(file));
}
catch (OperationCanceledException)
{
    Console.WriteLine("Cancelled by the user");
}

Exception Handling in Parallel Operations

If an iteration throws an exception, the parallel loop stops starting new iterations, waits for the currently running ones to finish, and then throws an AggregateException. This exception holds every exception that happened during the loop, because several iterations may have failed at the same time.

Two bad records, one AggregateException

using System;
using System.Threading.Tasks;

string[] records = { "10", "20", "abc", "40", "xyz" };

try
{
    Parallel.ForEach(records, record =>
    {
        if (!int.TryParse(record, out _))
            throw new FormatException($"Bad record: {record}");
    });
}
catch (AggregateException ae)
{
    Console.WriteLine($"Failures: {ae.InnerExceptions.Count > 0}");
    foreach (Exception ex in ae.InnerExceptions)
        Console.WriteLine($"  {ex.GetType().Name}: {ex.Message.StartsWith("Bad record")}");
}

Because the loop stops early, the number of failures recorded can vary on a multi-core computer. Always loop over InnerExceptions instead of assuming there is exactly one exception.

If you want every item to be processed even when some fail, catch the exception inside the body and record the error instead of throwing:

Keep going: collect errors instead of stopping

using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading.Tasks;

string[] records = { "10", "20", "abc", "40", "xyz" };
var good = new ConcurrentBag<int>();
var errors = new ConcurrentBag<string>();

Parallel.ForEach(records, record =>
{
    try
    {
        good.Add(int.Parse(record));
    }
    catch (FormatException)
    {
        errors.Add(record);
    }
});

Console.WriteLine($"Good total: {good.Sum()}");
Console.WriteLine($"Bad records: {string.Join(", ", errors.OrderBy(e => e))}");

PLINQ works the same way: an exception inside the query is thrown as an AggregateException when you read the results.

Parallel Programming Performance

Parallel code is not automatically faster. Splitting the work, starting workers, and combining results all take time. Parallelism only helps when there is enough work per item to be worth the overhead.

Rules for fast parallel code:

• Give each iteration a reasonable amount of work, not just a few instructions.
• Avoid shared state; use thread-local data and combine once at the end.
• Avoid locks inside the loop body where possible.
• Measure with Stopwatch on your real computer with real data. Never assume parallelism will help.

var sw = Stopwatch.StartNew();
for (int i = 0; i < images.Length; i++) Resize(images[i]);
Console.WriteLine($"Normal:   {sw.ElapsedMilliseconds} ms");

sw.Restart();
Parallel.For(0, images.Length, i => Resize(images[i]));
Console.WriteLine($"Parallel: {sw.ElapsedMilliseconds} ms");

On an 8-core computer, heavy CPU work often runs 4 to 7 times faster in parallel. It is never quite 8 times faster because some time is always spent on coordination and overhead.

Parallel Programming vs Concurrency

These two words are often confused, but they mean different things.

Concurrency means several tasks are in progress at the same time. They may take turns on one core. It is about structure: managing multiple tasks that may start, run, and finish in an interleaved way.

Parallelism means several tasks are running at the exact same instant, on different cores. It is about speed: doing multiple things at the same physical moment to get work done faster.

Think of a chef. A single chef who chops vegetables while the soup simmers and the bread bakes is concurrent: multiple dishes in progress with one pair of hands, working on them one at a time. Four chefs each chopping a different vegetable at the exact same moment are parallel: the same work happening on different people at the same time.

This browser playground demonstrates the difference: with only one core, it can run concurrent code (Chapter 1's async examples interleave their waits), but it cannot run anything truly in parallel.

Easy Way to Remember: concurrency is juggling (one person, many balls); parallelism is more hands (multiple people working at once).

Common mistakes

What to Remember

• Parallel programming splits heavy CPU work across all cores to finish it faster.
• Parallel.For and Parallel.ForEach are parallel loops; Parallel.Invoke runs several different tasks at once. They all wait until the work is done.
• Iterations run in no fixed order, and the body must be safe to run many times at the same time.
• Thread-local data (localInit, localFinally) lets each worker keep its own subtotal and combine once.
• Collect results in array slots, concurrent collections, thread-local totals, or use PLINQ.
• PLINQ is LINQ plus .AsParallel(); add AsOrdered() if order matters.
• MaxDegreeOfParallelism limits how many iterations run at the same time.
• state.Stop() ends a loop as soon as possible; state.Break() finishes every iteration before the break point.
• Parallel loops throw an AggregateException containing every error that occurred.
• Parallel only pays off when there is enough CPU work per item; always measure it.
• Concurrency is many tasks in progress (juggling); parallelism is many tasks at the same instant (more hands).

Next: Advanced Generics