Understanding DispatchQueues

Understanding DispatchQueues

What on earth are these "DispatchQueues"?
Why do I have to use it to send UI code to the main thread? It clearly still works if I do nothing.
What's the point of these "Quality of Services" queues? I use .main for everything and never had a problem.
Why do I get a crash if I call DispatchQueue.main.sync? What's the point of it?
Just what is this Main Thread anyway?

If you're developing iOS apps for more than a few weeks, then you have probably dealt with concurrent code before. If you have no background on Operating Systems, you may have asked yourself one of these questions.

Multithreading in general is a difficult thing to fully understand, but understanding how the CPU deals with concurrency is the key to writing good, fast code that does what you expected it to do. Otherwise, you might be abusing your user's CPUs but thinking everything is fine because they are too fast for you to notice that something is wrong.

But before we can answer these questions, we need to take a step back and understand how things work behind the scenes.

What's a Process?

The definition of a process is quite simple: it is a running program. Your app is a process, Slack is a process, Safari is a process, and so on. It contains a list of instructions (your code in assembly format) and sits there on your disk until the user wishes to run it. The OS will then load that process into memory, start an instruction pointer that tells us which instruction of the program is currently being executed, and have the CPU sequentially execute its instructions until they end, terminating the process.

        Address space of a single thread process
|- - - - - - - - - - - - - - - - - - - - - - - - - - |
|                    Instructions                    |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|                    Global Data                     |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|           malloc'd data (Reference Types)          |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
| Nothing (Stack and malloc'd data grow towards here)|
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|   Stack (Value Types (if possible), args, returns) |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -

Each process gets its own section of physical memory dedicated to itself. They do not share these addresses with other processes.

I'm reading something on Safari while listening to Spotify. How can the CPU run several processes at the same time?

It can't. What you are experiencing is an illusion caused by the absurd speed of a modern CPU!.

A CPU simply cannot do two things at the same time. Things are slightly different for CPUs with multiple cores, but for simplicity, let's assume we only have one core; what happens is that it executes something in Safari, then something in Spotify, then something in iOS, then something in Safari again, and so on. The OS will save whatever the CPU was doing for a specific process in memory (in the form of registers and pointers), decide what will be the next process to run, retrieve what it was doing for that process, have the CPU run it for a while, and repeat. This is called a context switch and it happens very, very quickly, giving the impression it can actually run several things at once. (In CPUs with multiple cores the work can be divided between the cores, actually doing several things at once. However, the same principles apply when all the cores are in use.)

The exact way the OS decides what should be the next process to run is rather complex (read the book at the end of the article if you're interested), but what you should know is that it's possible to dictate manually what's the "priority" of something in our app. (Are iOS's "Quality of Services" starting to making sense now?)

What's a Thread?

Instead of the classic concept of a single thread process that starts at a main() function and ends at some exit() a few lines below, a multi-threaded program has more than one point of execution (each of which is being fetched and executed from). Perhaps another way to think of this is that each thread is very much like a separate process, except for one difference: they share the same address space and thus can access the same data.

        Address space of a multi-threaded process
|- - - - - - - - - - - - - - - - - - - - - - - - - - |
|                    Instructions                    |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|                    Global Data                     |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|           malloc'd data (Reference Types)          |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
| Nothing (Stack and malloc'd data grow towards here)|
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|                 Stack of Thread 2                  |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -
|                 Stack of Thread 1                  |
|- - - - - - - - - - - - - - - - - - - - - - - - - - -

Just like processes, a CPU cannot run two threads at the same time - they are instead targeted by context switches just like processes. The CPU runs something in Safari's Thread 1 (which is doing some UI updates), then something in Spotify's Thread 3 (which is downloading a song), then something in Safari's Thread 2 (which is pinging a DNS), and so on.

iOS: The Main Thread

Your iOS app has several threads. The Main Thread is simply the intial starting point of execution in your app (starting for you at didFinishLaunchingWithOptions). The Main Thread executes a loop every frame (a RunLoop) that draws the current screen if needed, handles UI events such as touches and executes the contents of the DispatchQueue.main. It keeps doing this until the app is terminated. It has extremely high priority - pretty much anything on it gets executed immediately. That's part of the reason why UI code is expected to be executed in the Main Thread - by executing some UI-changing code outside of it, your code might start running properly only to suddenly get context switched for several miliseconds because something more important arrived to the OS (like a notification). Your UI updates will then be delayed, giving a bad experience to your users. (In practice the topic of threading requirements for UI code is a bit more complicated than this, but for this purposes of this article we can assume it to be just as described.)

However, you can't simply execute everything on the Main Thread. Since this thread deals with everything related to screen draws / UI updates, if you run a huge task on it, it won't be able to do anything else until it ends. That's why we need several threads (points of execution) to begin with.

@IBAction func actionOne(_ sender: Any) {
    //Button actions are in the Main Thread.
    //This takes about 5 seconds to finish
    var counter = 0
    for _ in 0..<1000000000 {
        counter += 1
        //The screen is totally frozen here. How can I scroll my screen (an UI action)
        //If I blocked the thread by doing this meaningless thing?
        //The scroll action is waiting to be run, but it can't because it's also a Main Thread action.
        //You can't simply context switch actions on the same thread.
        //This needs to be run in a different thread.
    }
}

iOS: Background Threads and DispatchQueues

A background thread is anything that is not the Main Thread. They can run alongside the Main Thread (like they were a different process, but remember the definition of a thread!), dealing with complex tasks without interferring with the Main Thread's UI updates. In iOS, the safest way of spawning a background thread is to use DispatchQueues. However, be aware that DispatchQueues are not threads - they are objects that manage a pool of threads. The queue will automatically create and reuse threads as it finds necessary in order to perform the work that you submit to it, abstracting from you the hassle of spawning threads manually and dealing with the potential issues of doing so.

Work dispatched to DispatchQueue.main will always be serially executed (that is, action 2 only happens after action 1 ends) on the Main Thread, while work dispatched to DispatchQueue.global(qos:) will concurrently (everything at the same time) perform work into one or more threads running in parallel (depending on the amount of work submitted to the queue) with priority equal to the priority of the selected QoS. If you'd like custom behavior (such as a queue that performs work on a background thread, but serially), you can create your own DispatchQueue.

You should be aware however that global queues may also perform work in the Main Thread! A very common misconception in iOS is to think that work dispatched to global queues will always run in a background thread, which is simply not true! Global queues can and will use the Main Thread every once in a while, so you should never write code that assumes that Main Thread == Main Queue. The following snippet is an example of this mistake that is present in many projects, and should be removed ASAP:

// Never write this:

if Thread.isMainThread {
    // We are 100% running on DispatchQueue.main!
}

Background Queue Priorities (QoS)

By assigning a Quality of Service to an action, you indicate its importance, and the system prioritizes it and schedules it accordingly.

Because higher priority work is performed more quickly and with more resources than lower priority work, it typically requires more energy than lower priority work. Accurately specifying appropriate QoS classes for the work your app performs ensures that your app is responsive as well as energy efficient.

There are a few levels of QoS for background threads for several different kinds of actions, but none with higher priority than the Main Thread. These are the currently available Quality of Services:

  • UserInteractive
  • UserInitiated
  • Utility
  • Background

Visualizing the impact of different QoS levels

By using Instruments, we can see how the different QoS levels affect the execution of our code.

Heavy task on the Main Thread

@IBAction func actionOne(_ sender: Any) {
    //We already are in the main thread, but we will use a dispatch operation
    //to see how long it takes for the task to begin.
    DispatchQueue.main.async { [unowned self] in
        self.timeIntensiveTask()
    }
}

The task got executed instantly after I pressed the IBAction, and took about 5 seconds to complete. However, the entire screen was frozen, as we blocked the thread.

Heavy task on an UserInitiated QoS thread

@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.global(qos: .userInitiated).async { [unowned self] in
        self.timeIntensiveTask()
    }
}

A new thread spawned, and the task got executed almost instantly after I pressed the IBAction, also taking about 5 seconds to complete. No screen freeze this time! This thread is completely independent.

Heavy task on a Background QoS thread

@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.global(qos: .background).async { [unowned self] in
        self.timeIntensiveTask()
    }
}

Just like UserInitiated, a thread got spawned, but in this case, not only it took some time for the task to start - and it took almost 10 seconds for it to end! This lower priority thread had delayed and reduced access to system resources. However, this is good: If you're sending a task to a background QoS queue, it means you don't want to ruin your user's CPU by focusing on it.

Visualizing Serial Queues versus Concurrent Queues

@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.main.async { [unowned self] in
        self.timeIntensiveTask()
    }
    DispatchQueue.main.async { [unowned self] in
        self.timeIntensiveTask()
    }
    DispatchQueue.main.async { [unowned self] in
        self.timeIntensiveTask()
    }
}
@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.global(qos: .background).async { [unowned self] in
        self.timeIntensiveTask()
    }
    DispatchQueue.global(qos: .background).async { [unowned self] in
        self.timeIntensiveTask()
    }
    DispatchQueue.global(qos: .background).async { [unowned self] in
        self.timeIntensiveTask()
    }
}

DispatchQueue.sync vs DispatchQueue.async

If the concept of a multi-threaded process wasn't mind-boggling enough, we need to careful with the definition of .async and .sync operations.

A common misconception is to think that DispatchQueue.async means executing something in background, and that's not true.

What will be the output on actionOne()?

@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.main.async { [unowned self] in
        print("async started")
        self.timeIntensiveTask()
        print("async ended")
     }
     print("sync task started")
     timeIntensiveTask()
     print("sync task ended")
}

private func timeIntensiveTask() {
    var counter = 0
    for _ in 0..<1000000000 {
        counter += 1
    }
}

The answer will always be:

sync task started
sync task ended
async task started
async task ended

If you thought the two tasks would start together, just think about the context of this method: we are dispatching a task to the Main Thread, but actionOne is already on the Main Thread! There's no way a thread can run two sequences of instructions at the same time, that's why we have different threads.

The async task will also only execute after the sync task (and never before) because DispatchQueue.main tasks will only start executing at the end of the Main Thread's RunLoop - which is blocked by our sync task. If actionOne happened to be in a different thread or the async task happened to be in a different DispatchQueue, the tasks would start together in an order dependant to how fast the async task would be dispatched.

What DispatchQueue.async means is: Make sure this task is eventually executed on thread X (main, or any other global background thread depending on what queue you are using), but I don't care about the details. I'll keep doing my stuff.

On the contrary, DispatchQueue.sync means is: Make sure this task is eventually executed on thread X. Please warn me when you do so, because I will also block myself (the calling thread) until this task finishes running.

Given that, what do you think will be the output of the following actionOne()?

@IBAction func actionOne(_ sender: Any) {
    DispatchQueue.main.sync { [unowned self] in
        print("a")
    }
    print("b")
}

A sync task is forwarded to the queue, and the main thread will freeze until "a" gets printed. The task gets sent to the Main Thread, which is frozen because it's waiting for the task to run. But the task can't run, because the thread is frozen waiting for the task to run, and on and on and on until your app decides to crash. You can't call sync dispatches from the thread itself, it has to come from somewhere else. Nothing will get printed here. As you most likely know, this is called a deadlock.

What else?

Concurrency is a very complex topic, and DispatchQueues are just one way to approach it in iOS. If you want to more about it, check out SwiftRocks's article about Thread Safety!

References and Good reads

Operating Systems: Three Easy Pieces
Apple Docs: QoS
Apple Docs: Thread Management
Basic of Parallel Programming with Swift