Golang – Iterate over a map

Hello again! In a previous discussion, we talked about using the native map container in Go. In this tutorial, we’ll be exploring more about maps in particular, by iterating through them. We’ll also have a brief overview of how this scenario changes in a multi-threaded environment as well, so keep reading for more!

Now, let’s first understand how we can normally iterate over a map. Usually, this operation may need to be done if you want to update all entries on a particular map, based on any condition that you define.

Iteration on a map container

To iterate on Go’s map container, we can directly use a for loop to pass through all the available keys in the map.

To understand better, let’s take a simple example, where we insert a bunch of entries on the map and scan across all of them.

package main
 
import (
    "fmt"
)
 
func main() {
	// Allocate memory for the map
	var myMap = make(map[int]string)
	myMap[0] = "test"
	myMap[2] = "sample"
	myMap[1] = "Golang is Fun!"
	
	// Iterate over all keys
	for key, val := range myMap {
		fmt.Printf("Key: %d, Value: %s\n", key, val)
	}
}

Sample Output

Key: 0, Value: test
Key: 2, Value: sample
Key: 1, Value: Golang is Fun!

Here, the iteration order is NOT dependent on any sorted criterion, so don’t be surprised if you get a different output here! This is because Golang’s map internally uses a hashmap in C, which uses a hash function to index a key.

Since our keys are integers here, it seems fine, but if you wish to iterate over a map[string]string, the order will be random.

So assuming we need to iterate over this map through some sorted criteria, how can we do this?

Iterating through the keys in a sorted order

To iterate through the keys in an ordered manner, we could simply sort the keys beforehand. But to do that, we need extra space to store a list of the keys from the map.

We can achieve this in the following manner:

package main
 
import (
	"fmt"
	"sort"
)
 
func main() {
	// Allocate memory for the map
	var myMap = make(map[int]string)
	myMap[0] = "test"
	myMap[2] = "sample"
	myMap[1] = "Golang is Fun!"
	
	// Let's get a slice of all keys
	keySlice := make([]int, 0)
	for key, _ := range myMap {
		keySlice = append(keySlice , key)
	}
	
	// Now sort the slice
	sort.Ints(keySlice)
	
	// Iterate over all keys in a sorted order
	for _, key := range keySlice {
		fmt.Printf("Key: %d, Value: %s\n", key, myMap[key])
	}
}

We first create a slice of all keys, and then sort it using sort.Ints() since our map is of the form map[int]string, so the keys are integers.

After that, we can iterate through the sorted slice and get the value using myMap[key].

Sample Output

Key: 0, Value: test
Key: 1, Value: Golang is Fun!
Key: 2, Value: sample

This is guaranteed to give you an iteration in a sorted manner. To play with this snippet, visit this link.

What happens in a concurrent scenario of map read/write operations?

Until now, we had assumed that only one function can access and modify the map, so iteration is simple. However, what if this iteration needs to be done in multiple functions, or more specifically, multiple goroutines?

Let’s take an example to investigate what’s happening.

Here is a program which uses multiple goroutines to access and modify a global map. All the goroutines will update all the keys and then finally iterate through them.

I have used concepts such as GoRoutines, Defer and WaitGroups. If these terms seem unfamiliar, I would suggest you to take a good look before coming back.

What could happen in this situation?

package main

import (
	"fmt"
	"strconv"
	"sync"
)

// Let's have a global map variable so that all goroutines can easily access it
var myMap map[int]string

func myGoRoutine(id int, numKeys int, wg *sync.WaitGroup) {
	defer wg.Done()

	for key, _ := range myMap {
		myMap[key] = strconv.Itoa(id)
	}

	for key, value := range myMap {
		fmt.Printf("Goroutine #%d -> Key: %d, Value: %s\n", id, key, value)
	}

}

func main() {
	// Initially set some values
	myMap = make(map[int]string)
	myMap[0] = "test"
	myMap[2] = "sample"
	myMap[1] = "Golang is Fun!"

	// Get the number of keys
	numKeys := len(myMap)

	var wg sync.WaitGroup

	for i := 0; i < 3; i++ {
		wg.Add(1)
		go myGoRoutine(i, numKeys, &wg)
	}

	// Blocking wait
	wg.Wait()

	// Iterate over all keys
	for key, value := range myMap {
		fmt.Printf("Key: %d, Value: %s\n", key, value)
	}
}

Here is what I got, although this may differ for you, since the dispatching of goroutines is random.

Goroutine #2 -> Key: 0, Value: 2
Goroutine #2 -> Key: 2, Value: 0
Goroutine #2 -> Key: 1, Value: 0
Goroutine #1 -> Key: 0, Value: 1
Goroutine #1 -> Key: 2, Value: 1
Goroutine #1 -> Key: 1, Value: 1
Goroutine #0 -> Key: 0, Value: 0
Goroutine #0 -> Key: 2, Value: 1
Goroutine #0 -> Key: 1, Value: 1
Key: 0, Value: 1
Key: 2, Value: 1
Key: 1, Value: 1

If you observe, since multiple goroutines are reading and writing potentially at the same time, this will give unexpected outputs. For instance, Goroutines 0 and 2 behave differently, as Goroutine 0 has written to the map before Goroutine 2 could read from it!

This type of situation – where multiple threads (goroutines) can access the same memory location for reading and writing, is called a data race condition, and can be extremely difficult to detect in production even if your development tests run fine.

Here, we cannot simply use a normal map. To deal with this problem, an approach would be to enforce concurrency constraints through mutual exclusion (mutex locks, for example). Another accepted practice is to use go’s sync.Map container. This provides synchronized access to our map container, so that it can be used safely in a multi-threaded scenario.

Hopefully this post has spurned you into thinking about WHEN you need to iterate through a map, and when you don’t need to. Until next time!