The bufio package in Golang

The bufio package provides a buffered I/O in Golang. This post provides some examples of how to use buffered I/O in Golang.

Why buffer I/O?

The IO operation in computer costs resources, especially system calls. So, we need to be more considerate when doing things like that. The write operation to a file, if done incorrectly, will come with an overhead.

This is inefficient if we write to a file one character or word at a time. The write to file works best when done in blocks or chunks.

Buffering the data needs to be written solves the problem.

Here’s how buffering works. If we send data to a buffer with a limit l then the buffer will take all those data and will not write until it reaches the length l. After which it writes the data and flushes the buffer. And the operation goes this way.

This is how buffered IO solves the problem.

Reader and Writer

The reader and writer are two different structs defined in the bufio package. These two have multiple functions suitable for buffered read and writes. Here, we are going to explore them.

1. Buffered writes using Golang bufio package

Buffered writes can be done using the writer. Here is a complete example of a buffered writer that writes to a file.

package main

import (
	"bufio"
	"fmt"
	"os"
)

func main() {
	// open file for write
	f, e := os.OpenFile("buffertest.txt", os.O_WRONLY, 0666)
	CheckError(e)

	// create a buffered writer
        // here we create a sized buffer of 4 bytes and the default is 4096 bytes
	bw := bufio.NewWriterSize(f, 4) 

	// write to buffer
	bw.Write([]byte("H"))
	bw.Write([]byte("e"))
	bw.Write([]byte("l"))
	bw.Write([]byte("l"))
	bw.Write([]byte("o"))
	bw.Write([]byte(" "))
	bw.Write([]byte("w"))
	bw.Write([]byte("o"))
	bw.Write([]byte("r"))
	bw.Write([]byte("l"))
	bw.Write([]byte("d"))

	// check how much is inside waiting to be written
	fmt.Println(bw.Buffered())            // 3

	// check available space left
	fmt.Println(bw.Available())           // 1
}

func CheckError(e error) {
	if e != nil {
		fmt.Println(e)
	}
}

When we run this program the file we written on contains:

Hello wo             // since the buffer can contain 4 bytes it will write 4 bytes at a time
                     // "hello wo" contains 8 bytes multiple of four
                     // The "rld" portion is inside the buffer that's why
                     // buffered returns 3
                     // available returns 1

Now we can write the leftover portion in the buffer and clear the buffer contents using the flush function.

The flush function will write the buffer contents to the disk.

// write to disk and clear buffer
bw.Flush()

Now, the fill will contain everything.

The write function has many variants that can be used for different writes like runes, strings, etc.

2. Buffered reads using bufio package

Buffered reads can be done in the same way the buffered writes are done. The API is very similar. Here is a complete example of a buffered reader in action.

package main

import (
	"bufio"
	"fmt"
	"os"
)

func main() {

	// open file for reading
	f, e := os.Open("buffertest.txt")
	CheckError(e)

	// create a buffered reader
	br := bufio.NewReader(f)

	// peek n bytes
	// bbuf is a byte buffer of size 10
	bbuf := make([]byte, 10)
	bbuf, e = br.Peek(6)
	CheckError(e)

	// bbuf contents
	fmt.Println(string(bbuf)) // Hello

	// num read
	nr, e := br.Read(bbuf)
	CheckError(e)

	fmt.Println("Num bytes read", nr) // 6

	// read single byte
	singleByte, e := br.ReadByte()
	CheckError(e)

	fmt.Println("Single byte is", string(singleByte))     // w

	// reset buffer
	br.Reset(f)
}

func CheckError(e error) {
	if e != nil {
		fmt.Println(e)
	}
}

Here we try reading the same file with the contents: Hello world!

Uses of Golang bufio Package

Buffered IO is useful in many different scenarios. Where many writes happen at the disk it can provide a solution to the problem by creating a custom buffered writer suitable for the needs. The buffered reads help to do the same but with reads.