If you haven’t checked out the previous part of this tutorial, go to the first section: System programming in Go – 1. We discussed file system operations in the last segment, so we’ll directly jump into i/o operations in this section.
Originally the domain of C and Assembly, it has slightly shifted to scripting languages in the past decade. Also, Go (v1.5) implemented a memory allocator written completely in Go and replacing the one previously written in native C, with comparable performance. This showed that garbage collection and performance is not a zero-sum game.
Nowadays, systems are more distributed and application software is packaged in containers using system software like Kubernetes to improve output performance. They achieve this by scaling (using more systems) or optimising resource allocation.
Learning the concept of system programming with regards to using the resource of the machine efficiently—from memory usage to filesystem access—will be useful when building any type of application.
File input and output
File input and output includes everything that has to do with reading the data of a file and writing the desired data to a file.
1. Reading binary files
There is no difference between reading and writing binary and plain text files in Go. So, when processing a file, Go makes no assumptions about its format. However, Go offers a package named binary that allows you to make translations between different encodings such as little-endian and big-endian.
Let’s create a new file readBinary.go, and convert terminal input to binary.:
func main() {
if len(os.Args) != 2 {
fmt.Println("Please provide an integer")
os.Exit(1)
}
aNumber, _ := strconv.ParseInt(os.Args[1], 10, 64)
buf := new(bytes.Buffer)
err := binary.Write(buf, binary.LittleEndian, aNumber)
if err != nil {
fmt.Println("Little Endian:", err)
}
fmt.Printf("%d is %x in Little Endian\n", aNumber, buf)
buf.Reset()
err = binary.Write(buf, binary.BigEndian, aNumber)
if err != nil {
fmt.Println("Big Endian:", err)
}
fmt.Printf("And %x in Big Endian\n", buf)
}
So, first we check the length of the argument to be not empty (value = 2)
Then we parse the integer argument input using strconv.ParseInt
Then we create a new buffer to put our binaries into
Then, we use the binary package to convert it to two encodings : BigEndian and LittleEndian.
If we go run readBinary.go 12, we get:
2. Writing text to files directly
The use of the fmt.Fprintf() function allows you to write formatted text to
files in a way that is similar to the way the fmt.Printf() function works. The fmt.Fprintf() can write to any io.Writer interface and that our files will satisfy the io.Writer interface.
So let’s make a function in fmtF.go:
package main
import (
"fmt"
"os"
)
func main() {
if len(os.Args) != 2 {
fmt.Println("Please provide a filename")
os.Exit(1)
}
filename := os.Args[1]
destination, err := os.Create(filename)
if err != nil {
fmt.Println("os.Create:", err)
os.Exit(1)
}
defer destination.Close()
fmt.Fprintf(destination, "[test]:")
fmt.Fprintf(destination, "This can be used to store data output from any function to %s\n", filename)
}
The output for go run fmtF.go test.txt is a new file:
3. Making a better copy
The ioutil.ReadFile() function reads the entire file, which might not
be efficient when you want to copy huge files. Similarly, the
ioutil.WriteFile() function writes all the given data to a file that is identified
by its first argument.
In this part, we’ll create a buffered copy that is more efficient than readFile + writeFile.
Create a new file called betterCopy.go, and first we import the packages:
package main
import (
"fmt"
"io"
"os"
"path/filepath"
"strconv"
)
var BUFFERSIZE int64
Then we can start creating our copy function:
func Copy(src, dst string, BUFFERSIZE int64) error {
sourceFileStat, err := os.Stat(src)
if err != nil {
return err
}
if !sourceFileStat.Mode().IsRegular() {
return fmt.Errorf("%s is not a regular file.", src)
}
source, err := os.Open(src)
if err != nil {
return err
}
defer source.Close()
We use the os.Stat function to check the mode, and then open the file using os.Open() if it’s a regular file. Next, we’ll read the file in segments:
_, err = os.Stat(dst)
if err == nil {
return fmt.Errorf("File %s already exists.", dst)
}
destination, err := os.Create(dst)
if err != nil {
return err
}
defer destination.Close()
if err != nil {
panic(err)
}
buf := make([]byte, BUFFERSIZE)
for {
n, err := source.Read(buf)
if err != nil && err != io.EOF {
return err
}
if n == 0 {
break
}
if _, err := destination.Write(buf[:n]); err != nil {
return err
}
}
return err
}
We create a destination file and write to the destination. We just keep calling source, Read() until we reach the end of the input file. Each time we read something, we call destination. Write() to save it to the output file. The buf[:n] notation allows us to read the first n characters from the buf slice.
And finally we’ll work on parsing the terminal input argument to get the source file:
func main() {
if len(os.Args) != 4 {
fmt.Printf("usage: %s source destination BUFFERSIZE\n",
filepath.Base(os.Args[0]))
os.Exit(1)
}
source := os.Args[1]
destination := os.Args[2]
BUFFERSIZE, _ = strconv.ParseInt(os.Args[3], 10, 64)
fmt.Printf("Copying %s to %s\n", source, destination)
err := Copy(source, destination, BUFFERSIZE)
if err != nil {
fmt.Printf("File copying failed: %q\n", err)
}
}
If we run this with go run file1.txt file2.txt, we’ll see a new file getting created. If there is a problem with the copy operation, you will get a descriptive error message.
Cool. That’s how file operations work. In the next part (part-3), we’ll look into how to manipulate processes and signals. So adios! Until next time.
References
- Hands-On System Programming with Go: Build modern and concurrent applications for Unix and Linux systems using Golang, by Alex Guerrieri
- Go Systems Programming: Master Linux and Unix system level programming with Go,by Mihalis Tsoukalos
- https://pkg.go.dev/os