Golang Parser Package

Hello , Gophers ! What a beautiful day to code something hardcore. So let’s get this started. Today we’ll learn to parse a Go file into ast format using the parser package.

NeoVim

I have been testing out text editors, and one thing I absolutely hate having to do is switch from terminal to editor and then back to the terminal to execute, then back to the editor to edit. It has bugged me for quite some time, and though I already know a Linux command called nano, it doesn’t offer proper syntax highlighting. Thankfully, I found NeoVim which is Open Source and which I shall be using for my demonstration today. Just apt install it if you want to.

File to be Parsed

Let’s move on to create the file to be parsed. So name it gopher.go and vim gopher.go it to edit:

//parser package
package main

import "fmt"

type Gopher struct {
        Gopher string `json:"gopher"`
}

func main() {
        const gopher = "GOPHER"
        gogopher := GOPHER()
        gogopher.Gopher = gopher
        fmt.Println(gogopher)
}

func GOPHER() (gopher *Gopher){
        gopher = &Gopher{Gopher : "gopher"}
        return
}

As you can figure out, it’s a function that does not do much. Try to figure out what it’s doing.

• The constant “GOPHER” string is fed into the gogopher, which is an instance of Gopher()
• The ampersand (&) character is used to access the memory address of Gopher struct, and encodes the string into json format. For more on JSON encode-decode in Go, check here.

Remember : In Golang, fields and variables that start with an Uppercase letter are “Exported”, and are visible to other packages. Fields that start with a lowercase letter are “unexported”, and are only visible inside their own package.

Golang parser file

Next, we’ll create the file parser.go that can be used to parse our gopher.go file. This will require the use of the parser package in Go.

The most commonly used method in this package is ParseFile, which returns an AST. An AST is short for Abstract Syntax Tree. To put it simply, you know how every piece of code has tokens? Well, AST creates a tree like structure of dependencies of these tokens. So, say a method depends on a struct which is called from main, then the main function will be a parent node, and the struct is its child, and the method will be child of that struct…the variables will be children of that method and so on.

To create the parser file, let’s create and edit another file with sudo vim, and import the required packages:

package main

import (
        "go/ast"
        "go/parser"
        "go/token"
        "log"
)

Next, we’ll create our main function:

func main() {
        fset := token.NewFileSet()
        f,err := parser.ParseFile(fset,"gopher.go",nil,0)
        if err != nil {
                log.Fatal(err)
        }
        ast.Print(fset,f)
}

The syntax for ParseFile is:

func ParseFile(fset *token.FileSet, filename string, src interface{}, mode Mode) (f *ast.File, err error)

• Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates). A FileSet represents a set of source files. Methods of file sets are synchronized; multiple goroutines may invoke them concurrently.
• parser.ParseFile cannot have an empty fset. Also, here we use Mode = 0,i.e, no flags are active. Otherwise, we have the option to parse only the import statements, or only the comments.

We are finally ready to run our parser, so let’s do that.

go run parser.go | less

less is a unix command that can let you read the parsed output in the terminal page by page,and not overrun the page with text. I am not sure if it works with Windows. You can view downwards with your keyboard down key. Your output should be similar to this:

References

• The official documentation page is such a big help.
• I would also suggest registering on the official Golang forum.