Learning Go (and serializing objects with it, too)

I had intended to make my next target the Go RPC services, simply because RPC is cool. Nothing makes me happier than pressing a button on machine A while watching the LED on machine B flash. Nothing.

As I got into the RPC code, I realized that underlying data serialization was handled by the gob module. The gob module appears to be analogous Python’s pickle approach.  Given that, it seems that the gob foundation ought to exist before I spend any amount of time on understanding the RPC mechanism.  So, that’s what this post is about. We’ll put together a module that serializes structure data. Of course, we’ll also provide reader functionality in order to close the loop.

Since we’re getting a little fancier (two files this time), the little gogogo test script won’t work here. Instead, we’ll have to do it the old fashioned way.  I promise we’ll just add a Makefile if we add a third file. That’s probably worth a post itself.

The gob Package

The documentation for the package can be found on the Go Language site. Based on those pages, it’s a rather flexible system.  Instead of terminating or triggering an error condition in many cases, Go will make an effort to follow pointers and do other creative things such as leave out elements if they’re not present in the type we’re attempting to reconstruct.  This is probably something to take note of as it could be slightly surprising if it is not a behavior you’re ready for.

The documentation on the encoding format is quite complete and it covers things such as integer encoding, structure formatting, and the handling of signed values.

Manipulating Gob Data

Our example application isn’t terribly complex.  It is composed of two files. The first file, goopy.go, contains all of the code necessary for handling the serialization.  The second file, main.go, acts as the driver and provides an interface to the user.  Here’s  quick run down of the application flow.

1. Startup and flag processing. Default to read.
2. If reading, load the gob file from disk and display it to the screen. We take care to handle missing files gracefully.
3. If writing, we encode the data in question and stream it to disk.
4. Application terminates, ensuring all files are closed.

The Stuff at the Top of the goopy.go File

This isn’t overly interesting.  We just include package dependencies and define a structure type that we’ll use for serialization.

package goopy

import "gob"
import "os"

type PersonInfo struct {
  Name string
  Age int
}

Serialization

The following method handles the serialization of data to a local file. The function accepts a pointer to a PersonInfo as its receiver and a path for the target data file.  We’ll return something implementing the os.Error interface if we run into a problem. Otherwise, the return value is nil.

func (p *PersonInfo)SerializePersonInfoToGob(to_file string) (os.Error) {
  /* Open file and check for error state */
  file_handle, err := os.Open(to_file, os.O_WRONLY|os.O_CREAT, 0600)
  if err != nil {
    return err
  }

  /* Automatically close when we finish in this function, consider
   * with open(to_string) as file_handle. */
  defer file_handle.Close()

  /* Serialize data out. */
  gob.NewEncoder(file_handle).Encode(p)
  return nil
}

Let’s step through this. It’s not overly confusing, but there are a couple of concepts covered here.

1. The first line opens the target file. Notice the similarities between the flags here and the flags available to the standard open(2) system call. This function returns two values. An *os.File, and a possible error condition.
2. We check the error condition. If it’s non-nil, we just return it.
3. Next, since the file was opened, we call ‘defer file_handle.Close().’ What’s this do? When a method exits, for any reason, anything that has been marked with the defer keyword will execute. This ensures that our file is closed.  If you’re familiar with Python, think “leaving a with statement.”
4. Next, we create an anonymous encoder and call the Encode method. We pass file_handle. Note that NewEncoder function expects something implementing the io.Writer interface. That is, anything with a Write method. Calling Encode flushes the serialized data down.

That’s it. We then return nil to signify a successful encoding.

Deserialization

Deserialization is a lot like the serialization process.  Here’s the code that does that for us.

func NewPersonInfoFromGob(from_file string) (*PersonInfo, os.Error) {
  file_handle, err := os.Open(from_file, os.O_RDONLY, 0600)
  if err != nil {
    return nil, err
  }
  defer file_handle.Close()
  var person PersonInfo

  decoder := gob.NewDecoder(file_handle)
  err = decoder.Decode(&person)
  return &person, err
}

Everything should look quite familiar. Note that we now follow the “value, error” idiom here as well. This is obvious as the returned elements are a pointer to a PersonInfo and something implementing os.Error. The other thing to notice is that we pass a pointer to our PersonInfo variable to the Decode method. This ensures pass-by-reference. Our structure will be populated by the deserialization routine.

Creating New Objects

We add one little helper function to create new PersonInfo objects.

func NewPersonInfo(name string, age int) (*PersonInfo){
  return &PersonInfo{Name: name, Age: age}
}

The main.go File

Now, we put together our driver code that handles user options.  Let’s look at this as one big listing and step through the interesting points.

package main

import "log"
import "flag"
import "./goopy"

/* File we read and write from. This is required in any case. */
var gobfile *string = flag.String(
  "gobfile", "data.gob", "A Go Pickle of a Different Flavor.")

/* Read mode? We default to this in order to be non-destructive.
 * we'll pull Gob data out and just dump it to the screen.
 */
var writing *bool = flag.Bool(
  "writing", false, "If specified, we write the command line data.")

/* Required only in writing scenario */
var age *int = flag.Int("age", -1, "Age for person record.")
var name *string = flag.String("name", "", "Name for person record.")

func main() {
  defer func() {
    if err := recover(); err != nil {
       log.Println("Fatal Error Encountered: ", err)
    }
  }()

  flag.Parse()

  if *writing {
    person_info := goopy.NewPersonInfo(*name, *age)
    person_info.SerializePersonInfoToGob(*gobfile)
    log.Println("Serialization Complete")

  } else {

    person_info,err := goopy.NewPersonInfoFromGob(*gobfile)
    if err != nil {
      panic(err)
    }

    log.Printf("Read Complete (Dump): %+v", person_info)
 }
}

Most of this should look familiar if you’ve followed my other posts to date. There are a few newer concepts (to me, as well!) here, so again, let’s walk through.

1. Required imports and package name. Notice the “./goopy” syntax used. This is because our goopy module is located in the current directory and not in a centralized library location.
2. Next up, we setup four command line flags. The file to perform IO on, whether we want to write, and then the values for our PersonInfo structure.
3. Now, look at the first couple of lines of our main function. Different, no? Here, we defer an anonymous function that calls recover(). If you look back at our deserialization routine, you’ll notice that we’ve called panic.  Panic causes the stack to unwind and the application to exit.  If, along the way, a recover call is executed, the error passed to panic is returned and control resumes at that function return.  Note that since the stack is unwinding, the only valid place (I believe) to stick a recover call is in a deferred function.  In this case, we just use it to print our error condition and exit.
4. We then parse the flags as we’ve done before.
5. If we’re writing, we call goopy.NewPersonInfo and then proceed to serialize that information.
6. Otherwise, we default to reading our data out of file and displaying it to the screen.

That’s fundamentally it in terms of our application. Notice that we’re using the log.Printf and log.Println functions here. That’s nice as it causes data printed to standard output to be prefixed with the date and time.

Compiling and Running

While this isn’t as straightforward as our previous tests were, it’s not difficult. First we’ll compile our application, link it, and display the command line options.

mcjeff@martian:~/my_go$ 6g goopy.go
mcjeff@martian:~/my_go$ 6g main.go
mcjeff@martian:~/my_go$ 6l -o gobber main.6
mcjeff@martian:~/my_go$ ./gobber --help
flag provided but not defined: -help
Usage of ./gobber:
  -writing=false: If specified, we write the command line data.
  -name="": Name for person record.
  -gobfile="data.gob": A Go Pickle of a Different Flavor.
  -age=-1: Age for person record.

First, if we run our application without a valid file, we’ll see our error handling in action.

mcjeff@martian:~/my_go$ ./gobber -gobfile=/no/such/file
2011/02/17 15:43:19 Fatal Error Encountered:  open /no/such/file: no such file or directory

Now, let’s run it again with the proper command line arguments needed to write new gob data out.

mcjeff@martian:~/my_go$ ./gobber -writing=true -name=jeff.gob -age=31 -gobfile=jeff.gob
2011/02/17 15:45:12 Serialization Complete
mcjeff@martian:~/my_go$ ls -l jeff.gob
-rw------- 1 mcjeff mcjeff 58 Feb 17 15:45 jeff.gob

Beautiful! It ran as it should, we created the file, and populated it with our command line data.  Finally, let’s run our new utility one more time in order to read the data from disk and ensure it prints the proper contents.

mcjeff@martian:~/my_go$ ./gobber -gobfile=jeff.gob
2011/02/17 15:46:51 Read Complete (Dump): &{Name:jeff.gob Age:31}

That’s all there is to it!

• About

  

  Bio: I'm a Linuxy Engineer with a Pythonic Twist!

• Categories

  Select Category books development django Go linux open source politics pycon pylons python religion Uncategorized virtualization work xen zope

• Book Links

  • Django 1.1 Testing & Debugging
  • Grok 1.0 Web Development
  • Python 3 Object Oriented Programming
  • Python Testing: Beginner's Guide
  • Python Text Processing: Beginner's Guide
  • wxPython 2.8 Cookbook

• Blog Archives

  Select Month August 2012 July 2012 April 2012 March 2012 February 2011 December 2010 September 2010 August 2010 July 2010 June 2010 May 2010 March 2010 February 2010 January 2010 December 2009 November 2009 August 2009 May 2009 February 2009 January 2009 December 2008 November 2008 October 2008 August 2008 July 2008 June 2008 May 2008 April 2008

• Twitter Feed

  • No public Twitter messages.