Recently, I was trying to learn how the io.Reader interface works in Go and implement my own custom reader. Before implementing a custom Reader let’s understand what they are.
Readers: What are they?
Readers are interfaces or classes that are implemented in various programming languages. Usually, readers take in an input interface that can usually be represented as an array and a length. The reader returns another stream that is usually a cut to the length of the original input. Readers are implemented in various programming languages as following:
- Java: readers are implemented by the interface java.io.Reader that implements read(char[], int, int) and close().
- C++: readers are implemented by the istream where the read function is depicted as follows istream& read (char* s, streamsize n);
- Go: readers are implemented by the io.Reader interface which has Read(p []byte) error.
In the above examples, readers in various programming language do the same thing; they read from the input stream and make a copy of the same into the char / byte array that is provided to the Read function.
Implementing custom Reader in Go
In this example, we will implement a reader that will add the capitalize the characters in the resulting byte array to which we want to copy the result into.
Note: This may or may not be suitable for production but is an example. I have also not seen examples of modifying the string.
Readers in Go need to implement the following language
type Reader interface {
Read(p []byte) (n int, err error)
}
We will use 2 ways to implement the reader:
- By natively implementing the Read function.
- Using an already existing reader (strings.Reader)
Native implementation of the Read function
package main
import (
"bytes"
"fmt"
"io"
"log"
"time"
)
// CapitalizeReader is a reader that implements io.Reader
// It read the bytes from a particular position and returns the number
// of bytes that are read. If an error is thrown, all the errors are thrown
// as it is.
type CapitalizeReader struct {
b []byte
i int
}
// Read implements the function for CapitalizeReader.
// p is the []byte where the data is copied to.
// n is the number of bytes that are read and if there is an error it is
// returned along with the bytes that are read.
func (cr *CapitalizeReader) Read(p []byte) (n int, err error) {
// By default, the size of the
// if the number of bytes are less than the bytes to be read, then we assign it.
var l int = len(p)
if len(cr.b)-cr.i < len(p) {
l = len(cr.b) - cr.i
}
var t []byte = cr.b[cr.i : cr.i+l]
n = copy(t, cr.b[cr.i:])
cr.i += n
t = bytes.ToUpper(t)
n = copy(p, t)
// If the bytes read is less than the length of input byte slice, return the number of bytes
// and io.EOF; it is different from Go's byte reader implementation where it will copy everything
// and always return 0, io.EOF in the next implementation.
// Ref: https://golang.org/src/bytes/reader.go?s=1154:1204#L30
if l < len(p) {
return n, io.EOF
}
return n, nil
}
// NewCapitalizeReader takes in a string and returns a reader.
// Store string as a slice of bytes so that we can read bytes
// and uppercase it on read.
func NewCapitalizeReader(s string) *CapitalizeReader {
return &CapitalizeReader{b: []byte(s), i: 0}
}
func main() {
str := "hello world"
cr := NewCapitalizeReader(str)
fmt.Println("length of the string: ", str, " is ", len(str))
var b = make([]byte, 2)
for {
time.Sleep(200 * time.Millisecond)
n, err := cr.Read(b)
fmt.Println(n, "\t", n, "\t", string(b[:n]))
if err != nil {
if err == io.EOF {
break
}
log.Fatal(err)
}
}
bytes.NewReader()
}
Using an already existing reader (strings.Reader)
The below example explains how to use an underlying Reader. Go does this quite often. For example, we have the LimitReader (https://golang.org/pkg/io/#LimitedReader). The LimitReader accepts a Reader and assigns this reader as the underlying reader. Once a set number of bytes are reached, it will return an EOF even if the underlying reader has not reached an EOF. This example uses strings.Reader as the underlying Reader.
package main
import (
"bufio"
"bytes"
"fmt"
"io"
"log"
"strings"
"unicode"
)
// CapitalizeReader is a reader that implements io.Reader
// The underlying reader is a strings.Reader that is used to
// read the bytes from a particular position and returns the number
// of bytes that are read. All the errors are thrown
// as it is.
type CapitalizeReader struct {
// sr is the underlying reader to use the reading operations for.
sr *strings.Reader
// offset is the index from where the string is read into the given []byte slice
offset int64
}
// Read implements the function for CapitalizeReader
// p is the []byte where the data is copied to.
// n is the number of bytes that are read and if there is an error it is
// returned along with the bytes that are read.
func (cr *CapitalizeReader) Read(p []byte) (n int, err error) {
// create a new array where modifications will be made
var t = make([]byte, len(p))
n, err = cr.sr.ReadAt(t, cr.offset)
// track the offset by number of bytes that were read
cr.offset = cr.offset + int64(n)
t = bytes.ToUpper(t)
// copy to the provided array only when all the operations are done.
// io.Reader interface explicitly specifies that p should not be held by Read.
copy(p, t)
return
}
// NewCapitalizedReader takes in a string and returns a reader.
// offset is set to 0 as that is we want to read from.
func NewCapitalizeReader(s string) *CapitalizeReader {
return &CapitalizeReader{sr: strings.NewReader(s), offset: 0}
}
func main() {
str := "hello world"
cr := NewCapitalizeReader(str)
fmt.Println("length of the string: ", str, " is ", len(str))
var b = make([]byte, 2)
for {
n, err := cr.Read(b)
// Notice that we read by slice b[:n] even in the case of error
// This is because io.EOF can happen we can still have a byte that is read and
// it is easy to miss it out.
fmt.Println(n, "\t", n, "\t", string(b[:n]))
if err != nil {
if err == io.EOF {
// exit gracefully with in case of io.EOF
break
}
// For all other errors exit with a non-zero exit code.
log.Fatal(err)
}
}
}
Summary
- Readers have a similar interface in various programming languages.
- Readers in Go can be implemented natively by implementing the Read(p []byte) (int, eror) method for any struct.
- Readers in Go can also have a backing or an underlying reader that can be used to implement a new reader. This new reader can be limit or enhance the capability of the underlying reader. An example of this is the LimitReader in Go
Manu's scratchpad 