The Clean Architecture: In Practice
Dec 27, 2014UPDATE Dec. 30, 2014
I realized that with the interface being required by a layer on the outer layer, it makes the relationship a bit unclear. So I moved them down one layer so its clear that a layer can’t referenced anything from the outside. ie. moving TodoAdapter interface from interfaces layer to domains layer. Note that these changes doesn’t break our tests since they all live on the same package. I updated the source code below as well as the one on github.
I’ve had my fair share of headaches when dealing with spaghetti codes, codes other developers have written and even code I wrote. Most of the time I told myself I wish I could rewrite everything from scratch, and indeed for personal projects, sometimes I did just that. But that’s not always possible, nor is it always the right way.
Like most startups, we have a fair bit of technical debt and in the last few months my job has been mostly focused on rewriting components and dealing with some of these technical debts. Given a reasonable amount of time thinking about a problem, one will eventually find better solutions on how to do things. But as you continue to write all these new code, it can quickly start to feel like its the same mess all over again.
This leads me down the path of seeking for better ways to structure and write software, I needed something that’s maintainable in the long run, easily testable, decoupled and introduces low barrier for developers trying to learn how the code works. Then I came across The Clean Architecture, which isn’t new, there exists several similar approaches but I was enlightened by how each layer easily made sense.
If you haven’t, you should read it, the idea is to have clear separation of concern between components of your architecture, and respect the dependency rule which says:
Source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the an inner circle. That includes, functions, classes. variables, or any other named software entity.
This post is about how we could apply the architecture to our own code, so we’re going to build a basic Todo app with RESTful API as the interface to interact with our list. Lets call it deferr, it’ll support four operations: List, Push, Pop and Defer. First three should be familiar except for the last one which just means you can procrastinate and move an item down the list so you can deal with it later. The architecture has layers and we need not follow what its called, but the idea on how each is supposed to interact. Here’s what we’ll call our layers:
Aside from the names, I took the liberty of picking my own colors as well. :) Note that its pure coincidence that we also have 4 layers for our app as there’s no hard limit into the number of layers.
Let me explain those names a bit, Domains will be our entities/models, Managers will be our use-cases or application specific business rules, Interfaces will be our repositories, web handlers which talks to storage and finally Infrastructures which are generic things like frameworks, low-level storage access, etc.
Now that’s out of the way, its time to build our app. Naturally those layers will translate to the following files: domains.go, managers.go, interfaces.go and infrastructures.go. We’re going to use Go as our language of choice, mainly because I find that with its static-typing and interfaces it makes implementing the architecture more constraining yet refreshing since you have to think carefully, making sure each component doesn’t violate the dependency rule. If you’re coming from dynamic programming language background like Python, you’ve probably been spoiled by monkey-patching, something that’s not available in Go so instead you have to give extra thought on how you design things.
Lets start with our domains.go file:
package deferr
type TodoAdapter interface {
List() []*Todo
Push(t *Todo) error
Pop() (*Todo, error)
Defer() error
}
type Todo struct {
Slug string `json:"slug"`
Name string `json:"name"`
}
Very simple, on line 3 we have our TodoAdapter which defines the contract needed to operate with our Todos. Line 10 is a struct called Todo which will server as our model. It has two fields Slug and Name which are both string and fields also have struct tags used when serializing the struct to JSON as we’ll see later.
Next comes the our managers.go file:
package deferr
import (
"strings"
"github.com/nu7hatch/gouuid"
)
type TodoManager struct {
todoRepo TodoAdapter
}
func NewTodoManager(todoRepo TodoAdapter) *TodoManager {
return &TodoManager{todoRepo: todoRepo}
}
func (tm *TodoManager) List() []*Todo {
return tm.todoRepo.List()
}
func (tm *TodoManager) Push(t *Todo) error {
t.Slug = tm.getSlug()
return tm.todoRepo.Push(t)
}
// Removes the first item on the list
func (tm *TodoManager) Pop() (*Todo, error) {
t, err := tm.todoRepo.Pop()
if err != nil {
return nil, err
}
return t, nil
}
// Defers the first item down to the bottom
func (tm *TodoManager) Defer() error {
return tm.todoRepo.Defer()
}
func (tm *TodoManager) getSlug() string {
uuid, _ := uuid.NewV4()
return strings.Replace(uuid.String(), "-", "", -1)
}
On line 9, we have TodoManager which implements TodoInteractor interface. It has one required field called todoRepo which should be something that implements the TodoAdapter interface. The TodoAdapter interface is what describes what a repository should look like. On line 13 we have the constructor function called NewTodoManager which accepts a single parameter, the required TodoAdapter. Then followed by all four methods corresponding to four operations and on line 47 we have a utility method called getSlug which just returns a uuid4 hex that we’re using as the Todo’s slug.
What’s important to realize here is that how we’re using interfaces so we don’t violate the dependency rule. The same way we’re using TodoAdapter which is from inner layer (domains), other outer layers could use any interfaces coming from innner layers, this also has an added benefit of easier mocking when we start writing our tests later.
Then we move to the next layer interfaces.go, this one is longer so we’ll split. First part is about repositories which gives meaningful abstraction on top of low-level store access, then the second part is about web handlers for our RESTful endpoints:
package deferr
import (
"encoding/json"
"fmt"
"io/ioutil"
"github.com/julienschmidt/httprouter"
"net/http"
)
///////////////////////////////////////////////////////////////////////
// Repositories
///////////////////////////////////////////////////////////////////////
type Storage interface {
Query() []interface{}
Push(i interface{}) error
Pop() (interface{}, error)
Defer() error
}
type TodoRepo struct {
Store Storage
}
func NewTodoRepo(store Storage) *TodoRepo {
return &TodoRepo{Store: store}
}
func (tr *TodoRepo) List() []*Todo {
items := tr.Store.Query()
todos := make([]*Todo, len(items))
for i, item := range items {
todos[i] = item.(*Todo)
}
return todos
}
func (tr *TodoRepo) Push(t *Todo) error {
return tr.Store.Push(t)
}
func (tr *TodoRepo) Pop() (*Todo, error) {
t, err := tr.Store.Pop()
if err != nil {
return nil, err
}
return t.(*Todo), nil
}
func (tr *TodoRepo) Defer() error {
return tr.Store.Defer()
}
On line 17 we have our Storage interface which defines what our low-level storage API should look like. Line 24 is the TodoRepo which implements the TodoAdapter interface from domains. It has one required field which is something that implements the Storage interface, we don’t know what that object might be nor how its defined but as long as it implements the contract we have for Storage our TodoRepo will just happily use it. Line 28 is the constructor function which accepts that Storage object as the parameter. Lines 32-55 are the methods for operations which mostly just proxies down to the store.
The List method is a bit interesting though, remember that as we move to the outer layer the more generic things become. What this means is that the Store interface implementation doesn’t actually know about Todo objects anymore, its a generic storage. As such we’re doing type assertion in this method for each returned item, from interface{} to *Todo.
Next lets take a look at the web handlers from the same file:
///////////////////////////////////////////////////////////////////////
// Web handlers
///////////////////////////////////////////////////////////////////////
type TodoInteractor interface {
List() []*Todo
Push(t *Todo) error
Pop() (*Todo, error)
Defer() error
}
type WebHandler struct {
todoManager TodoInteractor
}
func NewWebHandler(todoManager TodoInteractor) *WebHandler {
return &WebHandler{todoManager: todoManager}
}
func (wh *WebHandler) List(w http.ResponseWriter,
r *http.Request,
_ httprouter.Params) {
w.Header().Set("Content-Type", "application/json")
todos := wh.todoManager.List()
b, _ := json.Marshal(todos)
fmt.Fprint(w, string(b))
}
func (wh *WebHandler) Push(w http.ResponseWriter,
r *http.Request,
_ httprouter.Params) {
w.Header().Set("Content-Type", "application/json")
var t Todo
var result interface{}
body, _ := ioutil.ReadAll(r.Body)
if err := json.Unmarshal(body, &t); err != nil {
result = map[string]string{"message": "Invalid payload."}
} else {
wh.todoManager.Push(&t)
result = t
}
b, _ := json.Marshal(result)
fmt.Fprint(w, string(b))
}
func (wh *WebHandler) Pop(w http.ResponseWriter,
r *http.Request,
_ httprouter.Params) {
w.Header().Set("Content-Type", "application/json")
var result interface{}
result, err := wh.todoManager.Pop()
if err != nil {
result = map[string]string{"message": err.Error()}
}
b, _ := json.Marshal(result)
fmt.Fprint(w, string(b))
}
func (wh *WebHandler) Defer(w http.ResponseWriter,
r *http.Request,
_ httprouter.Params) {
w.Header().Set("Content-Type", "application/json")
err := wh.todoManager.Defer()
msg := "You have successfully procastinated."
if err != nil {
msg = err.Error()
}
b, _ := json.Marshal(map[string]string{"message": msg})
fmt.Fprint(w, string(b))
}
On line 5 we have our TodoInteractor interface, this defines what actions we expect from a manager which our TodoManager manager from managers layer implement. Line 12 we have our WebHandler struct that has four methods which each supports the four operation of our app. It requires one field which is an object of type TodoInteractor, on line 16, the same familiar constructor function and all the four operation methods are quite similar, they just proxy to the manager and return JSON response to the client.
Finally we have our last layer, infrastructures.go:
package deferr
import (
"fmt"
)
type StoreHandler struct {
Items []interface{}
}
func (sh *StoreHandler) Query() []interface{} {
return sh.Items
}
func (sh *StoreHandler) Push(i interface{}) error {
sh.Items = append(sh.Items, i)
return nil
}
func (sh *StoreHandler) Pop() (interface{}, error) {
size := len(sh.Items)
if size == 0 {
return nil, fmt.Errorf("List is empty.")
}
item := sh.Items[0]
if size > 1 {
sh.Items = sh.Items[1:]
} else {
sh.Items = nil
}
return item, nil
}
func (sh *StoreHandler) Defer() error {
size := len(sh.Items)
if size <= 1 {
return nil
}
item := sh.Items[0]
sh.Items = append(sh.Items[1:], item)
return nil
}
On line 7 is our StoreHandler struct which implements the Storage interface, again from the previous lower layer interfaces. It has an Items field which just holds a slice of interfaces, notice how it doesn’t know about the Todo model, this layer is completely generic and its main role is just to store and operate on a list of items that it doesn’t know what. Much of the operations are intentionally simple so you can focus on understanding the abstraction and the relationship between each layer.
Now that we’ve seen all the layers, here comes the fun part… testing them!
First one is testing the manager, managers_test.go:
package deferr_test
import (
"testing"
. "github.com/smartystreets/goconvey/convey"
"github.com/marconi/deferr"
)
type FakeTodoRepo struct {}
func (ftr *FakeTodoRepo) List() []*deferr.Todo {
return []*deferr.Todo{}
}
func (ftr *FakeTodoRepo) Push(t *deferr.Todo) error {
return nil
}
func (ftr *FakeTodoRepo) Pop() (*deferr.Todo, error) {
return nil, nil
}
func (ftr *FakeTodoRepo) Defer() error {
return nil
}
func TestTodoManagerSpec(t *testing.T) {
manager := deferr.NewTodoManager(&FakeTodoRepo{})
Convey("testing todo manager", t, func() {
Convey("should be able to set slug on pushed item", func() {
t := &deferr.Todo{Name: "Wash clothes"}
err := manager.Push(t)
So(err, ShouldBeNil)
So(t.Slug, ShouldNotBeBlank)
})
})
}
Notice that our test is under a different package (deferr_test), this puts our tests on client perspective and if we can test our code without knowing the internals of how its actually implemented, we’ve successfully black-box tested our code. We are testing the manager and we want to isolate our tests to the manager alone. Meaning we need to stub any objects the manager relies on, remember that our constructor function NewTodoManager requires something that implements the TodoAdapter interface, we can take advantage of this by defining our fake repo called FakeTodoRepo, it has stub methods to satisfy the TodoAdapter interface so we can pass it to the NewTodoManager constructor. Then on line 35 we have our test case which just checks that the slug has been set for pushed item.
Had we not decoupled the repo from the manager, we would have run into troubles trying to stub it. Next we have interfaces_test.go:
package deferr_test
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
. "github.com/smartystreets/goconvey/convey"
"github.com/marconi/deferr"
)
type FakeTodoManager struct{}
func (ftm *FakeTodoManager) List() []*deferr.Todo {
return []*deferr.Todo{
&deferr.Todo{
Slug: "todo-123",
Name: "Wash clothes",
},
}
}
func (ftm *FakeTodoManager) Push(t *deferr.Todo) error {
t.Slug = "todo-123"
return nil
}
func (ftm *FakeTodoManager) Pop() (*deferr.Todo, error) {
return nil, nil
}
func (ftm *FakeTodoManager) Defer() error {
return nil
}
func TestWebHandlerSpec(t *testing.T) {
handler := deferr.NewWebHandler(&FakeTodoManager{})
Convey("testing web handler", t, func() {
Convey("should be able to push item", func() {
body := bytes.NewBufferString("")
w := httptest.NewRecorder()
req, _ := http.NewRequest("POST", "/todos", body)
handler.Push(w, req, nil)
So(
w.Body.String(),
ShouldEqual,
`{"message":"Invalid payload."}`,
)
body = bytes.NewBufferString(`{"name":"Wash clothes"}`)
w = httptest.NewRecorder()
req, _ = http.NewRequest("POST", "/todos", body)
handler.Push(w, req, nil)
So(
w.Body.String(),
ShouldEqual,
`{"slug":"todo-123","name":"Wash clothes"}`,
)
})
Convey("should be able to list items", func() {
body := bytes.NewBufferString(`{"name":"Wash clothes"}`)
w := httptest.NewRecorder()
req, _ := http.NewRequest("POST", "/todos", body)
handler.Push(w, req, nil)
w = httptest.NewRecorder()
req, _ = http.NewRequest("GET", "/todos", nil)
handler.List(w, req, nil)
So(
w.Body.String(),
ShouldEqual,
`[{"slug":"todo-123","name":"Wash clothes"}]`,
)
})
Convey("should be able to pop item", func() {
// TODO: exercise, implement this yourself
})
Convey("should be able to defer item", func() {
// TODO: exercise, implement this yourself
})
})
}
Just like on the managers tests, we want to isolate to test the web handlers only, so we stub the TodoInteractor interface with our FakeTodoManager which implements just that. Its List method just returns a single item list and Push method just sets some basic slug. We have two test cases here, line 42 tests posting invalid payload as well as a valid one while line 64 tests for listing of items. I also left out two test cases, lines 80 and 84 as an exercise for you. :)
And finally we test the infrastructures infrastructures_test.go:
package deferr_test
import (
"testing"
. "github.com/smartystreets/goconvey/convey"
"github.com/marconi/deferr"
)
type FakeStoreHandler struct {
deferr.StoreHandler
}
func (fsh *FakeStoreHandler) Size() int {
return len(fsh.Items)
}
func TestStoreHandlerSpec(t *testing.T) {
handler := &FakeStoreHandler{}
Convey("testing store handler", t, func() {
Convey("should be able to push item", func() {
t := &deferr.Todo{Name: "Wash clothes"}
err := handler.Push(t)
So(err, ShouldBeNil)
So(handler.Size(), ShouldEqual, 1)
})
Convey("should be able to query items", func() {
items := handler.Query()
So(items, ShouldNotBeNil)
So(len(items), ShouldEqual, 1)
})
Convey("should be able to defer item", func() {
t := &deferr.Todo{Name: "Sweep the floor"}
handler.Push(t)
So(handler.Size(), ShouldEqual, 2)
items := handler.Query()
So(
items[0].(*deferr.Todo).Name,
ShouldEqual,
"Wash clothes",
)
So(
items[1].(*deferr.Todo).Name,
ShouldEqual,
"Sweep the floor",
)
err := handler.Defer()
So(err, ShouldBeNil)
items = handler.Query()
So(
items[0].(*deferr.Todo).Name,
ShouldEqual,
"Sweep the floor",
)
So(
items[1].(*deferr.Todo).Name,
ShouldEqual,
"Wash clothes",
)
})
Convey("should be able to pop item", func() {
item, err := handler.Pop()
So(err, ShouldBeNil)
So(item, ShouldNotBeNil)
So(handler.Size(), ShouldEqual, 1)
handler.Pop()
item, err = handler.Pop()
So(err, ShouldNotBeNil)
So(item, ShouldBeNil)
})
})
}
Here we’re testing the StoreHandler, but we’re going to the number of items on several of our test cases so we embed the deferr.StoreHandler to our FakeStoreHandler instead and add a Size method to it. This method is only used on tests so it would have been useless to modify the original StoreHandler just so our test could work. Here we have four test cases, each for the supported operation. Note that the StoreHandler has no external object dependency so we didn’t stub anything, we just instantiate it directly on line 20.
Now we need to make sure our code works so lets run our tests:
$ go test -v *_test.go
=== RUN TestStoreHandlerSpec
testing store handler
should be able to push item ✔✔
should be able to query items ✔✔
should be able to defer item ✔✔✔✔✔✔
should be able to pop item ✔✔✔✔✔
15 assertions thus far
--- PASS: TestStoreHandlerSpec (0.00s)
=== RUN TestWebHandlerSpec
testing web handler
should be able to push item ✔✔
should be able to list items ✔
should be able to pop item
should be able to defer item
18 assertions thus far
--- PASS: TestWebHandlerSpec (0.00s)
=== RUN TestTodoManagerSpec
testing todo manager
should be able to set slug on pushed item ✔✔
20 assertions thus far
--- PASS: TestTodoManagerSpec (0.00s)
PASS
ok command-line-arguments 0.013s
All our tests are passing and since we’ve isolated the components that we’re only testing, our tests runs faster.
I hope by now you appreciate the beauty of architecting your code this way, it makes your code decoupled and as a result easily testable and in turn you could write fast running tests. This also lowers learning barrier when on-boarding new developers since its easy to see how things fit together.
You can download Deferr’s source here.
Cheers and Happy Holidays! ;)