Raphael Simon, senior systems architect at RightScale, has created “goa”, a Go-based HTTP microservice framework that allows the definition of a service API via a Domain-Specific Language (DSL) and the automated code generation of the corresponding “boilerplate” server and client code and documentation.
The Gopher Academy blog post that introduces goa states that the RightScale engineering team are migrating from a monolithic Ruby on Rails application to a series of Go(lang)-based microservice applications. A primary challenge with this migration has been the creation of well-designed service APIs, and accordingly a series of tools have been created that assist with the design, review and implementation of microservice APIs. One of the results of this work was the creation of goa, a Go-based framework that was inspired from the RightScale Ruby-based API creation framework Praxis.
InfoQ sat down with Simon, senior systems architect at RightScale and primary contributor to goa, and asked questions about the design goals, usage and future of the goa microservice framework.
InfoQ: Welcome Raphael! Could you explain to us what 'goa' is please, and also explain your motivations for creating the framework?
Simon: RightScale has been moving to a microservice style architecture for quite some time, and more recently the team began using Go to write some of the services. Overall Go has proven to be a great choice both from a development and operation standpoints. The language is easy to pick up and the resulting services perform really well.
However, adopting Go also means adding a new stack to develop and deploy - until recently RightScale was mostly a Ruby shop. The team has a lot of experience building and running REST APIs (RightScale open sourced its Praxis framework for building REST APIs in Ruby), and it made sense to leverage that experience to build a Go-based service development stack, as there didn’t seem to be an existing solution that would fit the bill.
goa is the result of this work. It is a microservice development framework that makes it possible to describe the design of an API in code. goa comes with the goagen tool that leverages the design code to generate a number of outputs including http server wrappers, scaffolding, documentation, clients and even custom outputs. The goa package also contains a number of supporting modules that both the service and generated code can take advantage of to implement the service.
InfoQ: How does 'goa' make it easier to write microservices, in comparison with simply using the standard Go libraries (or Peter Bourgon's go-kit)?
Simon: goa makes it possible to describe the intent of the API - using the goa design language one can define the resources and actions (i.e. API endpoints) that the API exposes. The description for each action includes the expected request state and possible responses. There are a number of benefits to this approach: for one it helps a lot during the design phase where different teams can easily provide feedback looking for example at the generated Swagger specification using the Swagger UI. The teams in charge of building UIs or writing user facing documentation hugely benefit from this feedback loop. The best part is that it happens before any actual implementation code is written.
Another advantage of the approach is that once the design has been finalized goa takes care of generating all the boilerplate, which includes request validation and custom data structures describing the request state. The design becomes an integral part of the implementation. This is a very valuable aspect, as maintaining the implementation in sync with the design is otherwise a very tedious process. With goa the generated request handler contexts are tailored for each action exposing accessors that return the data structures described in the design. This alleviates the need for “binding” or otherwise manually validating the format of the incoming requests.
goagen can also generate clients for the API including a command line tool, Go package and Javascript clients. Apart from the obvious benefit in the reduction of time taken to manually perform this process, this also helps maintain consistency across the various clients and keep them up-to-date with the API. In a microservice world where the number of services keeps increasing the consistency aspect is very valuable as it makes building integrations a much easier process.
InfoQ: 'Goa' appears to offer a developer quite a lot, including a runtime 'engine', logging support, the ability to customise 'middleware', and error handling. Are you attempting to provide a single solution for go-based microservices?
Simon: The goa request context is at the heart of the runtime engine. It leverages the work done by the golang package developers around passing contexts across interfaces to enable powerful features like setting deadlines and sharing state across goroutines. The context can be passed all the way to software modules in charge of making external requests for example so that the entire request state including deadlines is available at all stages. The generated code wraps the context in data structures that are action specific exposing the validated request state via fields described in the design code.
Everything else is swappable: goa follows the “battery included” model. So for example if you don’t have a strong opinion on how error handling should be done it provides a sensible default (in this case returning a response with status code 500). However, the default error handler can easily be overridden to implement any custom logic and goa comes with an alternative error handler that won’t write callstacks in responses. Another example is the middleware support: goa defines its own type of middleware so that it’s possible to leverage its rich context but it also allows using straight http middleware.
Not all pieces are easily swappable yet (logging comes to mind) but this is an artifact of the initial focus being on code generation. The intent is to provide a complete microservice development framework that is modular so that for example, developers that use functionality from go-kit or other Go packages can keep doing so while still benefiting from the design-based API development approach promoted by goa.
InfoQ: We were particularly interested to read about the code generation for clients consuming service API. The discussion of code generation in general often gets a negative response e.g. resulting code can be complex, or the generation process in opaque and not customisable. Can you explain your reasoning behind this implementation choice?
Simon: The trick here is that the functionality that the generated code implements is the boring boilerplate. The code follows the same structure you would use if you were to write it “by hand”. goagen tries very hard to follow the ‘principle of least surprise’. One could even argue that the final code ends up looking more “standard” (and is more efficient) as for example it alleviates the need to resort to reflection to avoid having to write the thousands of lines of code it otherwise takes to do the equivalent. Looking at the client Go package for example, you would be hard-pressed to guess it was generated - well OK some of the local variable names may look a bit funky and give it away :)
Another important aspect of code generation in goa is that the generated code always lives in a different package that does not need to be manually modified. This makes the distinction between user and generated code very clear. goagen always regenerates the entire package so that it is impossible to get into a situation where both types of code mix. The one explicit exception being the scaffolding code that goagen can generate as a one time bootstrapping step.
Finally, goagen only generates what is necessary - the generated code uses the goa package for all the common pieces of functionality such as logging, error handling etc. This provides a nice way to customize the behavior of the generated code without having to modify it - for example, you can implement a custom service error handler and all of the generated code will automatically call the custom logic. In a way you can think of the generated code as “plugging” itself with the goa package, which then exposes the knobs to modify its behavior.
goagen supports plugins which really aren’t anything more than a standard Go package with a public Generate function. One of the first contributions to the project has been the gorma plugin written by Brian Ketelsen which generates gorm models from types described in the API design. The possibilities that code (and other artifacts) generation open has been one of the most surprising aspects of this project. It has become clear to me that this works very well in Go.
InfoQ: What is the best way for interested InfoQ readers to get involved with or contribute to the 'goa' project?
Simon: The github repository wiki includes a roadmap document with a number of suggestions, but these are just that - suggestions. One of the most rewarding aspects of the project has been its quick adoption in the community and the numerous contributions on aspects I had not even considered. goa is still very new and there are a lot of different directions in which the project can grow. There is also an active #goa slack channel hosted at gophers.slack.com. So head over to the repo, join the slack channel and have a go at it!
There *is* one specific area the project could use some help with: it needs a logo! and the web site could also use a face lift ;)
InfoQ: Thanks for your time today Raphael. Is there anything else you would like to share with the InfoQ readers?
Simon: Thank you for having me! I encourage all the readers that need to build APIs to check out goa as I truly believe it can help building services efficiently. In a world where microservice style architecture is quickly becoming a de-facto standard, having good APIs is ever more crucial and using good tools can make a big difference. I look forward to see goa evolve as its usage grows and can’t wait to see what others do with it.
Additional information on the goa microservice framework can be found on the Gopher Academy “goa: Untangling Microservices” blog post, and also the goa Github repository.