Angular Application Generator - an Architecture Overview

Software automation is an interesting theme in software development. My first contact with this kind of application was in 2004 when I met Middlegen: a free general-purpose database-driven code generation tool. I remember having to manage it through a CMP 2.0 layer generation up to JSP/Struts web pages construction. At that time, I wasn’t able to appreciate that some generation tools could do a lot of things in the blink of an eye.

Since then, years have passed and the theme “Software Automation” is still present in the community, dividing the opinions of specialists, developers and architects about its effectiveness. On one side, it reduces the time spent on building software. All the repetitive work can be delivered quickly while at the same time teams can focus on the development of business requirements. On the other side, it can be hard and dangerous when deciding to write a software generation tool without a defined scope.

In fact, it’s obvious that there are pros and cons to keep in mind before making decision to generate code, but what about Angular? What's the best approach to generating source code for Angular: templating or AST handling?

In this article, we’ll take Angular source code generation to the next level by diving into techniques to make this consistent and maintainable, based on a DSL mechanism.

Why Automate?

At first glance, software automation means that you can save time by making standardization of repetitive work (e.g: CRUD). But an interesting question emerges: why should we use a generic software builder?

It’s not necessary. We are often tempted to make things generically because logic, abstraction and patterns are true human concepts but the lack of experience can lead you to several bad decisions. Some developers would usually choose, for example, a generic approach when they try to foresee undefined decisions, unknown issues or even to solve future problems that don’t exist yet.

Others would argue that trying to abstract requirements and writing generic code for a few classes should be even easier than thinking in terms of a software generation tool; indeed a strong argument against software automation. But in fact, experienced developers could take advantage of software automation if they already know the scope of the application and have a deep understand of the requirements. Of course, we’re not saying this is an easy task, but we are pretty sure that is possible. For example, a team who are migrating a legacy application to a new technology might have solid knowledge and experience to determine if software automation would be a good fit.

To take advantage of a software generation tool, it’s necessary to identify the standards that can be automatized, limit the scope with the stakeholders and translate them into real product value. Basically, thinking in Agile is the key factor to build the software automation.

Why Not Just Use Templates?

To answer, we might ask the inverse question: why not source generation? Deciding between code templating and generating involves some steps that can enlighten us about the correct path to follow: templating, generation or even both.

Most of the time, tools like Angular CLI and Yeoman are helpful if we decide to simplify the source generation by following a very strict standard and pragmatic way to develop an application. In the case of Angular CLI, it can be useful when doing scaffolding. Simultaneously, the Yeoman generator delivers very interesting generators for Angular and brings a great deal of flexibility because you can write your own generator in order to fill your needs. Both Yeoman and Angular CLI enrich the software development through their ecosystems: delivering the bulk of defined templates to build up the initial software skeleton.

On the other hand, it could be hard to only use templating. Sometimes standardized rules can be translated into a couple of templates, which would be useful in many scenarios. But that wouldn’t be the case when trying to automate very different forms with many variations, layouts and fields, in which countless combinations of templates could be produced. It would bring only headache and long-term issues, because it reduces the maintainability and incurs technical debt. It’s reasonable to think that software development should rely on good principles, like KISS, YAGNI, patterns and etc.

Instead of choosing a biased paradigm according the developer’s preference, it would be great if a mix between templating and source code generation could be done.

Understanding the Basic Angular Architecture

First of all, we must understand how the architecture works and extract the concepts that can be grouped in two key parts: code to be templated and code to be generated.

The Angular is a component-based architecture in which the basic structure is HTML templates and TypeScript/JavaScript Components. The HTML templates are designed with language markups and have properties and events which are handled by Components. These Components are managed by metadata which tells Angular how to process them. All the logic services and Components are boxed with modules.

Figure 1. Angular Architecture

When deciding to abstract the system and generate code, it’s necessary to take some important decisions of what would be most important in the Angular architecture. That’s the moment to enumerate three keys factors.

First, template are HTML structured data, which can be handled as HTML entities through parsing, manipulation and rendering or even as files based on placeholders for templating. Components are dealt in the same way: parsing the abstract tree or handling JavaScript template files.

Second, some questions arise regarding the TypeScript code: why couldn’t we navigate in the TypeScript Abstract Tree in order to produce source code? We could, but it would lead to the consumption of additional memory and more processing because of the constant compilation from TypeScript into JavaScript code. Another big issue is the abstract tree handling. It’s possible to navigate the TypeScript tree, but our project timeframe required a strong library/API to navigate and build JavaScript Tree in an easier way.

Third, Metadata, directives and variable injectors are items likely to be more detailed. It could be painful to template dozens of them and hard to maintain pieces of logic code only through the templates. 

Figure 2. Angular Dependency Injection within a webpack bundle

Shown in Figure 2 is a typical Angular 5 application. The TypeScript code is compiled and then the generated JavaScript code is bundled within a webpack file. Angular provides a set of functions in order to translate the TypeScript metadata into meaningful JavaScript code:

• The __decorate() function is responsible for encapsulating an entire Angular component;
• The function Component() handles the HTML template and CSS. It works as a placeholder for root/other modules, which is known as a ‘selector’ in the architecture;
• The function __metadata(“design:paramtypes”, …) takes some dependencies and injects them into the respective constructor's parameters.

The three above points are quite a bit important to avoid some troublesome when generating source code by handling JavaScript abstract trees. This can be resumed in the following approach:

The more there are variables, parameters and logic control, better is to handle by the tree. Otherwise, go through templates.

Defining a Platform for Source Code Generation

In order to make a solid architecture for Angular Template and Component source code generation, it’s reasonable to choose tools that are supported by the community and are in constant evolution. In order to produce code from Angular components, it’s necessary to manage the HTML, JavaScript tree and template handling. Therefore we have chosen a couple of libraries to cope with our code production.

Angular Template

In order to handle HTML source code manipulation, it’s good to use a library that can navigate in the DOM tree and produce clean, safe and functional code. We have then selected Cheerio library. Cheerio is a JQuery based tool for HTML manipulation. It’s a long term project with a helpful community of developers and contributors.

Figure 3. Sample code Cheerio manipulation.

Angular Component

Manipulation of the Abstract JavaScript Tree is at the core of the Angular architecture. To implement this functionality without too many dependencies, avoid increasing the complexity and maintaining the robustness of the code, we have chosen Recast together with AST-Types.

Recast is a high-level API for reading and writing JavaScript code, while AST-Types is a low-level API for parsing and building the JavaScript abstract tree.

Figure 4. Recast parse and print high-level usage.

In Figure 4 we can see that it is quite simple to build a tree from code and vice-versa; AST-Types can read a specific node of a JavaScript tree. Recast is helpful to write/read the entire application and AST-Types to manipulate small portion of code.

Figure 5. AST-Types traversing the “return Statement” of a function using the visitor pattern

Template Handling

For template handling, we have chosen Yeoman in light of its simplicity. This tool automates the build process and their dependencies with a focus for web applications. This tool facilitates the integration of both static and generated parts of the project and we can scale the project build process without increasing the complexity.

Selecting an Angular Starter Kit to Take Advantage of Templating

We have selected the famous Angular Webpack Starter as our boilerplate for templating. The people behind this project are doing a great job and Angular Webpack Starter has an initial setup that combines the best libraries that work together. The base app template of our generator is built with this starter kit and will make our life easier, giving us more time to spend on issues with more complexity.

Writing the DSL Code with Best Practices

Writing the initial app generator code is not so easy. A Minimum Value Product (MVP) in our initial scenario brought a couple of JavaScript template classes which were orchestrated by the Yeoman Template. These JavaScript templates classes were dealt with by domain classes, which implemented some visitor functions in order to find a reference in the abstract tree and insert the piece of code.

  Figure 6. Template of a Component handled with Recast and ast-types

In Figure 6, for example, the Constructor Domain Class should search the default constructor of the template by a specific visitor and then insert the functionalities (variable declaration, instantiation, references, etc). In the next example, there is a visitor function.

Figure 7. Ast-type visitor for constructor declaration

We could imagine that a large application with many templates and components would result in a performance drawback. Those problems would degrade the Node JS Virtual Machine, because the navigation in the abstract tree has a strong cost of processing and memory utilization. A natural evolution would replace visitor functions with a more elegant and fashioned domain that we called Angular Tree Domain (ATD).

This ATD is basically an architectural concept to isolate complexity and make transparent functionalities of Angular components (both HTML template and JS Component) fluent to the generator, as described in the following figure.

Figure 5. Angular Source Code Architecture

Figure 5 shows  the general architecture of an Angular application generator.

Angular Application Generator

This is the entry point of the application and it is responsible for reading Metadata and transforming them into technical data supplied to the ATD. We will not detail this part of the application, but only give a small overview. The components are:

• Metadata XML/JSON Input - File that describe in high level information the modules of the systems;
• Application Rules Transformer -  Responsible to read XML metadata with meaningful application rules and transform them in a in-memory database of Angular modules;
• Module Data - In-memory database with modules. These modules are system module objects that describe their form fields with components and dependencies;

Angular Tree Domain (ATD)

This is the most important component of the architecture. The ATD is the system domain and contains the core DSL builders for a Angular application construction. These DSLs are processed and orchestrated by the by the Angular Module Orchestrator.

Angular Module Orchestrator

This module is a simple Yeoman generator and it is responsible to connect different components and orchestrate their executions. There are a set of ordered Processors that are executed in a Chain of Responsibility processor implementation. These processors are tasks that take care of each part of application, such as Angular components and templates, model, SASS files and menu system update. We won’t go into detail on this module, but we will introduce the two most important components used by the processors:

• Angular Template - A fluent API of objects responsible to generate HTML source code;
• Angular Component - A fluent API of objects responsible to generate JavaScript source code.

The fluent Angular Component API is divided into 3 essential parts, which will be detailed as follows.

DSL Builders

The DSL Builders are a set of high-level builders (based on DSL patterns) that handle each important part of a JavaScript component’s construction. At this angle, most of Angular developers would be encouraged to develop using this high-level implementation, which would be helpful in the creation of new components to the generator.

The granularity of the builders increases according its responsibility in the tree parsing. For example, an Angular Component Builder has a root granularity because it is the entry point and through a composition of builders the whole code is generated. The Class Builder has a lower granularity and doesn’t know anything about the Angular Component Builder, which is at the top of tree.

Figure 6. Angular Component Builder

As shown in Figure 6, the Angular Component constructor calls each fluent method which builds each part of the component. The Angular Modules Orchestrator processor sends a command to the entry point of the Angular Component Builder.  As shown above in the constructor of the AngularComponentBuilder, a sequence of builder calls is performed in that order. Each builder takes responsibility for itself and keeps a high-cohesion class with low-coupling.

As a result, each builder has their own piece of an abstract tree and the main AST model can be built any time through the root builder. This final AST is known as Semantic Model. This notation means that the AST model result is built progressively.

Later we will talk a bit more of it to have a better comprehension of this concept, but now we will take deep into the Angular Component building, stepping into the ClassBuilder.

 Figure 7. Class Builder

In Figure 7, the ClassBuilder has a reference to the class which is responsible to call AST functions in order to create and parse the abstract tree. Through the Bridge and Composite pattern, all builders in the architecture delegate the low-level implementation to the Syntax Tree classes and keep the API fluent.

It is clearly evident when we take a look at the method addRequire(): it creates a reference of the RequireSyntaxTree class and adds it into the ClassSyntaxTree reference. Later, it returns the self reference of the builder, keeping it fluent. At any time, as previously mentioned, the AST model can be recovered because all the builders hold a reference to its own SyntaxTree class.

Figure 8. Syntax Tree abstract class

All of the SyntaxTree classes are responsible for handling the low-level code of tree parsing. As the project evolved through constant refactoring, most of tree nodes parsing were delegated to common classes (Figure 10). It helped these classes to keep a clean code with strong and meaningful functions. The following piece of code shows such case.

Figure 9. A getAst() implementation that mounts a REST path params

Figure 10. RequireSyntax class - getAst() returns the tree representation

As stated in Figure 10, it is very useful to make a composition of functionalities which help in the parsing and generating of the tree. In this example, a class “utilsCommon” has small features to create attributes, variables and arrays.

Taking into the class, we can describe them as low-level implementation of small and shared functionalities from the AST-Types. These are presented in Figure 11.

Figure 11. Two functions that are shared both by themselves and the SyntaxTree classes

By splitting the Fluent API among different layers with low-coupling, it helped us to make countless unit tests to guarantee the consistency of the entire ATD. Of course there are unit tests for all Builder, SyntaxTree and Common/util classes. The libraries and tools like mocha, expect.js and assert make together a solid combination to any kind of JavaScript application. To finish our article, we would like to present in Figure 12 a simple unit test scenario that tests the “utilsCommon” functions.

Figure 12. A simple test case of the function “createVariableRequire”

So Why Automate?

In the final words we should pick up the initial question presented in this article: Why automate?

Indeed, it is not an easy decision to make. For many developers, the topic of “Code Generation“ seems to be exciting, but for managers and CTOs, we don’t think so. There are two thoughts we must always bear in mind: what are the real advantages of this approach and how could it be aggregated to the final product value? We must always think and debate what are the pros and cons when deciding generating source code, but one point has got very clear for us: the maturity and expertise of the team make the difference.

About the Authors

Jonatas Wingeter Rodrigues is Senior Software Consultant at IS Tecnologia, a small IT consulting firm. As Consultant at the customer, Jonatas works for a large store department in the south of Brazil. He started in the programming world when he was teenager. Since the early 2002’s, he has spent most his time developing software, defining architecture, team mentoring and leading small teams in both national and international projects. In free time he likes to be in the nature with his family and learn new languages, like German and French. He can be found on Linkedin

Luciano Augusto Yamane is Senior Software Engineer at IS Tecnologia. As Consultant at the customer, work together with his peer Jonatas in different technologies, such as Android, JavaEE, Angular and NodeJS. He has at least 10 years of expertise in software development. In free time he likes to play Tennis. He can be found on Linkedin