Fetching strategy implementation in a J2EE application using AOP

A typical J2EE application using an O/R Mapping tool is faced with the task of fetching the required data using a minimal set of SQL queries. But this is not always an easy task. By default the O/R Mapping tool loads data on demand unless otherwise instructed to. This behavior known as Lazy Loading ensures that dependents are only loaded as they are specifically requested and hence makes it possible to avoid unnecessary creation of objects. Lazy loading is useful when the dependent components are not used for a business use case we are trying to achieve and solves the issue of unnecessarily loading components.

Typically our business use case knows what data is required. Because of the lazy loading, DB performance reduces when a large number of Select queries are executed, as all the required data for a business case are not fetched at once. This might prove a bottleneck for an application which needs to support a large number of requests (a scalability issue).

Let's look at an example, a business use case that requires a Person with his Address information to be fetched. As the Address component is configured to be lazily loaded more SQL queries are run to fetch the required data i.e. first the Person and then his Address. This increases the communication between the database and the application. This could have been avoided by loading both the Person and the Address component in a single query because we knew that our business use case required both components.

If we start writing business use case specific Fetching-APIs in the DAO/Repository and lower level service layer we need to write different APIs to fetch and populate the same domain object with different set of data. This will bloat the repository Layer or the low level service Layer and will become a maintenance nightmare.

Another issue with lazy fetching is that the database connection has to be retained until all the required data is fetched else the application will throw a lazy loading exception.

Note: The above issue resolution is not solvable if we are eagerly fetching the second level cached data in our query. In case of Hibernate O/R mapping tool if we are using eager fetching for second level cached data it will retrieve that data from database instead of retrieving it from cache even if the data is already available in the second level cache. Hibernate has no solution for the above issue. This in turn indicates that we should never use eager fetching in our query for second-level cached objects.  

An O/R mapping tool which allows us to tune the queries for the objects that are configured to be cached, will read from cache if the object is already cached otherwise fetch it eagerly. This will resolve the above transaction/DB connection issue since cached data is also fetched during query execution instead of allowing it to read on demand (i.e. lazy loading)

Let us take a sample code to see the issues faced using lazy loading and a solution to overcome it. Let us consider a domain with three entities Employee, Department and Dependent.

Relationships among these entities are:

• Employee has zero or more dependents.
• Department has zero or more employees.
• Employee belongs to zero or one department.

We take three uses cases to execute:

1. Fetch employee details.
2. Fetch employee with dependent details.
3. Fetch employee with department details.

The above use cases require different data to be fetched and rendered. Using lazy loading we have following disadvantages:

• If we use lazy loading feature on employee entity for dependent and department entities then in use case 2 and 3, more SQL queries are fired to fetch the required data.
• Database connection needs to be maintained for the duration of the multiple query cycle else a lazy loading exception is thrown, resulting in inadequate use of resources.

On the other hand using Eager-fetching has the following disadvantages.

• Eager fetching of dependents and department entities for an employee will result in fetching unnecessary data.
• Queries cannot be tuned for a specific use case.

Addressing the above issues, using use case specific APIs in the Repository/DAO or lower level service layer will result in:

• Code bloat - of both Service and Repository/DAO classes.
• Maintenance nightmare - any new use case for both service and Repository/DAO Layers, new APIs need to be added.
• Code duplication - If some business logic needs to be applied on a retrieved entity on the lower level service layer. Similarly in DAO/Repository layer query response needs to be checked for data availability before returning the data...

In order to overcome the above conundrum, the Repository/DAO layer needs to be aware of the query that needs to be executed to retrieve the entity based on the use case. To do this the default fetching pattern in the Repository/DAO class is overridden by a different fetching pattern based on the specific use case as defined in the Aspect class. All fetching pattern classes implement the same interface.

The repository class uses the above fetching pattern to retrieve the query that needs to be executed as shown in the sample code below:

public Employee findEmployeeById(int employeeId) {
	List employee = hibernateTemplate.find(fetchingStrategy.queryEmployeeById(),
		new   Integer(employeeId));
  if(employee.size() == 0)
  return null;
  return (Employee)employee.get(0);
}

The employee fetching strategy in the repository class needs to be changed based on the use case required. Decision about changing fetching strategy in repository layer is kept outside the repository and service layers in an aspect class, hence adding any new business use cases just needs modification only at aspect and one more fetching strategy implementation for the repository’s use. Here we use Aspect Oriented Programming for deciding which fetching strategy needs to used on business use case basis.

So what is Aspect Oriented Programming?

Aspect Oriented Programming (AOP) enables modularized implementation of crosscutting concerns that abound in practice: logging, tracing, dynamic profiling, error handling, service-level agreement, policy enforcement, pooling, caching, concurrency control, security, transaction management, business rules, and so forth. Traditional implementation of these concerns requires you to fuse their implementation with the core concern of a module. With AOP, you can implement each of the concerns in a separate module called aspect. The result of such modular implementation is simplified design, improved understandability, improved quality, reduced time to market, and expedited response to system requirement changes.

Then reader may refer AspectJ in Action by Ramnivas Laddad for a detailed treatment of the AspectJ concepts and programming and AspectJ Development Tools for aspectj tools.

Aspect plays a very important role in fetching strategy implementation. Fetching strategy is a business level cross cutting concern and it changes based on the business use case. Aspect helps in deciding which fetching strategy needs to be used in a particular business use case. Here fetching strategy decision management is kept out side the lower level service or repository layer. As any new business use case might require a different fetching strategy and can be applied with out modifying the lower level service or repository API.

FetchingStrategyAspect.aj

/**
     Identify the getEmployeeWithDepartmentDetails flow where you need to change the fetching 
     strategy at repository level 
*/
pointcut empWithDepartmentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDepartmentDetails(int)));

/**
    When you are at the specified poincut before continuing further update the fetchingStrategy in   
    EmployeeRepositoryImpl to EmployeeWithDepartmentFetchingStrategy
*/
before(EmployeeRepositoryImpl r): empWithDepartmentDetail() && target(r) { 
r.fetchingStrategy = new EmployeeWithDepartmentFetchingStrategy(); 
}

/** 
   Identify the getEmployeeWithDependentDetails flow where you need to change the fetching 
   staratergy at repository level 
*/
pointcut empWithDependentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDependentDetails(int)));

/** 
   When you are at the specified poincut before continuing further update the fetchingStrategy in 
    EmployeeRepositoryImpl to EmployeeWithDependentFetchingStrategy 
*/
before(EmployeeRepositoryImpl r): empWithDependentDetail() && target(r) { 
r.fetchingStrategy = new EmployeeWithDependentFetchingStrategy(); 
}

Thus  decisions about the particular query that needs to be executed by the repository is kept out side the service and repository layers, such that new use cases do not require modification to the lower level Service or Repository layers. The logic of deciding which query needs to be executed is a crosscutting concern that is retained in an Aspect. Aspect decides which fetching strategy needs to be injected in the repository before the service layer executes the API on the repository based on business use case. By this we can use the same service and repository layer APIs to satisfy different business use case requirements.

Let us discuss this with an example business use case fetching both Department and Dependent details of an employee. Here we need to make changes at our business service layer by adding the use case getEmployeeWithDepartmentAndDependentsDetails(int employeeId). Implement the new Fetching strategy class EmployeeWithDepartmentAndDependentFetchingStaratergy that implements EmployeeFetchingStrategy and overrides the API queryEmployeeById that returns the optimized query that helps in fetching the required data in one shot.

The decision of injecting the above fetching strategy for the required business use case is placed in aspect as shown below.

pointcut empWithDependentAndDepartmentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDepartmentAndDependentsDetails(int)));

before(EmployeeRepositoryImpl r): empWithDependentAndDepartmentDetail() && target(r) { 
     r.fetchingStrategy = new EmployeeWithDepartmentAndDependentFetchingStaratergy(); 
}

As you can see we haven't modified our lower level service or repository layers to achieve the above new use case. We used aspect and a new FetchingStrategy implementation to achieve our new business use case.

Now let us discuss the issue with the query optimization for the objects which are configured for second level cache. In our example code let us modify department entity to be configured in the second level cache. If we try to eager fetch the department entity then for every fetch of same department the database is hit for department information even though it is available in our second level cache. If we don't fetch the department entity in our query then our Business layer (use case layer) will participate in the transaction because the department entity is not cached and it will be fetched by lazy loading.

Thus transaction demarcation is moved to the business layer from lower layer even though we know what data is needed for our business use case, but only if the O/R mapping tool doesn’t provide a mechanism to solve the above issue I.e. eager fetching of cached data.

This approach works well for all non-cached data items but for cached data items it depends on the O/R mapping tool which resolves the cached data issue.

See the attached source code for complete working example illustrating the fetching strategy. The zip file contains a working example illustrating all the scenarios discussed above. You can use any IDE or run it from command prompt using aspectj compiler to execute and test the attached source code. Before executing make sure you have edited jdbc.properties and created required tables for demo application.

You can try it using the Eclipse IDE and AJDT plugin by following these steps:

1. Unzip the downloaded source code and Import the project into eclipse.
2. Configure the database configuration in jdbc.properties file under Resources/dbscript.
3. Execute the resources\dbscript\tables.sql on the database configured above which creates the tables required for the demo application.
4. Run the Main.java as AspectJ/Java application to create default data and test the above fetching strategy implementation.

Conclusion

This article showed how a fetching strategy can optimize the data retrieval process from a back end system on use case basis in a modular way without bloating your lower level service or repository layers.

Bio: Manjunath R Naganna is working in Alcatel Lucent as Senior Software Engineer with a specialization in the design and implementation of enterprise applications using Java/J2EE. He is mostly interested in Spring framework, Domain Driven Design, Event Driven Architecture and Aspect Oriented Programming. Manjunath would like to thank Hemraj Rao Surlu for editing and formatting of the content. He would also like to thank his leads Ramana and Saurabh Sharma for reviewing the content and providing important feedback.