Revealing the Structure Within
Developing software is usually a complex process. There are many places where complexity can creep in and cause problems. When you get down to the level needed to resolve some of the issues that you might find in a typical large intricate system, it is often difficult to switch focus and remember the big picture that these details belong to, if you were ever lucky enough to have a big picture to remember in the first place.
We design our code in layers to try and help contain complexity. We develop simple rules like references can only go in one direction in the architecture stack. For example, the presentation layer can access the business layer, but the business layer cannot access the presentation layer. These help, but they can be difficult to track. Because Visual Studio will not allow cyclic dependencies across projects, we can help to maintain this separation by setting up separate projects, but this does not help to monitor such things at the micro level within a Visual Studio project. What about dependencies between namespaces, or dependencies within a namespace?
The Dependency Structure Matrix Plugin is an add-in for .NET Reflector by Tom Carter that will allow us to track, view and analyze inter-module dependencies in a format that scales nicely for real world applications. By getting a feel for the dependencies, we can also get a better understanding of the underlying architectural stack.
Sometimes a system’s architectural stack was never formally defined. Sometimes it was, but the layers were never documented, or the original layers were not maintained. The Dependency Structure Matrix will reveal the architectural structure that is actually being followed, and provide guidance on how to improve the existing structure or re-enforce the original design.
Understanding the Goal
The DSM shows dependencies in a weighted adjacency matrix. The modules for any given layer are listed down the side and along the top. A number representing the degree of dependency is listed for each intersection if there is no value, there is no dependency. The goal is to sequence the modules so that every dependency is below the diagonal line bisecting the matrix. Any dependencies that cannot be moved below the diagonal represent a “cyclic dependency” at that level.
A cyclic dependency occurs when there are mutual dependencies across objects. For example, suppose you have two classes that each call methods in the other class. This forms a cyclic dependency. Or you may have two namespaces each with classes that reference classes in the other namespace. Some cyclic dependencies are easy to identify while others may be much more subtle.
You can also click on the “Show / Hide Cyclic Dependencies” to quickly identify cyclic dependencies, but by re-sequencing the modules to force all dependencies below the diagonal, we can reveal the architectural layers or at least see where the intended layers are not being adhered to.
Here is a sample DSM showing no cyclic dependency at one level for reflector’s source code. Note the color coded diagonal.
Here is a sample DSM showing multiple cyclic dependencies in the source code for Nhibernate. The cyclic dependencies are highlighted in yellow. Note that these all represent dependencies across namespaces.
This does not necessarily imply that the code is bad. It simply tells us that the inter dependencies may not be as clear as we would want. This suggests that the code base will be more difficult to support and that new developers beginning to work on the code base may need extra ‘ramp up’ time to understand the interrelationships.
This may also mean that this is the most natural way to represent the objects once you are familiar with domain.
Interpreting the Results
The greatest strength of the DSM is that it gives you the ability to easily navigate through the various levels in an application. You can identify the first level where there is a problem and then drill down to the class level to find its source. Once you have found the source of the problem, you can easily pop back up to the higher levels and see how what you found out fits into the bigger picture.
Another benefit of using the DSM is that it can showcase the architectural structure of an application or show where the intended structure is not consistently followed.
Here is a DSM showing a cyclic dependency from the highest level. We are able to deduce the intended architectural structure, but we see that there is a cyclic dependency. The intention is that the UI layer be dependent on the Business layer. Unfortunately there is at least one class in the Business layer that is dependent on the UI layer.
Here is the same DSM having the cyclic dependencies obscuring the architectural structure resolved. Note how the architectural layers are now prominently shown.
Here is the same DSM with the Business Layer expanded. Note that there are cyclic dependencies within the business layer. BusinessLogic is dependent on Lookup and Lookup is dependent on BusinessLogic.
Depending on your specific requirements, you may or may not be concerned about cyclic dependencies at any given level. While every cyclic dependency can be resolved, not all are worth worrying about. This is a guideline that can easily be taken to extremes and you end up with unreadable code if you overdo it; but it can also help identify code areas that are potentially problematic. Code should be reviewed periodically to ensure that the identified cyclic dependencies are still not worth worrying about and that new dependencies have not been created.
Despite our best efforts, cyclic dependencies and poorly structured objects creep into even the best designs. It may not always be immediately clear where a new method or class should go, but by carefully tracking cyclic dependencies, we can get some guidance. We may put new methods where they make sense when they are first needed. Later changes may reveal that these methods make better sense somewhere else.
The matrix itself provides a nice graphical view of the structural relationships within the application. This makes it easy to see how the individual components fit together, but it can be tedious to search through all the layers for potentially problematic code. Fortunately, the DSM also generates a report.
The report is a simple text file listing out the cyclic dependencies and potential design errors. Navigating through the matrix makes it easier to understand how the various objects interact and the nature of the dependencies. Reviewing the report gives you a good check list for your clean up exercises. You can also use it to track that new problems are not introduced over time.
Correcting the Problems
By identifying cyclic dependencies or simply dependencies that are not easily resolved, the DSM helps us identify opportunities for refactoring. These code-refactors will help to solve problems with the structure of our objects or the way they interact with other objects. By carefully following these code-refactors, we can reinforce the initial architecture or move towards defining an architectural design.
Sometimes one of the simplest code-refactors is all that is needed. Here we simply move a class in its entirety from one namespace to another. This can resolve issues where a class makes heavy use of other classes in a different namespace. This is how we resolved the cyclic dependencies between the UI and Business layers identified earlier. This may be used to resolve issues where a class was not created in the correct abstraction layer. Sometimes it is hard to know whether a class is part of the Data Layer, the Business Layer or best defined as a vertical service. Tracking cyclic dependencies can often help us identify when we have not made the right choice.
Sometime a more subtle code-refactor is needed. ‘Move class’ is simple. It is obvious that the whole class needs to be moved to a new namespace, but sometimes, part of the class needs to be moved to a new namespace while the rest of the class needs to stay where it is. Sometimes, the class really needs to be split apart over several namespaces. In such cases, we systematically move each method that needs to be moved to appropriate objects in the namespace(s) where they best fit. I often find this is needed in utility or helper type classes. These are the classes that seem to pop up over the course of an application’s life cycle that are really just a hodgepodge or useful but unrelated methods. These classes start off innocently enough. You find a method that you need to use more than once. Rather than rewrite it every time that it is needed, you define a utility class and put these various methods there. Patterns in the cyclic dependencies may reveal a better grouping for these unrelated methods or may even identify a better home for a specific method.
Sometimes the logic that needs to be moved has not been isolated to a separate method yet. Sometimes the dependent logic is embedded inside of an existing method and needs to be extracted to its own method before it can be moved. The chances are that the original methods had side effects that were causing problems in their own right and needed to be simplified. By extracting the method, you may end up with two or more methods, each of which are much simpler than the original. Each of these methods may in turn find new homes in other classes that will simplify and resolve cyclic dependencies.
Extract Super Class
Sometimes the cyclic dependency reveals an implied inheritance relationship that has not been fully developed. For example, in the Business Layer we may have a Supplier object and a SalesRep object that are interdependent. These shared dependencies may mask the fact that the two common classes should share a common base class. In this case, we would identify the common functionality, pull this functionality to a common base class and let the interdependent classes derive from this base class.
Pull Up Method
Sometimes there is an existing inheritance hierarchy, but the interdependencies identify a method that is at the wrong level of inheritance. You may be able to resolve the cyclic dependencies simply by moving the method from one level of inheritance to the next
New Design Best Practices
As you analyze your assemblies with DSM plug-in, a few new best practices emerge.
- Limit the number of objects in a namespace.
- This is more of a convenience for reviewing the details in the DSM. The DSM still works even when there are a tremendous number of classes in a namespace, but it will be easier to view when the entire matrix can be viewed on the screen at one time. This will have no impact on application performance and will not impact ease of implementation or ease of support. I suggest this best practice solely from a sense of laziness
- Namespace names should reflect the architectural layers.
- There is also not a great deal written about naming conventions for namespaces. Usually the recommendations read something like:
“The general rule for naming namespaces is to use the company name followed by the technology name and optionally the feature and design as follows:
Prefixing namespace names with a company name or other well-established brand avoids the possibility of two published namespaces having the same name. Use a stable, recognized technology name at the second level of a hierarchical name.”
I would extend this convention to have the namespace names support and reflect the architectural layers. For example:
- Report the DSM Dependencies as part of the release documentation.
- Finally, I recommend making the DSM report part of the deployment details for project releases. This provides documentation that new cyclic dependencies are not being introduced and keeps any existing cyclic dependencies on everyone’s radar.
Where to Get More Information
The paper Using Dependency Models to Manage Complex Software Architecture, by Neeraj Sangal, Ev Jordan, Vineet Sinha and Daniel Jackson, provides extensive details into various additional architectural analyses that can be done with a DSM. This is an excellent primer for getting the most out of this plug-in.
TomCarter.Developer: Dependency Structure Matrix PlugIn for .NET Reflector The home page for the plug in where you can download the plug-in and get more details of its use and installation procedure. It was developed by Tom Carter, and is based on the work quoted above by Sangal et al 2005