Referencing the internal members of an aggregate.

There is a lot of confusion around the Aggregate pattern [DDD] and especially around the question: whether or not it's OK for an external object to reference an internal member of an aggregate.

First of all, let's see what the DDD book has to say about it:

"Choose one Entity to be the root of each Aggregate, and control all access to the objects inside the boundary through the root. Allow external objects to hold references to the root only. Transient references to internal members can be passed out for use within a single operation only. Because the root controls access, it can not be blindsided by changes to the internals."

This is a little bit confusing. On the one hand, the root controls all access to the internal members and it can not be blindsided by changes to the internals, but on the other hand, transient references to internal members may be passed out. This means that an external object can mess around with the state of the internals and thus, blindside the AR...
Sounds like a paradox, or is it?

Consider the following example: suppose we have a Travel Agency web site in which users can make flights reservations. A reasonable invariant would be: total reserved sits for a flight can not exceed the total number of sits on the plain. This rule is a true invariant since breaking it will cause a total chaos on the day of the flight. Imagine 60 people claiming their sits on a plain with 50 sits only...

To enforce this invariant we will probably have the following objects in one aggregate:

Here Flight is the AR

Our mission is to make sure that the state in the DB NEVER(!) violates this invariant.
There are several techniques to achieve that.

Tell, don't ask.

The first technique is to let the AR encapsulate all the internal members and every changes to their states will be done through it. In our example, Flight (which is the AR) will encapsulate Reservations. Flight will expose methods like ReserveSits, UpdateReservation, CancelReservation and thus will be able to enforce the invariant. This technique might work, but only if all the internal members of the aggregate are fully encapsulated. Unfortunately it breaks the rule of: Transient references to internal members can be passed out for use within a single operation only

It used to be my favorite technique but it's quite a pain in the ***. What if the aggregate consists of a dipper graph of objects? Eventually you will end up with an AR that has an endless list of methods which all their purpose is to encapsulate every action on the internal members.

Brute force validation.

We need a different technique, a one that will allow external objects to hold transient references to internal members and at the same time will not blindside the AR. The one i prefer is what I call a "brute force validation" (BFV). With this technique you ask the AR to check all its invariants before each time you are about change its state in the DB. You will probably have a method in the AR called CheckInvariants or something like that.

There are a few issues to consider with BFV. First of all, you MUST not forget to call CheckInvariants before each time you are saving the AR to the DB. This means you need to find all the places in code in which you are saving the AR and to invoke this method there. Ouch...
And what if some developer will add a new place in code that saves the AR to the DB? If this developer will forget to call the CheckInvariants method - your DB will be corrupted...

Fortunately, the Repository pattern is coming to the rescue. According to this pattern, each aggregate should have its own repository (usually with the name of the AR as a prefix e.g. FlightRepository). Each AR repository should have a method that saves the AR in the DB along with all its internals and only them. According to this pattern, the AR repository should be the only place to save the AR to the DB. This sounds like a good place to call the CheckInvariants method - inside the repository itself, right before the saving action.

But there is another issue: what happens if an external object modifies one of the internals and then tries to save this internal directly to the DB? This will bypass the CheckInvariants method which is located only at the AR. Actually, if you are following the Repository pattern correctly, you don't have to worry about it - repositories should only expose methods that save ARs and not regular entities. Therefore this scenario is not possible.

One last issue to consider. Imagine the following scenario: some AR holds a reference to an internal member of another AR. In our example, let say that another AR, the class User, is holding a list of all the user's Reservations. Those Reservations are also internal members of some Flights. 

Is this possible?

What if some User object modifies a Reservation and then this User object is sent to UserRepository to be saved? According to the Repository pattern this is all perfectly legal - a User is an AR and hence should have a repository to save it.
But still, we do have a problem here, the modification to the Reservation object may violate some of the invariants of a Flight and this Flight won't even know about it. Do not worry, if you're following the Aggregate pattern correctly, this scenario is also not possible. It's true, at some point, a User object may hold a reference to some Reservation of some Flight, but this reference is Transient, meaning, the Reservation is not a member of User. Therefore, even if some User object will modify a Reservation and then this User object will be saved to the DB, the Reservation will not be saved along with it.

An entity can be a member of only one AR

Conclusion

Brute force validation allows you to expose the AR's internal members (if needed) and yet, to be confident that even if some of the invariants are violated - these violations will be discovered before saving the AR to the DB.

Aggregate [DDD] - boosting the performance.

In this post I would like to tell you about some experiment I've made - put to the test the Aggregate [DDD] pattern.

From Eric Evan's book:
It is difficult to guarantee the consistency of changes to objects in a model with complex associations. Invariants need to be maintained that apply to closely related groups of objects, not just discrete objects. Yet cautious locking schemes cause multiple users to interfere pointlessly with each other and make a system unusable. [DDD, p. 126] 

According to this, the Aggregate pattern should provide a more efficient way to enforce invariants in a multiple users environment by significantly reducing DB locks.
OK, so I've put this to the test.

I've simulated an eCommerce store with 80,000 concurrent users which try to add/edit different OrderLines of 1000 different Orders. One or more users can work simultaneously on the same Order.
Invariant: each Order has a MaximumTotal that cannot be exceeded by the sum of the Amount of all of its OrderLines. 
I've used SQL Server 2005 + NHibernate 3.1.0.

So first I tried to enforce this invariant without the Aggregate pattern. I've created an OrderLineService with 2 methods:

This method gets an orderId and an amount. It fetches the Order eagerly with its OrderLines, find the first OrderLine and tries to update its Amount with the amount passed as parameter. Before updating the amount, we must check that the invariant is not going to be violated, so we calculate the sum of all of the OrderLines of the given Order. 

But what if right after we found out that the invariant is not going to be violated and right before committing the changes - a second concurrent user has added a new OrderLine that might violate the invariant?
For example:

Order #123 has a MaxTotal of 100$. It has 2 OrderLines with 40$ each.

Two requests (r1 and r2) are arriving to the server simultaneously - r1 wants to update the first OrderLine to 50$ and r2 wants to add a new OrderLine with an amount of 20$. If both will succeed the invariant will be violated.

r1 fetches Order #123. 

r2 fetches Order #123. 

r1 checks the invariant, find it to be ok and updates the amount (but yet to commit).

r2 checks the invariant, find it to be ok and adds a new OrderLine.

r2 commits.

Now in the db we have 3 OrderLines to Order #123: 40$, 40$, 20$ - invariant is kept. 

r1 commits.

Now in the db we have 3 OrderLines to Order #123: 50$, 40$, 20$ - invariant is violated and we don't even know about it.

To prevent this we must lock the table with RepeatableRead isolation (in MSSQL 2005 this isolation level also prevents the phantom reads). This means that until that transaction is not committed - NO OTHER USER CAN INSERT A NEW ORDERLINE, NOT EVEN IF THIS NEW ORDERLINE BELONGS TO ANOTHER ORDER!

The next Method:

This method gets an orderId and an amount. It fetches the Order eagerly with its OrderLines and tries to add a new OrderLine with the amount passed as a parameter. Before adding the new OrderLine, we must check that the invariant is not going to be violated, so we calculate the sum of all of the OrderLines of the given Order. 

Again, for the same reason as stated above, we must lock the table with RepeatableRead isolation.

Here is the code that simulates 80,000 concurrent users, all are trying to add/edit different OrderLines with different amounts.

By not using the Aggregate pattern it took ~200,000 milliseconds to process 80,000 concurrent requests.

Now let's simulate a scenario in which we do use the Aggregate pattern.
First I've added two new methods to class Order: UpdateFirstOrderLine and AddOrderLine.

Next I've also add 2 new methods to OrderLineService: UpdateFirstOrderLineWithAggr and InsertNewOrderLineWithAggr:

As you can see, in these 2 methods I haven't used the RepeatableRead isolation level for none of the transactions, meaning 2 concurrent users can simultaneously add 2 different OrderLines to the same Order and potentially violate the invariant. So how can we tolerate this? 
Let's look at the pattern's definition again:

Choose one Entity to be the root of each Aggregate, and control all access to the objects inside the boundary through the root.

Any changes to Order.OrderLines list, whether if it's adding a new OrderLine or modifying an existing one, will be done through the root (Order) and will increase the root's Version (optimistic-lock). Therefore, if two concurrent users will try to add two different OrderLines to the same Order - one will succeed and the other will fail for trying to update an out-of-date instance of Order.

With this mechanism, I don't have to lock the whole OrderLines table any more - i can prevent simultaneous modifications through the root object.

Let's simulate this.
Order #123 has a MaxTotal of 100$ and Version 2. It has 2 OrderLines with 40$ each.
Two requests (r1 and r2) are arriving to the server simultaneously - r1 wants to update the first OrderLine to 50$ and r2 wants to add a new OrderLine with an amount of 20$. If both will succeed the invariant will be violated.
r1 fetches Order #123. Version is 2
r2 fetches Order #123. Version is 2
r1 checks the invariant, find it to be ok and updates the amount (but yet to commit).
r2 checks the invariant, find it to be ok and adds a new OrderLine.
r2 commits.
Now in the db we have 3 OrderLines to Order #123: 40$, 40$, 20$ - invariant is kept. Also, the Version is now 3.
r1 tries to commit but with Version 2 and fails.

For that mechanism to work properly, we need to be 100% sure that any changes to one of the Aggregate's members will increase the root's version. As Evan says:

Because the root controls access, it cannot be blindsided by changes to the internals  (see also)

Here is the code that simulates 80,000 concurrent users, all are trying to add/edit different OrderLines with different amounts:

By using the Aggregate pattern it took ~54,000 milliseconds to process 80,000 concurrent requests.

Around 4 times faster!!!

All source files can be found here