Lessons learned using Reactor

After being a full-fledge supporter of Reactor for about the past 10 months, I have to admit that while it is without a doubt a *wonderful* tool, like anything it does have its limitations.

As many know, we rolled InstantSpot out as a MachII /Reactor project, and have been using Reactor exclusively for managing our objects and database abstraction. We built it so that all data is accessed and mutated through the reactor objects. From the very beginning of the rollout, we knew that we had some performance issues that we need to deal with, as page requests on the Spot sites themselves were taking a bit longer than we would have liked (OK, a *lot* longer)

As we started getting a little load on the system we saw some scary things. In heavy traffic periods the server just choked with the processor staying pegged out. Even just a single page request was sending the processor to 100% for several seconds. Immediately we knew we need to change some things, and finally did so over the past 3 days. Here is what we learned and what our resolution was:

First, let me be clear… Reactor was NOT the problem. However, the way that we used it was.

To put it bluntly object instantiation in Reactor is pretty expensive. Especially as the complex relationships get bigger and bigger. We found that out server was having to do *far* too much work for what we really needed out of it. For instance, when we load up a site record, Reactor does a wonderful job of transversing through the entire object hierarchy and having whatever properties and iterators ready for use. As we spec’ed out the project we thought we had accounted for all this work pretty well by using some fairly clever caching techniques. However, we found that we hadn’t accounted for it well enough!

While it is really cool to be able to use the getters and setters with such ease, we came to the realization that when it comes to simply displaying output of a site, there is really not any need for all that overhead. We decided to step back and re-evaluate.

What we ended up with was a series of complex structures of data (yes structures!) that are lazily loaded rather than all loaded being up front. Additionally instead of instantiating Reactor objects and using them, we instead chose to use the Reactor query objects to return whatever recordsets we needed.

For the control panel functionality, we continue to use Reactor objects, as it makes adding/modifying/deleting data an absolutely pleasurable task, and we are no where near crisis mode on the performance in those areas.

So, the outcome?

The difference has been incredible! Now rather than a page request pegging the processor for a couple of seconds, we see it spike way under 100% for a split second. Hopefully this will be of help some of you that are considering using Reactor for your projects.

From our experience to date, I think it is still a great choice for database abstraction, just be very mindful of how you use it.