LinkTrust SAN Project

I'm excited to share some details about my latest project at LinkTrust, something we've been calling the SAN Project. Hopefully this will be the first of many posts on what is going on behind the scenes at LinkTrust.

My brain is always thinking about something. About a year ago it started pondering how we store all the data in LinkTrust. At the time we had about a terabyte of data and it was growing fast. I realized we needed to start looking at a new storage solution.

The problem for us isn’t so much that we have a lot of data. Our problem is we need to access the vast amount of data quickly. Performance and reliability are key to keeping LinkTrust running well for our partners.

These considerations lead me to begin looking at a SAN. In a nutshell, a SAN is a big black box full of hard drives that work together. It provides high-performance data storage with a high degree of reliability and redundancy.

We began the project by evaluating different SAN vendors. We had to separate what the sales people told us from what their products actually do. The importance of this decision meant we had to carefully look at all our options. In the end we decided to go with Compellent. Their product offers some very unique flexibility and optimizations that I felt made it our best choice.

Once we decided on a vendor the real work began. There are a myriad of configuration choices to be made when implementing a SAN. I really wanted to make sure we got this right. It was important to me we didn’t have any single points of failure.

To ensure every piece of the system worked well, we also upgraded our switching fabric to a new set of Juniper switches. I purchased new Cat 6 network cables to ensure we weren’t bitten by an old cable starting to give out. The devil is in the details as they say. Every little piece must be considered to ensure success.

Whenever we embark on a project I want it to make things better for our partners. It needs to be worth the time effort and money we spend on it. I also don’t want there to be a lot of pain associated with upgrades or changes we make.

I work hard and plan meticulously to ensure successful project implementation. With the efforts of my great team we successfully moved our first database to the SAN last week.

To ensure success we started relatively small. The database that was moved was selected because it allowed for a seamless move with no visible impact to our partners. At the same time, it had enough load on it we could see a visible benefit when it was moved to the SAN.

Even I was surprised to see we were able to index more than 20 million records in a matter of a few seconds with no impact to user performance. This was something our previous infrastructure couldn’t do.

There is still much work to be done. We’ve grouped remaining tasks into a series of phases to allow for minimal impact to our partners and maximum success. The next phase, which will begin this evening, will require we take the Partner Center, Affiliate Center and Merchant Center offline for about an hour. This is to ensure no changes are made to configuration settings while the move is occurring.

This project really started about nine months ago. Looking back it has been a long road. We can finally see the light at the end of the tunnel. We can tangibly see the benefits that this new infrastructure will bring to the LinkTrust system. We’ve been able to make the upgrades with minimal impact to our partners. I’m thrilled to see the success we’ve had with this project thus far. I’m sure I’ll only be more excited by the final outcome.