Looks Do Matter

I had a bit of inspiration about software development recently. The circumstances are perhaps a bit unusual. I was sitting with my wife in labor and delivery at the hospital. The monitors for mom and baby are hooked up to a computer. The application which graphs the vital information for both had two features which really caught my eye. First was an animated stork which flew across the top of the screen. The second was that a nurse changed the colors to be shades of blue because we were having a boy.

Neither of these features provided additional functionality to help the doctors or nurses attending to us. What they did do was provide us with a bit of comfort. We spent most of our time watching the screen to ensure all was well. The little UI touches made what was a very sterile medical application a bit more comforting to expectant parents.

The “ah ha” moment for me was that too often I focus on providing raw functionality and information. I don’t always consider the impact of those little UI touches as I should. I’m very excited though about some recent UI development breakthroughs with LinkTrust.

Our upcoming release features some new graphing capabilities in the affiliate center. They will be the first of a number of graphs we plan to add in coming releases. We are also working on some other major UI enhancements in coming releases. While functionality is very important I’ve learned so are those little UI touches.

Now before you get too concerned, none of the UI enhancements will involve flying storks or dancing babies. Although perhaps a animated mouse being clicked might be in order.

SAN project complete!

Bryan asked me to post this for him today. He's in the hospital with his wife and their new baby boy! Congratulations to Bryan and his family!

Welcome to the year 2010. The year had a very exciting start for me. On Dec. 30 we finished migrating the last database to our new SAN. This was the largest and most complex part of the project. Thanks to excellent planning by our database administrator, Lee Crain, the move went smoothly.

The completion of this SAN migration project brings increased reporting and inbound speeds, better redundancy and virtually unlimited expansion capabilities. I am really excited by the results we are seeing. We have observed 72,000 IO requests processed in a single second. Page life expectancy is considered adequate with a value of 300. At one point recently, it was at 300,000 (higher is better). Other performance metrics indicate our servers spend much of their day waiting for more work to do.

There was an issue with our hourly statistics rollups that developed after the migration. We’ve been dealing with an intermittent lock contention issue during the rollups. We hoped the new servers and SAN would improve the situation. Unexpectedly, we found the increased performance caused the system to trip over itself because it was running so fast. The net result was a tortoise and hare problem. Sometimes slow and steady wins the race.

We caught the issue quickly and made the necessary changes to resolve it. By changing how the rollup works, we are taking full advantage of the performance our new systems give us. This process is running 18 times faster than it did previously. We are speedy and steady. Several of our engineers: Kacie, Art, Myong and Lee put in a heroic effort to make these changes happen so quickly.

I’m very excited to start the New Year with the completion of this project.

January Features

Bryan asked me to post this for him today:

I wanted to comment on a recent change that was made to the 6.0 Partner Center Login Page. We have some exciting features coming in January. Our development staff is putting the final touches on this great release.

The most exciting features are the Affiliate Fraud Scoring, Custom Affiliate Sign Up Filters, Affiliate Payment Log and Mass Creative Import.

Affiliate Fraud Scoring will allow you to see if an affiliate application matches a previously denied sign up or suspended affiliate throughout the LinkTrust system. It will also check for duplicates among existing affiliates in your account.

Custom Affiliate Sign Up Filters allow you to automatically deny applications which meet criteria you enter.

The Affiliate Payment Log will allow you to track payments made to affiliates. Affiliates will be able to see information on these payments in the Affiliate Center. This is the first of several features we are working on to help with accounting.

The Mass Creative Import will allow you to upload a tab-delimited file with creative information.

These are just a few of my favorite enhancements. More details will be posted here and on our support site in the next few weeks. Happy Holidays from the LinkTrust Technology Department!

SAN Project Update

I am pleased to post that the second phase of our SAN project was completed successfully Tuesday night. After a full day of monitoring, things are running great. Everything went well with the move and the performance gains are awesome.

Coordinating everything within a short maintenance window can be a challenge. Our careful planning paid off though, and we had the Partner Center, Affiliate Center and Merchant Center back up within 45 minutes. This will be the last phase of the SAN project we complete until after Christmas.

LinkTrust SAN Project

I'm excited to share some details about my latest project at LinkTrust, something we've been calling the SAN Project. Hopefully this will be the first of many posts on what is going on behind the scenes at LinkTrust.

My brain is always thinking about something. About a year ago it started pondering how we store all the data in LinkTrust. At the time we had about a terabyte of data and it was growing fast. I realized we needed to start looking at a new storage solution.

The problem for us isn’t so much that we have a lot of data. Our problem is we need to access the vast amount of data quickly. Performance and reliability are key to keeping LinkTrust running well for our partners.

These considerations lead me to begin looking at a SAN. In a nutshell, a SAN is a big black box full of hard drives that work together. It provides high-performance data storage with a high degree of reliability and redundancy.

We began the project by evaluating different SAN vendors. We had to separate what the sales people told us from what their products actually do. The importance of this decision meant we had to carefully look at all our options. In the end we decided to go with Compellent. Their product offers some very unique flexibility and optimizations that I felt made it our best choice.

Once we decided on a vendor the real work began. There are a myriad of configuration choices to be made when implementing a SAN. I really wanted to make sure we got this right. It was important to me we didn’t have any single points of failure.

To ensure every piece of the system worked well, we also upgraded our switching fabric to a new set of Juniper switches. I purchased new Cat 6 network cables to ensure we weren’t bitten by an old cable starting to give out. The devil is in the details as they say. Every little piece must be considered to ensure success.

Whenever we embark on a project I want it to make things better for our partners. It needs to be worth the time effort and money we spend on it. I also don’t want there to be a lot of pain associated with upgrades or changes we make.

I work hard and plan meticulously to ensure successful project implementation. With the efforts of my great team we successfully moved our first database to the SAN last week.

To ensure success we started relatively small. The database that was moved was selected because it allowed for a seamless move with no visible impact to our partners. At the same time, it had enough load on it we could see a visible benefit when it was moved to the SAN.

Even I was surprised to see we were able to index more than 20 million records in a matter of a few seconds with no impact to user performance. This was something our previous infrastructure couldn’t do.

There is still much work to be done. We’ve grouped remaining tasks into a series of phases to allow for minimal impact to our partners and maximum success. The next phase, which will begin this evening, will require we take the Partner Center, Affiliate Center and Merchant Center offline for about an hour. This is to ensure no changes are made to configuration settings while the move is occurring.

This project really started about nine months ago. Looking back it has been a long road. We can finally see the light at the end of the tunnel. We can tangibly see the benefits that this new infrastructure will bring to the LinkTrust system. We’ve been able to make the upgrades with minimal impact to our partners. I’m thrilled to see the success we’ve had with this project thus far. I’m sure I’ll only be more excited by the final outcome.

Hello from the LinkTrust Technology Department

Meet Bryan.

Bryan Call is the VP of technology at LinkTrust. He's responsible for overseeing the implementation and architecture of the LinkTrust system. When he's not working on the latest version of LinkTrust, he watches reruns of Star Trek or takes his wife ballroom dancing.

We're excited to have him become our first contributor to our blog... and bring some exciting news that will help LinkTrust perform at an even higher level. You'll definitely want to keep an eye out for his posts about the new SAN and what that means for you.

Welcome to the blog, Bryan!