highs and lows

We launched a major product earlier this week at work.

The last few weeks have been a complete rollercoaster. It seemed like every day, a potentially launch-blocking issue would arise, and then a bunch of engineers would wrestle it into submission, so we would wake up the next day still convinced that we were going to launch on time. I think there were literally seven days of mini-crises and solutions.

Two Fridays ago, we found a bug that we thought would actually prevent us from launching. Many engineer-hours later (I think Tuesday around noon), it was discovered that we could fix the problem without making a new build of a particular server, and we were on track once again.

The following Thursday, we found yet another seemingly intractable blocking issue, and the launch was cancelled. People were pretty bummed the rest of the day. Then, Friday at noon, a crazy plan was concocted to launch on time anyway. Many, many people volunteered to work weekends, early mornings (starting at 4am, I kid you not), and otherwise went above and beyond the call of duty. Still, it was probably 50-50 whether the plan would succeed.

The final call was made the day before launch, after making sure everything was proceeding as expected. The rest of the day, I was too excited to get much done, but all the people involved in the launch process must have done their jobs, because we did indeed roll out our feature according to the original schedule. I think maybe all the adversity made it even more awesome when in the end, it actually happened.

Whew! Now I'm buried in a pile of user-reported bugs, but so far none of them seem too serious, knock on wood.