So, like everyone, I waited all through the off season, kept up with most updates of the NFL draft Class of 2016, trade rumors, trades, some more rumors and the preseason on ESPN Fantasy application. Fantasy Sports (not the daily kinds but the traditional kind) give you bragging rights, and everyone who plays it, would agree with me that two things are crucial – Fantasy Draft day and Game day morning, when you scramble last minute to make sure your lineup is optimal using the news on starters on game day.
ESPN fantasy has been the go-to for me throughout and let me tell you, one of the best resources for me and for most people I know. ESPN Fantasy’s redesigned UI for the mobile app, web app and most importantly, the user experience changes have all been great this season. Awesome job folks! This blog post is not to undermine any of those fine aspects.
Picture this. It is Monday night. Your Fantasy team is doing great and you are winning by 0.2 points with a minute forty (1:40) on the clock with the opposition having no time-outs remaining. You clink glasses with your buddies to celebrate the win for the week of your fantasy team, suddenly content that you have a team that will take you to playoffs. Then suddenly your fortunes change; your QB takes the victory formation thrice leading to -0.3 on your score and an eventual loss!
Exactly how it went for ESPN Fantasy App. Preseason – Great, Draft process – Great, Kickoff weekend morning – Site Down! It made me wonder, is ESPN Fantasy application as brittle as bones of the particular QB that I love so much, crashing to the turf at first hit of the season??
Come on fellas! You are better than that, I mutter to myself. As I prepared myself for a last minute scramble to edit my lineup as my starting RB2 was reported “out sick”. I take to my mobile application to make amends to my game plan and line-up. I see my worst fantasy fear (second only to a first quarter injury to any of those on my starting lineup) come true. I launch the app and I see an error message pop on the screen, unable to connect. I reckon it is due to my Wi-Fi connection; I disconnect Wi-Fi on my phone and try with LTE –no change in fortunes. I wait, try, wait, try again and finally give up. I am sure you can relate to the restlessness, my Week 1 bragging rights are at stake. I cannot check my scores for all afternoon games, which was starting to bother me more. ESPN finally tweeted something to the order of “We know, hang tight”.
The technologist in me made me wonder, couldn’t one of the most respected names in the game be prepared for the kick-off weekend? I was upset at first; then I smiled, they didn’t expect as many fantasy players? Really?? I thought to myself. Cannot be true, may be its just my phone. I called a few friends and they were experiencing the same, then I noticed a hate feed growing on Twitter.
I thought this couldn’t be right; they had to be better prepared. Case in point, according to CNN, ESPN is the leader and the data published last October says they had 7.1M unique users; and just last month, they reported 1.7M teams were drafted in less than two days during their televised fantasy football marathon. So, in essence they had to be expecting this level of traffic! Since they have not yet disclosed why they went down, I can only assume they crashed due to a higher than expected load followed by functional failures of specific modules. A combination of the two seems more likely.
My reasoning is that, when they came back online, they did so incrementally, piece by piece and then the whole application. Even when they did, they were slower than normal. So it had to be a combination of the two! But again, I ask myself how can the leader in sports in general and more specifically fantasy sports miss such a thing and that too on the biggest weekend for the football in the Country?
This could have been avoided. All those wonderful folks at ESPN could have enjoyed their Tailgate parties and even the games, if they had followed some simple steps, which I call the essence of Continuous Delivery.
An application and platform redesign isn’t the easiest undertaking. It requires extreme levels of planning, collaboration and preparation. If not done right, it could mean less than optimal results when it comes to Go-time.
How to approach Continuous Delivery
- Continuous Delivery must be adopted! Have a Deployment Ready Build throughout the development cycle – This approach is crucial to any enterprise IT team that supports key customer-facing applications. The way to approach this is adopting Continuous Delivery. 3 Steps to achieving this:
- Continuous Development: Acceptance Test Driven Testing (ATDD) or Behavior Driven Development with Continuous Integration need to be adopted for fast paced application teams. This must be the norm of application development.
- Continuous Testing: Functional testing and automated functional testing alone will not cut it. Non-functional aspects of the application are extremely important in the engineering lifecycle. It is of immense value to have early performance engineering and security engineering. Incremental load tests for each build within the lifecycle help alleviate the heartburn teams experience when things move further down stream. Plus, it makes sense to stick to the old adage, a stich in time, saves nine!
- Continuous Deployment: This involves automation of several manual activities and automated environments to say the least. Have a foolproof way of deploying releases on demand and in an automated fashion. Automated deployment is key for success in high velocity and fast paced environments. Read here for more - Are Deployments Taking Away Your Happy Hour
- Once you have this process, approach and mindset in place, next thing to ensure is a version controlled build/binary files in Pre –prod and a strong, stable pre-prod. The pre prod must be production like; not necessarily in terms of capacity but everything else (Capacity is a nice to have). It must be guarded to be your pristine environments, just how you treat your prod, so, enforce all controls you would to prod. The benefits are multi-fold – This could be your dress rehearsal stage, performance engineering and validation environment, security tests can be run, roll-back binary in the event of an eventuality and much more!
- Agility – It doesn’t mean do everything and do it fast. The approach for enterprise must be small, incremental changes – avoid forklifts. Make sure these builds are deployable at any given instance. With agility, the key component is planning. Especially when you have a customer facing application that can dent your brand image. Just like the retailers prepare of the holiday season by testing for load at 1.5 to 2.5X of their peak holiday traffic, ESPN Fantasy could use similar numbers for their load and capacity planning. 1.5X-2X of their peak kick-off weekend traffic. Such planning ensures less heartburn and more end user delight.
These steps sound simple enough, and believe me, they are! They are not very different from how football teams prepare for the regular season; training camps, coaching, watching tape, preseason, final prep and a Final Playbook to kick things off, which is tweaked several times during the season, to match the situations they are in. An enterprise application engineering team needs a playbook that must be based on philosophies of DevOps and Continuous Delivery. It requires tremendous amount of discipline, preseason practice (pre–prod environments) and collaboration /communication (One team, One dream) to ensure success in production (Game time).
A smooth release must be the norm; this again is very similar to the calm teams experience under champions like Bill Belichick. Pont being, the principles must be instilled across the various teams in the Continuous Delivery value chain; it must be driven top-down, enforced and built bottom up.
I hope all of you reading this were fortunate as I was with their unchanged line-ups and this is the last time ESPN Fantasy has to ever receive tweets such as this. (Image Source www.Complex.com)
To learn more about what we do at Qentelli and how we do it, reach out to one of us / subscribe below to stay in the loop!