TicketFire is a digitization platform that brokers and consumers can use to digitize hard copy tickets and turn them into usable electronic and mobile tickets. TicketFire is also an online ticket marketplace supported by many inventory partners, which allows users to buy and sell tickets to live entertainment events. Founded in 2014, TicketFire digitizes over 100,000 tickets each month.
TicketFire’s core business since its launch has been around the digitization of hard copy tickets. But as that business has become more commoditized, TicketFire identified strategic opportunities to fuel their continued growth. One opportunity the team identified was assisting venues and teams to combat the no-show problem by allowing the strategic oversale and resale of tickets that are not going to appear at an event.
To capture these strategic opportunities, Tony Santucci, Director of Technology, knew that the development process would need to move faster. When Tony joined the team, the engineering team was operating entirely off of CloudWatch Alarms from a monitoring and alerting perspective. According to Santucci, this approach “tended to be nebulous at best and confusing at worst. There were random system outages that would happen that we would never get an alert for.”
As a result of not having an adequate monitoring and alerting solution in place “people were literally afraid to deploy code...we were doing maybe one production deployment a month,” recalls Santucci. He knew that he needed a suitable alerting tool that would give his team confidence to deploy code faster if TicketFire was going to meet its strategic objectives.
As Santucci began to investigate monitoring and alerting solutions, he had some basic requirements:
“Now I’m doing multiple production deployments per day just because I know that if something does go wrong, I’m gonna know it sooner rather than later.”
During Santucci’s research, he stumbled across Blue Matador based on the recommendation of a previous colleague. He quickly identified that Blue Matador greatly surpassed all of his requirements. “It’s kind of been a lifesaver for me,” says Santucci.
As a result of automated alerts and notifications that Blue Matador provides, the TicketFire team is able to move much faster and deploy much more frequently. According to Santucci, “Our speed to market has definitely improved because of Blue Matador. Like I said before, we were doing maybe one production deployment a month. Now there’s some days where we do eight, nine a day because if something goes wrong, I’m going to know.”
Because of Blue Matador’s limited configurability, TicketFire has been able to achieve this drastic increase in speed to market without a significant time investment like with traditional monitoring tools. For example, when TicketFire spins up new AWS resources, they don’t have worry about setting up monitoring and alerting for those new resources. According to Santucci, “Because Blue Matador integrates seamlessly with my platform, I don’t have to worry about configuring an entirely new set of either server credentials or AWS CloudWatch monitoring.”
“I just create the new resources in AWS and poof, it’s being monitored. I don’t have to worry about it.”
Blue Matador has also helped TicketFire’s speed to market by being able to identify potential critical issues earlier. By leveraging Blue Matador’s Projects feature, TicketFire has separated out their dev resources from the production resources from a visibility perspective. “I absolutely love the Projects feature because it allows me to track everything with respect to my critical, non-critical, and pseudo-critical environments. So I can still see everything that’s going on, but I don’t get that fatigue and alarm desensitization. So I only have active alarms occurring on production. So when I get one, I know it needs to be addressed.”
By sending critical alerts related to production via text message and sending other lower priority notifications to Slack, TicketFire is able to quickly address critical issues but also proactively address potential issues before they hit production. By reviewing the Slack notifications, “we can identify issues on that level before it ever gets to production,” says Santucci.
The TicketFire team has the Blue Matador dashboard up on a TV in their team room. At a glance, the entire team can get a sense of the current health of the system. Says Santucci, “The dashboard is in front of everybody, and everybody can see it. I think one time we were up to 14 days without a production alert and that’s a pretty damn good feeling.”
“The at a glance power of it not only helps the IT team, but it gives the business as a whole a good bit of confidence in the system.”
One of the biggest reasons the business has been able to gain this confidence is due to the speed at which Santucci and team are now able to address potential issues. Explains Santucci, “With Blue Matador, I’m able to receive that notification, process it, address it, and fix it before people even know there’s something going on. Not only does it preserve face in the eyes of our consumers, but it prevents people from walking up to my desk saying, ‘Hey, what’s wrong?’”
The TicketFire team is now able to pay very close attention to issues that may be affecting customers. According to Santucci, “A lot of the anomalies that pop up are just kind of neat to look at, kind of drill in and figure out what the hell’s going on. But the ones that I really care about are the ones that are telling me what the end user’s seeing. So when I see my average latency go above 10 seconds, there’s a problem.” The TicketFire team is now able to rapidly address live issues and get ahead of other critical issues. For instance, Blue Matador notified the TicketFire team weeks in advance that their SSL certificates were going to expire. “I was able to address the SSL certs expiring by planning it out in the next week and not having to worry about it,” says Santucci.
With timely, relevant notifications and alerts like these, Blue Matador is enabling TicketFire to continue to increase their speed to market and capture the opportunities in front of them.
