Luke Freiler

Rahul Abhyankar [00:00] Luke, when people start companies, they are consumed by a problem that they desperately want to solve with some unique technology that they've invented. There's this compelling drive and desire to make a difference. So how did that happen with you and the space of product testing?

Luke Freiler [00:20] When I initially started my career, I actually started out in hardware. I was in hardware engineering at Samsung. I was having a good time. I'll admit it was a bit slow for me. Hardware has a long turnaround. And at the time the internet was really coming into its own and becoming mainstream. I took an early interest in that. As a result, I ended up running a software team within a couple of years at Ericsson, and a product manager came to me with a challenge. He wanted me to run a beta test for the product that we were coming out with. My first impression was, why me? And his response was quite simply, because we don't have anyone else and you do things online and your team does things online, so it makes sense. Immediately I started looking for anywhere in the company where we had a solution to this. The short version is I didn't find one.

At the same time, I had fallen in love with sort of the early era of what was at the time referred to generally as usability. But ultimately it was sort of a charge against the hassle factor of technology. I was very frustrated by how technology was to use for most people. Despite loving technology and being close to it my entire life, I always had relatively low patience for it. I just kind of wanted it to work.

So in short, these two things coalesce—this love of the idea that technology should be impactful, it should be useful, it should be friendly. But also then that thing that's absolutely necessary to starting a business, which is seeing a need in the market, seeing an opportunity. When I started digging into it and first of all figured out that Ericsson at the time didn't have a solution, I then went externally and figured out that most companies also didn't have a solution. But at the same time, virtually all product teams were running some sort of beta test for whatever technology they were bringing to market. Everybody had the anxiety of how is this going to perform out in the real world? What are going to be those variables that we didn't account for in our quality efforts and in our research efforts that the real world is going to throw back at us and could create risk?

It all just came together for me as this amazing opportunity to address what at the time very much felt like a necessary evil. Everybody felt this had to be done, but it was painful. It was challenging. I saw that as an opportunity to leave corporate America, frankly, control my own destiny and build something exciting. So that's what I did.

Rahul Abhyankar [02:49] Interesting. So everybody has a different way of doing something that everybody needs to do. What you're doing is really bringing some structure, some organization, and some consistency in how companies can go about doing beta testing.

Luke Freiler [03:04] Yes. Every company was doing this, but everybody was doing it differently. Nobody had really a process brought to them, which meant everybody was sort of forced to roll their own. We saw that as incredibly wasteful and not typical in most cases, these kinds of things that everybody were doing. Somebody recognizes pretty early on that we could do this better. We could do it more efficiently. No need to reinvent the wheel. But in our case, everybody was reinventing the wheel. One of the first things that we set out to do was not just to find the technology to solve the problem, but rather to turn beta from not just an idea, but a system and something that could then be reproduced and scaled and learned from and benefited from without reinventing the wheel. So that was very critical to our success and growth.

Rahul Abhyankar [03:48] So the shift from waterfall to agile—was that a good trigger point for CenterCode to say this is how we can really emphasize the value of testing as a continuous testing process and not just prior to a release?

Luke Freiler [04:10] Absolutely monumental for us. In fact, our earliest success were with very large companies that were effectively agile before agile. Our first customers were companies like Sun Microsystems, who was making an operating system, and an operating system is never done. There's always more to be done. It's always iterating even from the earliest days. Following that was Symantec, who at the time was working on security software, virus scanners—again, never done.

So we sort of fell into the agile development because these were the companies that had a constant need, which forced them to centralize this effort. Otherwise it was going to be too decentralized. They were never going to get anything out the door. Since then, what we've seen is most modern companies evolve into the same general idea, because most products have become services. Most products are getting more complex, which means that more can go wrong. They're getting more connected, which means their success is reliant on something that's typically outside their control and it's a moving target. As a result of those first two, they're iterative, they're always changing. Customers are expecting more out of them and going to be very vocal about that.

That shift from decentralized, where every product manager is effectively rolling their own and kind of forgetting everything they learned and having to relearn it every two years as they get a big product out to market, has shifted to most modern companies recognizing that continuous user testing is very critical because there's always new risk being introduced into the marketplace. Our sweet spot for customers for some time was most tech companies that were founded in sort of the last five to 10 years, because those are companies that are at scale to recognize the problem and really need to centralize it, but also grew up in an era of agile. They don't have that waterfall behind them. They've always been in an iterative mindset.

Rahul Abhyankar [05:53] And the interesting part that you talked about was really the products that need to integrate outside of so many other things that they need to touch and integrate with. In old days, a toaster was a toaster, but now all of a sudden the toaster has to work with your router, for example.

Luke Freiler [06:12] Yes. My thermostat has Spotify support for some reason. It is absolutely insane how many products are connected now. It's actually very difficult to think of a product in the tech space at all that doesn't have some sort of connectivity or interoperability that they're reliant on. It's kind of mind blowing just how much it's grown.

Rahul Abhyankar [06:35] Luke, you at CenterCode have a really impressive roster of customers. You could take any letter of the alphabet and you've got a significant company that's one of your customers, more than one, right?

Luke Freiler [06:51] Thank you. Yes, it is my proudest achievement.

Rahul Abhyankar [06:55] You could perhaps have your own alphabet rhyme song—A for Apple, A for Amazon, going down all the way to Z for Zoom. But given that every company does some type of testing, has to do some type of user customer validation, who do you think is not the ideal customer for CenterCode?

Luke Freiler [07:28] The only companies that we typically don't work with that are producing technology are companies that don't really have end users, meaning their product is part of a broader solution. So if somebody's making chips, chances are we don't work with them. We work with whoever is going to ultimately implement that chip into their product. Now even those companies are getting more diversified. Once upon a time, I used to list Broadcom as the type of company we wouldn't work with because at the time they were making chips primarily. Since then, they've acquired four or five of my customers in those spaces. So the VMwares and the computer associates and whatnot, as they've grown, they've incorporated those, and now Broadcom's one of our biggest customers. So it really does change.

The other thing that's kind of funny is we used to always list the consumer brands as the brands that we didn't typically go with. If it's toothpaste, it's not really in our wheelhouse. There's only so much that can go wrong. Frankly, other than your toothbrush, it's not connected. But even those things have evolved—not so much toothpaste, but the toothbrush, in many cases, they're now connected. So we're now dealing with a lot of brands and companies that never would have been on my target list, but they found us as they introduced more technology into their products. And we're introduced to all the complexity that that brings, and again, all the risks that that brings as well.

Rahul Abhyankar [08:54] That's fascinating. When you are creating a company such as CenterCode in a category that you're really creating the category, what have you learned in terms of how do you establish that, define that market, define that category, and then get people to become aware that there is a solution in this category that satisfies their need?

Luke Freiler [09:14] It's difficult, it's time consuming, and I think the alternative is that it can be very expensive. In our case, we bootstrapped this company initially, so expensive wasn't an option. We were definitely a bit ahead of the curve and companies were starting to find us. Those first companies were thankfully important enough names in the world that they brought others.

But I would say that there were a couple of things that really shaped it for us. One, we fell in love very early with content marketing. I discovered inbound or content marketing and it was explained to me quite simply as whatever your company does, regardless of your product, you need to think of yourself as a publisher. You can't subscribe to secret sauce. You really need to publish everything. You need to think of your website as a magnet. The stronger that magnet, the better.

We had done so much experimentation and learning and so on that we were just bubbling over with knowledge, and that created a great opportunity to put that knowledge online for free to basically build ourselves as an authority. We were putting all of everything that we were learning, basically bottling it up and giving it away for free, and we used that as our primary means to attract an audience.

From there, we then decided, okay, what we need to do next is build a community. The neat thing about having all those logos is they're very interested in talking to each other and learning from each other. We can be the source and sort of the orchestrator of that. So we started to do user events and user groups and kind of a road show across the country. It was very inexpensive, frankly. We'd go to a different city every month, and with us we'd bring food and invite a bunch of people and just have conversations. We'd try to get a customer to come speak and so on. It just sort of built its own momentum around there.

We started by just giving out as much knowledge as we could for free, being that thought leader, being that authority, and then leveraging everything everyone was learning to get them all in kind of the same room and talk about it and build a community of people. For us, it meant establishing vocabulary. One of our biggest challenges early on in something that is more idea than system is everybody has a different name for it. Everybody talks in almost a different language. We once queried our own audience and we found something like 90 different names for what they call these programs within their companies. This was quite a while ago. That was actually an inspiration for us to say, okay, that's problem number one. We need to be speaking the same language for this thing to grow. So we started establishing our own terminology, baking it into our platform, baking it into our content, our events, and making sure that our customers were saying the same things to each other so that they could communicate more effectively and efficiently and things would move. That was a big win for us—kind of establishing the vocabulary and doing everything we could to permeate it out there.

Rahul Abhyankar [12:03] That's very interesting because now you have an opportunity to shape how people speak, talk, think about a particular category that has not existed before.

Luke Freiler [12:18] A big part of it, and this is a tip I would give anyone, is many of our customers needed to build teams around this and they didn't know how to hire. They didn't know what they were looking for. Something I learned very early on is if you're willing to do the work for people, they'll take advantage of it. So we did everything we could to literally build their job postings for them to help them understand kind of how these teams would function. That meant those job postings were for titles that we created. Those titles then became things that people were looking for, and it just grew and spread from there.

Rahul Abhyankar [12:48] I was reading the case study on your website about WebEx, and I noticed that they have a beta team lead. Now this is a job category that probably didn't exist before, right?

Luke Freiler [13:04] Yeah, and that was the problem. We've always described it as a product manager goes out and does it. By the time they have to do it again, they have forgotten everything they learned and they're just constantly wasteful in that time. Because the reality is if something is only needed every couple of years, you're never going to centralize it. But once you start recognizing that this product is iterative—again, thinking services, not products—that's when you start to think about that centralized need. That's what most of our customers have evolved into.

Rahul Abhyankar [13:33] One area that I would love to dig into is understanding how you demonstrate the value of CenterCode to customers. On the surface, the value proposition sounds great. Everybody needs to do beta testing. How do we get organized? How do we get structured? There is a platform. But how do you communicate and convince that their investment in CenterCode is really giving them the return? How do they quantify that return? Talk about how you've learned through the years to do that.

Luke Freiler [14:03] Absolutely. That was another epiphany for us. I would say vocabulary was one of the first epiphanies. One of the later epiphanies was that this is a space that due to its infancy had no metrics. It had no KPIs. Everyone was making those up as well. If they didn't have confidence in those, if they weren't proven, then it could take years to build that proof.

Our SaaS offering, our platform, is what most of these companies use. We learned very early that for us to impress our services customers, we needed to provide metrics that would quantify what we were doing. So we invented and iterated over those over the years. Now everything really boils down to three core metrics that every test has.

We have what's called the health score, which is how engaged the audience is. There's really two components to it. Are they testing everything? Are they getting the widespread coverage of the test that they need? And are they actually providing actionable feedback as a result? So health is score number one. That's sort of our core first key metric.

Our second key metric is product success. How successful is this product in the eyes of the customers? The way we get that is actually with another metric behind the scenes, which we call feedback score or impact score. How impactful is addressing this feedback going to be on the success of a product? What are the things that are broken in the product? What are the things that could be better? What do people love? So a success score in our world is a score from zero, meaning it was nothing but issues—the product is effectively broken, there's nothing nice to say about it at all—all the way up to a hundred would be the absolute perfect product. It has no issues. People are just throwing praise at it left and right. What this is showing is how an audience truly feels about a product that they've been handed after a short period of time, understanding how it works in those real environments.

Now, no product ever gets a zero, no product ever gets a hundred. Everything is somewhere in the middle. So the next score is then the third and final KPI is what we call impact. For example, let's say you had a bunch of bugs and there were a handful of really significant blockers there and you addressed half of them. We're going to understand sort of the before and after for how that product would have existed in the market had those issues not been fixed. We can run the score again for how it exists in the market, understanding that you have fixed those things. And I haven't said this part, but typically it's heavily integrated with tools like JIRA. So engineers are just fixing issues and whatnot as they normally would. That then tells the platform this has been addressed. The platform can do the math and say, okay, this has had a 20% impact on your product success score. In other words, you started out and maybe your product was a 60 and we've improved that by 20%.

So we show them exactly what the most critical and widespread issues are. If they fix those, that's going to have the biggest impact on their score. That was a lot to unpack, but in short, are we getting the testing we need? How does the customer feel about the product? And then how much did we improve the product as a result of the test? Those three things really encapsulate the whole process as well as the value prop that we bring to the table.

Rahul Abhyankar [17:11] Interesting, because I was wondering about that last part that you said, which is the engineers and the product team are getting a clear view of which issues, which bugs have the most impact on the acceptance, the successful perception of the product.

Luke Freiler [17:35] Not all feedback is created equal and you never have time to address all the feedback. Those are two objective facts in our view. Our goal is to take those two facts and allow you to maximize the resources you have to produce the best product possible.

Rahul Abhyankar [17:50] And the value of the platform is that it automates elements around this because the three scores that you mentioned—the health score, which is how much are users engaged, the success score, which is how successful do users perceive the product, and then the impact score—all these three scores, a product manager or the product team, engineering team listening to this conversation, these are three things that they could track. But doing that manually becomes cumbersome and onerous.

Luke Freiler [18:20] Yeah. Not only that, but the system is also automating the hard parts like engagement. The single biggest challenge in the beta testing world is getting customers to engage consistently. I give them this product and outside of our world, you're typically going to see maybe 20 to 30% engagement rates or participation rates, meaning one out of five people actually took the product and gave you feedback. In our world, what we see typically is 80 to 90%, which means we're getting a lot more out of ultimately fewer people because we don't have all that drop off. That engagement is managed by the platform.

Rahul Abhyankar [18:58] Interesting. Now when you are creating a new category and there is no benchmark for pricing, what lessons have you learned over the years and how have your thoughts on pricing evolved as a result? Maybe customers have played a role in shaping that.

Luke Freiler [19:16] I spoke at a licensing event once and I'll never forget something that one of the other speakers said because it resonated so true with me. He said, we've spent as much time pricing our product as we have building it. I thought that was brilliant. His name was Rick Nucci from Boomi. I have to give him credit. It was really smart because it has been a challenge. Pricing is also not static. Pricing has to shape with the market. At one point in time, we would price by say the number of concurrent projects they could run. But then the economy would shift and all of a sudden they would have a very different usage profile because maybe they wouldn't have a dedicated user. They would have a whole bunch of individual users, or they would try to share projects or all sorts of different things. So it's really shaped over time.

What I will tell you is that our first price, and I think many companies sort of face this when you're going into a market where you don't have direct competition—our first price was effectively made up on the fly. The reality is we went in and threw a price out that was high and it felt like a fit. I'll just give you the numbers and be very transparent here. It was $300,000 for our first customer. That was something that a person I brought in to help with that call threw out there. At the time I was thinking about a PLG type offering that was going to be a couple hundred bucks a month. That was where my head was. But he was an enterprise guy and we were selling to an enterprise, which at the time I was incredibly unfamiliar with. This is again, 20 years ago. When we threw out that price, I thought they were going to laugh at us, and their exact words were, that sounds fair. We can do that. And that is exactly what they paid.

Rahul Abhyankar [20:51] And now you're wondering if you've left money on the table.

Luke Freiler [20:54] Yes, I've told the story a hundred times, but I'm thinking a hundred bucks, they took 300,000, and I'm thinking should have said 500,000. But the interesting part is at the time, we were high-fiving and we were a bootstrap group of young guys. So that paid the bills for the year. It was wonderful. They had no idea. They basically put us on the map. But the problem was at that point, we thought that's what the market would bear. That became the mindset of, okay, if we can sell it for that, we can sell it for that again. We did, we sold it for that a number of times after, but it was very slow and no deal was ever as easy as that first deal. What it took us too long, frankly, to recognize was that while they were very happy to spend that, it was because their context was very unique. Most companies weren't thinking like that, but Sun was. They had a big team that was going to live in our platform and we were going to be a force multiplier for that team. So $300,000 was a very easy spend, but we got hooked on that price, not recognizing that most of the market wasn't in the same profile as Sun.

Where it got really interesting is that I was now going to bat as the product guy in my company. I wasn't the original CEO. I was CTO and running product at the time. I was going to bat against the sales guy who said, look, if they'll pay for it, others will pay for it. I was sitting here going, okay, but how about instead of 300,000, it's 100,000 a year. And typically we keep our customers at least three to five years. So it'll actually be more, it'll work out very well. I got him to accept that. Then later I went back to him and I convinced him, hey, what if it were 25,000 a user? We typically have an average of four to five users, so it's basically the same. At one point I realized that after about a decade of doing that, the prices that we were using at that point were me pulling out some sort of backwards rationalization five times in a row from a price that was made up with absolutely no research. It was actually at that point in time that we kind of threw down the gauntlet and said, okay, we've been living off that first random decision that was never even discussed for a decade now. We've grown with it. It's been good to us, but there was no real research to it. That's when we started to really take pricing seriously, invested in working on pricing as a very significant thing, and we've learned endless lessons since. Value-based pricing was—

Rahul Abhyankar [23:18] Would love to hear what you can share about some of those lessons.

Luke Freiler [23:25] Probably the most important thing was to really recognize how the market aligns their usage with the value of the product. Is it the number of products that they're releasing? Is it the number of users that are using it? I think what most companies immediately zero in on is kind of the most generic thing possible, seats, for example. But we actually had a lot of success saying it's not really the seats that matter, it's the amount of products that they're releasing. Because every product that gets tested is where the risk is. We can associate a lot more value with reducing that risk than we can just another user getting access to the system. So really aligning it with whatever the vectors of value are was one of the most important things.

For us, it was very unique in that it was never about competition at all. We never had direct competition. The companies that have existed in our space have typically been targeting small niches or big niches, like mobile specifically, things like that. So we really couldn't draft off of anybody.

The other thing that we learned is that the bigger the company, the more value that would scale with what we were providing. It wasn't just about, they're a bigger company, they can pay more. It's no, they're a bigger company. The value that this product, that we bring to this product that's going to be sold to 10 million people is much more significant than a smaller company producing a product that's only going to be sold to half a million people. So that value scaling with that was a really, really key part. Those are some of the big ones.

I will say that more recently, the thing that's kind of lighting the product world on fire is PLG, product-led growth. I think PLG is fascinating. I'm in love with it. It's actually where I wanted to start this company 20 years ago, long before PLG was really a concept. That kind of self-serve thing was what I wanted to do, mostly because we were primarily engineers and that's what we felt we could deliver. We weren't salespeople, but we could develop a system that was effectively e-com for software.

For us, we've been experimenting with PLG, but it's challenging. It's a very costly thing to drive the amount of interest in it. We've never been a company that is really about thousands and thousands of companies using our platform. It's really been about hundreds of much larger companies using it. So we experimented with PLG, but then basically paused some of our efforts, realizing that the sort of free version nature of PLG, the ability to go in and try a product, that's brilliant. You could do that on our platform today. We love that. But the self-serve thing, as obvious as it sounds, didn't work for our audience. Most of our customers, we kind of recognize that while it's really low friction to be able to get in, there's so much security and things like that that large companies expect that are going to—security compliance, GDPR, all of that—that they need to get past their internal teams. That was actually the hurdle. If they didn't have a direct connection to talk to us about it, then spending 40 or 80 bucks a month to get started didn't even matter.

So for us, we've experimented with pretty much every side of it. I love the free adoption, low friction, get in the door side of PLG. I think it's brilliant. But the kind of self-serve, put it on a credit card and go, wasn't really compatible with our audience. It didn't really make sense. Much smaller companies will love that, but the larger the company, the closer we need to be to help them get over that sales hurdle. Because them actually bringing software inside a large company is a lot of work. There's a lot of friction on their side that a human can help them with, but self-serve just didn't do it.

Rahul Abhyankar [26:55] And then there is also the aspect of having services, professional services, along with the product, the platform, and part of the sale, where that PLG self-serve doesn't really align with.

Luke Freiler [27:10] No, our goal for PLG was actually never to replace our market. It was to augment it. What we wanted to do was have an offering for those companies that haven't yet adopted their own program. PLG never made sense for any of our customers that have built a team around this. This is an implementation, right? They need to build this program. They need to understand these KPIs. This is going to be a key critical aspect of their business. PLG was never going to make sense for that as the end game.

Our goal was all the companies out there that have product managers that don't have a solution available to them, that we wanted to provide something low friction for them so that the company could start to see and experience that value and then move up. What we ultimately realized is we can actually just achieve that with the free version. It's very inexpensive for us to offer that to them for free, let them use it, but then once they do want to move in and bring a human in, that part in the middle of them just kind of signing up with a credit card and growing organically was unnecessary. It was sort of incompatible with that enterprise play. So by having a free offering and a direct sales model, we get the best of both worlds and they get to choose what makes sense for them at the time.

Rahul Abhyankar [28:20] That's a great product decision from your side to be able to say what should we not do and why. Now, back to that point about value-based pricing and identifying the variables and the elements for which you can define value. There is an element of, I guess there is a ceiling to that because there is a budget that the customer has allocated to what they can spend or how much they are willing to spend for a beta program, beyond which it doesn't make sense. Do you feel that that budget really comes from, well, if a product has a certain revenue profile or a certain market footprint, then a customer going in their mind thinking, we are willing to spend X percentage of that to defray the risk of this product not meeting the target for customers. Is that how you're seeing that play out? Because where I'm going to get to is we all talk about value-based pricing as the thing that as product people we should do, but there is the corresponding element of where the customer is drawing the line for budget.

Luke Freiler [29:46] We have both an advantage and a disadvantage there. It goes back to we're not typically in a hyper competitive bid situation, first of all, which is also another killer of value-based pricing—it ultimately ends up a race to the bottom. We're not in that. But what that means is they don't necessarily have an expectation going in. Now the con of that means they probably don't have a budget for this. They have a pain, but they don't have an established budget. They don't even know what it should cost. That leaves it open to interpretation. The other side of that means it doesn't really have a ceiling per se. So it's all about how well we can sell the value prop, which is why associating things like KPIs directly to value gained is so important for us and why that became very critical.

For us from the earliest days, thinking about ROI, building a convincing calculator that allowed them to put in all their own variables and tell the right story was very important and something we still use today. The nice thing is we address a lot of things that are well studied—quality and those costs.

I will say that the easiest deals, and this is sad but good I guess, the easiest deals for us, and they happen reasonably frequently, is when a company has some sort of critical event that goes poorly and they contact us shortly after. What that often means, and this is hyperbole, but CEO freaks out and they've basically got a blank check to go make sure it never happens again. At that point, there is no such thing as a true blank check or I'd be on a yacht right now, but the reality is at that point, it's all about associating the value of how can we make sure that thing doesn't happen again. It really is a true value-based sale when you're not in a competitive bid situation.

Rahul Abhyankar [31:40] Now let's switch gears to generative AI. I'm assuming that a lot of products that you are seeing being tested through your platform have some element of AI, machine learning models, generative AI. These are scenarios that you don't know how the system is going to perform. How do you take that into account when you are working with your customers to define these beta tests, define what gets tested?

Luke Freiler [32:06] I'm actually living both sides of that right now because we are a technology company ourselves. We are building generative AI and agents into our own platform. So we're seeing that firsthand. We're also seeing it in our customers. We're seeing a lot more opportunity in that space. I will say that we're just now starting to see it in a lot of products we're testing because while it's been around for a couple of years, when you do work with primarily large companies, they're reasonably conservative about it and they've got a lot of concern about putting it out. Ultimately, it's very good for us in that we need a broad audience to test those things, and our customers are recognizing that.

Rahul Abhyankar [32:48] So when you build it into the CenterCode platform, as a product manager, what kind of questions does generative AI within CenterCode help answer?

Luke Freiler [32:59] We're approaching it a little bit differently. It's not exactly generative AI that's interesting to us. It's actually kind of the next generation of agents. This is something that my product team has been talking about a lot. Over the last couple of years, we've all either fallen in love with or despised the idea that I can go ask a question and get this answer. It's been a fascinating point in technology. But the next generation, where it impacts products like us, isn't just you ask a question and get an answer, but rather it goes out and does work for you. It clicks the buttons, it does the things that you would have previously had to do that frankly in most cases are the boring stuff.

A couple examples of things we're doing right now that I'm very excited about. One of the most unique aspects of our platform is the idea of a test plan—the idea that I want to break my product up into a series of features, and each of those features needs to have some sort of direction associated with it to make sure it gets tested. That concept, that system is something that in order to build a good test plan, you have to understand. Now you can learn it pretty quickly and most product managers are pretty bright, so they adopt it rapidly, but we don't need to do that anymore. What we sort of discovered is we can take generative AI and have it read whatever product materials they want to feed the system. They can give us a PRD, an MRD, brochure, support docs, whatever, even handwritten notes about the product. We can turn that into a test plan. We can then give that to them in a format that allows them to review and approve it and have that human element.

Ironically, and this is kind of what's so interesting about AI, rethinking the whole user experience and making it so easy to review the AI's work and whatnot was actually more effort than the AI part itself. The AI part itself, I won't call it a solved problem because it's evolving so quickly, but the very nature of it is that it does obscenely complex things nearly for free in terms of work. For us, it was all about how do we take that AI magic and encapsulate it in a user experience that's very simple to adopt. What we have now, and it's something we've been demoing a lot over the last few weeks because it's coming to market very soon, is you can feed it whatever information you want about your product and it's going to build a comprehensive test plan.

Those kinds of things are great. Really, really fun to develop, honestly. Just speaking as a product manager, it's incredibly exciting stuff to work on. I smile every single time it runs and creates a test plan for me. Like it hasn't gotten boring yet. If we can take an hour and turn it into five minutes, then that's reclaimed time they can do a lot with. That's kind of where we see it all going right now. It's really awesome.

Rahul Abhyankar [35:42] Interesting. As you've been building CenterCode, the company, over all these years, what are some leadership, hiring, team building lessons that you've learned?

Luke Freiler [35:56] I'll tell you one of my favorites. For the longest time, I did not like, for lack of a better word, the HR or human side of the business wasn't as exciting to me as the product side. But when you're leading a company, you're building a team, there's humans, there's interaction. I always loved the idea of building a great culture. I wanted to work in a great place. So that was important to me, but it's a hard thing to define. At the same time, I was dealing with regulations and just things that all felt like they were slowing me down and I was getting frustrated.

I went out to lunch with another product manager and I was expressing this frustration with the sort of operations side of the business. He gave me a piece of advice that in hindsight was one of those major epiphanies in my journey. He said, look man, you need to treat your company like a product. You are the product manager of that company. Your employees in this case are your primary audience. Your customers are actually your secondary audience. You can't have the second without the first. You need to design the company. You need to develop systems for the company.

There was something about that. It was so obvious in hindsight, but as soon as I started treating the company like a product and I started literally versioning how we did performance reviews and how we thought about the culture and our core values and so on, and just truly treated it like an engineer turned to product manager would treat a thing, I started to love it. Now it's actually one of my favorite parts about running a company is designing the system that runs the company, designing how we onboard employees with my HR team and how we think about performance and how we optimize that just like you would features on a product. It has been one of the funnest parts of the journey in recent years.

Rahul Abhyankar [37:42] Beautiful. Going back to what you said in the beginning, which is coming to it from your background of design and usability, I can imagine why that got you excited looking at it that way.

Luke Freiler [37:51] Dead on. Absolutely. I like to design things. I like to build things. Designing a company is really no different than building and designing a product. There's so much you can learn from both.

Rahul Abhyankar [38:06] Over all these years, how have you kept yourself inspired, motivated, and learning?

Luke Freiler [38:14] I am and remain incredibly enthusiastic about technology. Last night in my free time, I was digging into generative AI image creation locally and how models work. Frankly, it was one of those things where I was just trying to learn the definition behind every term that I've had float past my desk in the last year. I am enamored by technology and what it can do for the world.

The other area of it is I've always had kind of a side interest in music production and specifically technology infused music production. My other early job was actually moving into music, working in a music studio. Originally I was going to go either the music route or the tech route. Eventually, when I got the offer for Samsung and fell in love with that side, I went the tech route, but I've never given up on the music route. One of the funnest things is that we do business with probably most of the companies in that space now. Also, if I truly have creative juices flowing, you can't see it behind me now, but there's synthesizers and drum machines and all kinds of music production stuff that is my real outlet.

Rahul Abhyankar [39:27] Love it. Awesome. Any parting words of advice on product quality testing to our audience?

Luke Freiler [39:35] I think the main thing is everyone needs to be curious. You can't be in this space and really enjoy it if you're not consistently curious. We consider it a core value at our company for a reason. I at least am very motivated by being around other curious people, people who want to learn and teach. I just would encourage everybody to keep that up regardless of the size of your company, the size of your audience. There's so many opportunities out there to build amazing things, and it's only getting easier frankly. So definitely just keep learning.

Rahul Abhyankar [40:08] Fantastic. Luke, thank you so much. Thanks for being on the show. This was such a fun conversation touching upon so many different aspects of product.

Luke Freiler [40:18] Happy to. Thank you for the invite, very much appreciated. Definitely if anyone in your audience wants to reach out, I'm very approachable, luke@centercode.com. Always happy to talk about this stuff.

Rahul Abhyankar [40:29] Fabulous. Thanks, Luke.

Luke Freiler [40:31] Thank you.