Ep. 123 | Customer-driven Data Science with Albertsons Colleen Qiu

This week Colleen Qiu, head of data science at Albertsons, the second-largest grocery store chain in North America joins Allison Hartsoe in the Accelerator. Colleen takes us through several customer-centric data stories from Providian Financial, eBay, Paypal, Chegg, Poshmark, Tesla and of course Albertsons. You will hear how some techniques remain the same and what critical factors create data science success.  

Please help us spread the word about building your business’ customer equity through effective customer analytics. Rate and review the podcast on Apple Podcast, Stitcher, Google Play, Alexa’s TuneIn, iHeartRadio or Spotify. And do tell us what you think by writing Allison at info@ambitiondata.com or ambitiondata.com. Thanks for listening! Tell a friend!

Read Full Transcript

Allison Hartsoe: 00:00 This is the Customer Equity Accelerator. If you are a marketing executive who wants to deliver bottom-line impact by identifying and connecting with revenue-generating customers, then this is the show for you. I’m your host, Allison Hartsoe, CEO of Ambition Data. Every other week I bring you the leaders behind the customer-centric revolution who share their expert advice. Are you ready to accelerate? Then let’s go!

Allison Hartsoe: 00:28 Welcome. Everyone. Today’s show is a series of stories about customer-driven data science. And to help me discuss this topic is Colleen Qiu. Colleen is currently the head of data science at Albertsons, which is the second largest grocery store chain in North America. But if you look at her resume, it reads like literally like a who’s who of Silicon Valley. Colleen has held data science positions at Providian financial, eBay, PayPal, Chegg, of course, Poshmark, and Tesla on the list as well. Colleen, welcome to the show.

Colleen Qiu: 01:06 Thank you, Allison. Hi everyone. It’s a pleasure to be here.

Allison Hartsoe: 01:10 So normally we start out with your background and talking through that, but we’re going to cover so much of that detail today. Could you instead start with what your team at Albertsons does and what you specifically do today?

Colleen Qiu: 01:23 Yeah, certainly happy to share that. So I joined Albertson in late October last year, and then my role is to help the internal core data science team.

Colleen Qiu: 01:34 So what we focus on is to really apply modeling techniques and algorithms to help really all different types of problems that the company has. And that can include, for example, from a customer perspective where our working customer lifetime value modeling, we are also working on customer term modeling as well as other types of analysis, looking at engagement and digital marketing. How do we build recommendation engine personalization for our shoppers in-store experiences as well as their online digital experiences?

Allison Hartsoe: 02:09 You know, I think it’s interesting that you mentioned churn modeling because I sometimes think of that as something that’s really related to a subscription service, either you’re on, or you’re off, you know, I have a healthcare subscription, or I’m a member of a gym, or I’m not a member of a gym, or I’m getting the magazine, or I’m not getting the magazine. Could you talk a little bit about how you think about when somebody is churned just briefly?

Colleen Qiu: 02:35 Yeah, So to simply put right now, we define churn as in the next month or next three months, the customers are not making purchases with us. So that is how we label churn today. Definitely, we don’t have subscription. We do have a loyalty program where customers can play with the coupons and apply their personalized deals. But as far as the charity defined is really simple here, we look at the customers, the who typically would make some purchases in a three-month duration, but when we noticed that they stop making that purchases, then we consider that as a churn.

Allison Hartsoe: 03:11 That makes sense. I think a lot of people do that as they have to put a stake in the ground somewhere, and three months seems like a good logical place. And there’s probably a lot of modeling that goes in behind that. Let’s first clarify the difference between when we talk about product-driven versus customer-driven data science. I always think customer lifetime value is the heart of customer-driven data science, and you might agree, or you might have another take on that, but it seems like it’s been lately growing in popularity. Do you think that’s true, or has it always been there?

Colleen Qiu: 03:47 I feel that has always been there mostly from my personal career because of when I first started working for Providian financial around 2000 in San Francisco. My first job is actually to look at the customer churn lifetime value, and we have a program where we call it retention program. Basically, we look at our cardholders to predict, uh, who are about to, to kind of maybe get another card, and we use that model and now with their lifetime value to make a decision on, okay, what other offers can we provide it to the customer, trying to keep the customer with us and to keep them engaged. So from my perspective, I think my very first job is already applying modeling and doing predictive type of work there and really focused on making sure the customers are staying with us.

Allison Hartsoe: 04:37 Was that using more recency frequency analysis, or was it through projecting forward CLV?

Colleen Qiu: 04:45 CLV wise is a different, more like a model, but the attrition was a logistic regression, which today you actually would to consider it part of a mission learning algorithm

Allison Hartsoe: 04:54 Early on, no doubt. But I think a lot of the old science is new in that, you know, customer lifetime value and the early database systems, even though they were just using zip code data and demographic data, we’re using more advanced modeling programs. Is that what you had here to work with at Providian?

Colleen Qiu: 05:16 Yeah, that’s kind of, we’re already looking at the customer level data. We’re calculating the lifetime value of the customers based on their chest fashion, their usages with the credit card that Providian offered back then.

Allison Hartsoe: 05:29 And did you have external data that you were able to pull in to try to understand whether they were ripe for retention modeling or whether they were going to get another card?

Colleen Qiu: 05:41 Not at that time. I believe there are like a research team inside the company who are looking at that, but from our internal use cases, we’re actually a hundred percent relied on internal data, mostly kind of the hall customers engage transact to using their cards and some of the demographic attributes around the house as well.

Allison Hartsoe: 06:03 That makes perfect sense. Thank you. So obviously coming, you’ve worked for these amazing brands. Can you share some examples from your experience about how customer-driven data science was conducted and maybe a little bit about how it evolved?

Colleen Qiu: 06:19 Yeah, certainly. I think my second job is with the PayPal and the eBay. And when I was at eBay, the company at that time, we had a strategy where we call it a triple-A. So it’s really turning into customer focus, acquisition, activation, and attrition. So at that time, we believe we can solve the triple eight, and we actually can really grow the community and grow the business. And then, from then, I worked at a startup company called the Chegg, which is really focused on college-age students. And there, I was kind of supporting the so-called a student advocacy team and that team, their primary job is kind of like customer support function, but the team was put in a core position for the company’s future. And the reason for that is that as a student out of a kid, you want to really speak on behalf of students.

Colleen Qiu: 07:12 You want to see what students are going through when they are using our platforms to order their textbooks or to solve their homework help needs. So from there, kind of you started bringing the insights and the feedback that you gather from the students, and you take it to the different departments of the business. So where we work with the product team, operational teams, et cetera, et cetera, the goal is to really help drive the business to more effectively serving our students in the community. So that was at Chegg, and then later on at the postmark, it’s a social media platform. I mean, it’s really fun to be there. And their job is really kind of trying to sell fashion through the community. Right? And I wasn’t there briefly, but definitely, the company has a huge focus, and I think they kind of say, you know, they need to love their community and really make their community love their services.

Colleen Qiu: 08:08 And so there’s a lot of like focus on community and at both eBay and the Poshmark, they used to order organize these large events, which is really bringing all the online user groups together physically. So kind of become like you get really to see each other and interact. And the face to face. We basically called the eBay live and Poshmark. I forgot the name, but definitely, they organize it once a year. It’s a major event, the whole community customer gets together. Everybody enjoys each other’s company and also gives the company more feedback on call the product they are building can serve the community better.

Allison Hartsoe: 08:48 I have to laugh because I remember I’ve been to several eBay lives. I used to have a company that on eBay and these events, I mean, it’s not just any old event. E-bay live is like Disney World for a week in a convention center. It is crazy. And all of these people have all of these different types of badges and banners that they put on their name tag, and it is really special. And you could see the heart of the company coming through that. But I didn’t know, until you just said it was how much the company was listening to the community at that point, instead of just gathering people together, really using it as a bi-directional event.

Colleen Qiu: 09:32 Yeah. And most recently, so before I was heading up the global service data science analytics team at Tesla. So at Tesla, as everyone knows that we have almost zero marketing budget. So the philosophy there is if we make the product and we serve our customers well, then we really don’t need to spend the money on marketing. And I think that the company, the huge at least expect chiming that to go as well.

Allison Hartsoe: 10:01 What was that for experiences we talked about? And I know that’s just kind of skimming the surface of your resume, but yeah. We take eBay and Chegg and Providian, Poshmark, and Tesla, that group of five. Do you think that all those companies had in common, the ability to combine the quantitative and qualitative side to get an intuitive sense of the customer?

Colleen Qiu: 10:25 Yes, definitely. I definitely feel that. I think there’s always a huge focus on the quantitative side because these companies are mostly at a massive scale, so it is dangerous. And, you know, if you need just to talk to a subset, a small sample of customers, and you think that you’re getting the clear read on the sentiment from your community. So definitely we are looking besides the qualitative survey support, phone calls, or other type of gathering the feedback from the community like research studies, et cetera. There’s a huge focus on leveraging data and data science to trying to get effective read with no buyer across the board, using all the data the company has.

Allison Hartsoe: 11:08 So that can be difficult. So, for example, I’ve seen in other companies where they gather feedback from the community, but it is anonymous feedback, or it’s not connected to the data set, or it’s siloed in one part of the company and the other parts of the company can’t access it. So was there something about these companies that said we really must have the quantitative match and aligned to the qualitative?

Colleen Qiu: 11:35 Yeah, I think sometimes it’s just to be Mario macho where that what we’re trying to make or business and trying to do is make the best decision given all the information they can gather at the time. So the best thing to do is try to imagine like the different types of information that can help you make that decision. So definitely, I will just say you do not want to make a bias decision, and the best way to do that is trying to gather as much information as possible before you make that final call. And you don’t want to say, Hey, you know, I read ten different feedbacks and therefore this product thug or does the customer experience is not the optimal. You actually also want to look at the transactions and the conversion, all these activities that the community is doing so that you get a clear read because, in the end, I don’t think that there is such a thing as a perfect customer experiences or perfect product right? Today we focus a lot more on personalization because we realize different people have different preferences. They behave differently; they expect different things differently. So how do you give into users the best optimal experience that they prefer? I think the best thing to do is trying to connect all the dots and to make the best final decision after you review all the information that you have.

Allison Hartsoe: 12:56 So I think that sounds logical on the surface, but I’ve also seen a school of thought that says, and it’s particularly from a Uson training seminar that, that trains people on user experience and they made the argument that you don’t have to hit the pothole a hundred times to know that there’s a pothole there. And essentially that small data can be very valuable for making quick decisions, especially when it comes to UX and design. But I think there’s a similar argument for big data in that you don’t want to throw so much information in there that you’re correlating with the number of sheep herders in Afghanistan. So how do you decide, or how do you shape the data to be meaningful enough, but not end up with spurious correlations?

Colleen Qiu: 13:44 Yeah, that’s a very great question. I think from my perspective, what I observed the best is actually started from the objective. What kind of problem are we trying to solve for, and what type of solutions are we trying to do? I think that there are times that, for example, Tesla, before we make the car, no one knows how popular the car would be. It wouldn’t be crazy that we say, Hey, we come to make any car unless we gather a lot of data about this car. So you do have to rely on experiences there, may be, you also rely on user studies, et cetera, trying to get a read on what is the product that you want to do. But I think though, in today’s environment, there is so much data out there. I think that even sometimes, the company itself do not have data because you have not announced the product, but there are actually data out there that you can borrow to formulating the study before you kind of match forward based on just the user experience of study. So I think it goes back to the objective, what kind of problem you are trying to sell. And then from there, you kind of figure out, okay, what kind of data is useful in the circumstances?

Allison Hartsoe: 14:54 I think what was interesting too, that you said in the very beginning was not just the objective, but you also mentioned what kind of solutions and in a way, that’s a really interesting angle that I don’t think everybody thinks about. A lot of times, a data scientist will approach the problem and say, okay, what kind of data can I get together? And what kind of output can I figure out that’s interesting in this data? But the ability for the company or whoever is the stakeholder, whoever receives the data, the ability for them to take action on that solution is what I heard. When you said, what kind of solutions could you have? Is that something that you actively take into account when you’re forming your objective?

Colleen Qiu: 15:35 Yeah, definitely. I think when we take on any conceptual projects, we tend to try to figure out several aspects of the project, start from what problem are we solving? What’s the objective, but it also kind of goes down the path of what is our vision on the solution, what type of data, and what type of in the data science space, what type of modeling or algorithm do we want to try? And how do we envision in the end, this whole experience, whether it’s a customer experience or as a business decision holder or decision-makers, they experience the change using whatever we are building based on data?

Allison Hartsoe: 16:12 It sounds like you’re really thinking through the problem. And I want to go back to what you said at Chegg, where you were part of the student advocacy team. Cause I think that’s a really interesting place to pick up your customer feedback. In this case, student feedback. Did you have any interesting challenges there where you might’ve had feedback in one direction, and maybe you didn’t have data on the other side, and you’re trying to rectify the two?

Colleen Qiu: 16:36 Let me say that’s a great question. I think that there are a few projects that I remember to date very clearly. One project is related to how we change the whole checkout flow on the site. So it was initially just primarily hearing from the community that from the students that the checkout process takes a lot of time and it’s not the most easy, exciting experiences for them. You know? So what we did is we went to the product team. We look at, okay, how many pitches does the student have to go through to do the checkout? And also, how are we differentiating existing customers versus brand and new customers check out the experiences. So just by doing some of the most basic checks there, we actually identified the opportunity for existing customers where we basically can prefill a lot of the information. Just ask her for their double-check and the confirmation before we place the orders. So really reduce the number of pages they have to go through to finish the checkout. And similarly, we did the improvements for new customers trying to make their checkout experiences less step with the fewer staff and easier for them to finish. So that itself, we were able to improve the checkout conversion rate by double digits, it helps the business, but also, we get great feedback from the community.

Allison Hartsoe: 17:59 I can just say like most of us who have been online and have had to exhaustively fill out a form where someone should have prefilled it for us. That is so wonderful when that happens, and the company can actually think through the customer perspective. Essentially, you’re putting yourself in your customer’s shoes, not just picking up the qualitative data, but really thinking through the process to say, how can we both win? How you want more conversions? The customer doesn’t want to fill in this hassle with all the different forms, what a great place to unite both of the needs.

Colleen Qiu: 18:34 Yeah. And then the other example, there is really not too much of the data-driven, but starts from data. So based on the data we see on complaints about both arriving late, which is generally happening at the beginning of the school because a lot of colleges don’t finalize their schedule until tomorrow, the squeeze starting the students are rushing to order their books. And however, it takes several days minimum for the book to arrive on the campus. So a lot of students have the challenge where they rushed together both. And they’re frustrated that on their first day of school, they don’t have books, which actually my kids are going through this right now for the virtual learning program. But what happened is based on this complaint from the community, it was one of the top complaints. And what happened is that the company introduced read while you wait program, which is really to offer me a book to the students after they placed the order before their books arrived. So really solve the pain and create a lot of value for our customers.

Allison Hartsoe: 19:37 So once they placed the order, they didn’t have to wait. They could just start reading and catch up immediately. That’s nice, instant gratification. I bet they loved it. Yeah. I want also to go back to eBay. And one of the things that were so interesting at eBay is, forget exactly how many categories that were, but let’s say that there are a dozen top-level categories. And then within those, there are branches upon branches upon branches. So that maybe there are several thousand categories at eBay for different products. Would that be about right?

Colleen Qiu: 20:08 Yeah, I think so. It’s an ever-changing product catalog.

Allison Hartsoe: 20:13 Yes, yes. The very complex. And so if you were to take a product-focused approach and maybe eBay did at one point, was it just simply an easier path to take a customer focus? Because there was so much product, and the product was always shifting, or was there a balance between product-focused analysis and customer-focused analysis?

Colleen Qiu: 20:36 They actually ecosystem is a platform to get to the buyers and the sellers together. So different from Amazon’s model today where Amazon actually runs its own operation, have their warehouses is actually solely serving as a platform when I worked there. So eBays focus actually has always been into customers, but between the customers, we have the buyers and the sellers. So the company you’re already for a period of time, we’ll focus on growing the buyers, and it put all the focus on buyers are trying to improve their experiences. And then later on in the business, I know during some transition, they actually moved it towards the to be more seller focused, to help the sellers, to be able to do better businesses on the site. In the end. I think eBay actually always focuses on their customers. Its actually, they’re focused on product is mostly just trying to see how can we do the better product to serve our customers better, whether it’s the sellers or buyers.

Allison Hartsoe: 21:35 Now that is really interesting because I think a lot of companies struggle with the definition of who the customer is. And here you’re defining the customer in two ways, buyer and seller, which must be difficult. And I can see why. And I remember being on the inside of this and feeling the shift between the buyer’s wind orphans campaign, it’s fun to buy and seller focused, look at all these tools and look at all these new features we’re rolling out. You could definitely see that if you were close to the eBay ecosystem, without the company, putting the focus and saying you have to focus on buyers, or you have to focus on sellers, could it have been a challenge to rectify both of those audiences at the same time?

Colleen Qiu: 22:16 Definitely. I think so because there are different, the priority depends on whether you are coming from a buyer, the focus, or where this is, from a seller focus. So definitely, I think by shifting the strategy to focus on buyers, that’s during a specific timeframe of eBay. And then, later on, they actually do try to focus on sellers, trying to enrich their selection and grow their supplies, diversified their kind of offering to the buyers. And so I think definitely when you make these strategic decisions, they definitely shaped the community one way or the other. And also, sometimes, the product are tuned to different to me to serve the different priorities as well.

Allison Hartsoe: 22:59 It’s an interesting philosophy that they took in that lever pulling between the two. I wonder if you had a technology company and you had, and maybe this is apt for Tesla, I’m not sure, but if you had to focus on value-added resellers and you had to focus on direct to consumer, and you had to focus on B2B and dealers or multiple audiences, I wonder if you could still do that kind of shifting or if you would really have to almost take a certain layer of them all at once and then maybe focus a little harder in one area or another, how do you think a company should balance all of those different potential customers?

Colleen Qiu: 23:39 I think that, yeah, you did it’s kind of unique because then I thought Tesla, it’s not a platform it’s really trying to make the most energy-efficient a car. So I, Tesla, I think when we think about customers, mostly we’re trying to treat all the customers the same and also understand who has been through a little bit of rough ownership experiences and how can we help them. But on the other side, it’s really also about to revolutionize the relationship with the customers because, in the traditional vehicle, the manufacturing industry is there’s the manufacturer. Then there are, the dealers are better, but Tesla is really the first company on the end to end a relationship. So Tesla not only manufacturer, but Tesla is its own dealer. Tesla ran its own service centers and trying to control the best customer experiences to the buyers, to the owners. So from that perspective, I think even though you do segment the customers differently, we know some customers had more issues with their cars than others.

Colleen Qiu: 24:44 We typically try to think about how to do it better. We also try to take a totally different angle at customer relationships and to say, okay, traditionally, when you buy a car, you spend several months, and you have to talk to the different dealers, negotiate the price. What kind of experience do we want to offer to a Tesla owner is that we don’t really want you to have to negotiate. We want to offer you the best price the company can afford to give you. And in terms of services, we don’t need to make any money. We don’t want to make money from servicing your car. We just want you to be so delighted with your vehicle, but in the traditional manufacturing, uh, vehicle setting industry, the dealers actually make a significant chunk of revenue from services. So I think from Tesla’s perspective, because of an own, the end to end experiences, the company also tried to be efficient. Okay, how do we want to see customers and service needs being met? Okay. We want to drive to their offices or their homes to service their car when they don’t need to come to a service center. So there’s a lot of kind of innovation happening in the company who can control that end to end the journey with their customers. And I think Tesla is definitely way ahead of other industries.

Allison Hartsoe: 26:01 I could see that, and I could see the data advantage there too. Does the information that Tesla sees in the car? I imagine it. It also flows through to the second and third owners. So even if I don’t connect with Tesla, if I change and sell the car to someone else, will Tesla know that as it pulls through,

Colleen Qiu: 26:22 Not to necessarily some of the, you are sending your car second hand to another owner, we do not necessarily, know, not that I’m aware of, but I’m not there to handle that part of the data. There’s also definitely privacy concerns. We as employees there, we’re not going to see every single transaction or whatever you’re doing your car, so there’s a certain control there as well.

Allison Hartsoe: 26:46 I think the centralized process that you’re talking about owning the process end to end, must give the company a lot of rich data to work with, even if it’s anonymized. And if you’re trying to understand, is this vehicle delighting the customer or not?

Colleen Qiu: 27:02 Yeah, definitely, we can see at the body collision center, what parts are being used to repair? What kind of a collision we can also see in our repair service center, when the car comes you for annual services, what type of issues are the top challenges for the owners and et cetera, et cetera. So we get almost end to end. And the, we, sometimes we can also track there’s a particular issue probably originated during the delivery process or maybe origin from the manufacturing process. Then we would actually go talk to the stakeholder to say, okay, here’s the size of the problem? How can we work on a plan to improve the situation?

Allison Hartsoe: 27:41 So it’s interesting that I know that Pete fader over at Wharton and other people who love customer lifetime value has suggested, and I, I’ve never heard of a company that’s quite done this yet, although they may be out there, but they have suggested that when you use customer lifetime value deeply within a company, you can decide what problems to fix or how to run your supply chain or different things based on the value of the customer to the business. Was that something that Tesla entertained in that maybe if there’s an issue with different types of parts performing, and I don’t mean catastrophic issues by any means, but maybe if it’s a higher value product or a higher value customer that is attached to that issue, did it get any priority? Was there any usage of CLV that way?

Colleen Qiu: 28:35 Not really at Tesla cause the way we look at the size of the problem, every problem that’s happened into a vehicle is the theory of the problem. So we can look at how long does it take to take care of this type of problem and cost related to taking care of it and the negative impact on the ownership. So generally there, we actually treat the customers the same, I think because it is, we are trying to sell our product, and the mandate is to build the best products that we can at that moment. So, therefore, we treat all problems equally mostly. And I’m trying to think if I have a use case where we would prioritize the problem. Differently, I think actually you might currently role where we are retail. So we will have customers who spend a lot who are the most loyal and the elite customer groups with that. And that’s the where if we notice, Hey, this type of experience has negative impact on longterm engagement from customers. That’s where we actually prioritize accordingly.

Allison Hartsoe: 29:38 Because in that case, it’s not a one time purchase. And then I might purchase again in a couple of years. It’s a recurring series of purchases. And then the items within the purchase, I imagine, are also giving you a sense of the engagement and the value that someone is finding every time they walk inside the store.

Colleen Qiu: 29:58 Yeah. Yeah. Because in grocery retail, we all consume groceries every week to do our life. So the lifetime value can be really huge if you look at the whole life span of the customer or their household. So it’s a different type of environment that grocery retail.

Allison Hartsoe: 30:15 I imagine the answer to this is going to be yes, but have you seen demand in your stores? That’s more than just people buying more? Have you seen interesting inflections in demand at Albertson’s

Colleen Qiu: 30:30 What people buy more and definitely the customers that are also very different, there are more kinds of DOC career types of customers. Then there are customers who enjoy a certain type of food, like there are the baking holiday, right. They come to the store. They buy a lot of a flower. And especially during this time of the year. So there’s like a very different type of consumer behavior. So we’re observing, which is very fun and they’re all very different.

Allison Hartsoe: 30:59 Oh, that’s nice. I oftentimes use grocery stores as an analogy when we talk about data science because it’s an easy way for people to understand the different complexities in data, just like the ingredients in a recipe. So I’ll be sure to mention interesting Albertson’s uses of data science when I make those analogies along the way. So when we think about what affects a company’s ability to have good customer relationships, is it the strategy that comes down from the top that you think makes the most difference? Is it the ability to collect that customer feedback? Is it the feeling of people inside the company that they’re all operating as a team? Is it the data that’s present? In your experience as you’ve gone across these different companies and you’ve had these different stories, what really stands out and helps them connect to the customer?

Colleen Qiu: 31:57 Definitely multiple folds to the answer that definitely the top one would have come from the top. I think companies like Amazon, Tesla, Albertsons now where we have any focus on customers. That’s part of the core strategy from the leadership team. So definitely, I think you do need to try to avoid to prioritize because otherwise, you can prioritize something else rather than customer focus. So it is also proven success to create how you can grow your business and run your business as it’s going to be a forever business. So definitely I think is a key to the success of the company. And then besides that, I think it definitely comes from the bottom, right? Like from my team’s perspective, we don’t get to serve our customers day to day, but we get to see customer’s data. So what we try to do is to figure out, okay, my connection to our customers is through data. How do I leverage data to give them better shopping experiences online and offline? How do we use the data to drive a lower cost for the groceries that they’re trying to buy at a higher quality? So that’s the world we are trying to do, but I will definitely say that it has to come from the top and really make it the whole journey a lot.

Allison Hartsoe: 33:15 So when it comes from the top, it enables the tactical element. Do you also see that if you are representing any given company in the past, did it really help you to be the customer for a day, to drive around in the Tesla to buy and sell on Poshmark and eBay to shop at Albertsons and a collection of different stores?

Colleen Qiu: 33:39 Yes, definitely. Yes.

Allison Hartsoe: 33:42 And is that something you encourage your team to experience?

Colleen Qiu: 33:45 Yes, for sure. Before the COVID-19, actually, I toured several of our stores. And so before I would have just to go buy groceries as an individual, but after joining the company, I really want to also be that from a store manager perspective of how they run that store, how they greet the customer, what are their most challenging days? And at Tesla, my team, we used to with the service center, once three times per quarter, every team member, we will do that, or we can monitoring for the delivery team, but we do all sort of engagement where we really try to talk to our own customers and touch our own product.

Allison Hartsoe: 34:22 That makes perfect sense. And I think that’s one of the most undervalued pieces that I find some data science teams struggle with. They’re kind of left in a box, and they don’t get that customer interaction. Yes. Okay. So if I’m going to start my own smart customer-centric data science team, obviously, I need support from the top. Are there any other elements I really need to have to make sure that I’m staying focused on proper customer-centric data science approaches?

Colleen Qiu: 34:54 Yeah, I think I want to start by kind of sharing or create a vision. How do you want to become a customer-centric data science team? And then shortly after that, I would imagine that you would kind of map out your customer’s journey. You figure out, you know, where do you interact with the customers? How are you collecting data? And then, based on that, I think you can start applying a different type of modeling in the algorithm, trying to help optimize and automate or some of these experiences for customers.

Allison Hartsoe: 35:26 That’s an excellent summary. Do you find that there are specific areas once you start to apply those different algorithms and automation that you usually have low hanging fruit?

Colleen Qiu: 35:38 Yeah. I still feel like a CLV is a grade to know hanging fruit. Even though all companies know that they want to have their CLV, but I think there’s still a lot of opportunities for customer lifetime value to be used in a lot of decision making by different teams in a company.

Allison Hartsoe: 35:58 I certainly couldn’t agree with you more, so, Colleen, thank you so much for giving us these insights about the tremendous companies that you’ve worked with and the fact that reinforcing that CLV is valuable. Is there anything else you’d like to add?

Colleen Qiu: 36:15 No, I think I covered plenty of the stories today.

Allison Hartsoe: 36:18 Well, as always, links to everything we discussed are at ambitiondata.com/podcast. Colleen, thank you for joining us today. It’s been a pleasure to have you.

Colleen Qiu: 36:28 Allison, thanks for inviting me to this session. It has been a wonderful experience.

Allison Hartsoe: 36:33 Remember when you use your data effectively, you can build customer equity, just like Colleen did. It’s not magic. It’s a very specific journey that you can follow to get results. See you next time on the customer equity accelerator.