Subscribe to Our Digest
The actual number of data centers to be built in the U.S. remains in flux, but there’s no question demand for compute power is rising exponentially. Data access, storage and protection are key requirements for storage systems used in enterprise and cloud data centers. Microchip Technology provides solutions that improve server storage performance, reliability and security while reducing overall power consumption—a major concern for communities hosting data centers.
Brian McCarson, Corporate Vice President of Microchip’s Data Center Solutions business unit, recently spoke with Data Centre Digest contributing editor Barb Jorgensen about the technologies that make data centers hum, the impact of the memory shortage on the supply chain and the challenge of forecasting demand in an AI-driven environment.
Jorgensen: Hi Brian. Can you tell us about your role at Microchip?
McCarson: I’ve been at Microchip for about a year and a half. I came in as corporate vice president overseeing our data center solutions business. We were anywhere between No. 2 and No. 3 in market segment share in the areas that we play in in this business. We’ve been focusing very heavily on AI training and AI inference data centers along with traditional enterprise data centers. And we cover a broad range of products from storage controllers, memory controllers, and advanced networking equipment.
My background is in the semiconductor industry. I started working at Motorola in the 90s. I spent quarter of a century at Intel, and then just recently joined [Microchip].
Jorgensen: Based on recent earnings calls it’s clear Microchip is expanding its product offerings targeted at data centers. Tell us about your core technologies.
McCarson: So, let’s start with your experience when you go to YouTube. You go to YouTube and it opens up your homepage. If you click on any of those videos, they just pop up and start running right away.
If you go and search for something, and it’s a video from like four years ago, it doesn’t have very many views. You click on it; the little circle runs for a little while. That’s because the data comes from two locations. For viral videos and things that are very frequently used, you know, companies are smart. They put all that in a fast high-speed memory, but they can’t afford to store everything in fast high-speed memory.
So once people stop viewing something, they move it over to older storage, knowing that someone’s going to pull this up but I’m only getting 1,000 hits a year. So it’s okay if it takes an extra 15 seconds for that to get to the end user. You experience the same thing when you open your e-mail.
That information is stored in two completely different locations in every data center. We build storage controller solutions and memory control solutions, so whether it’s the fast, high-speed memory that you need or all that information that goes really quick, we have some of the world’s best controllers for doing that and for managing all that memory. If it’s longer-term storage, then we have controllers and software solutions that make that compatible, redundant and highly reliable.
So, we make both chips and boards and software for that portion of the market. So those are two of our three main businesses, memory controllers and storage controllers. And that’s the two main functions.
But when you’re waiting for that data to come from the memory controller or the storage controller — someone sent a command to a GPU and a CPU in a data center — and it’s told, ‘hey, go process this information and render it so it shows up nice and crisp, wonderful.’
All that data goes through network switches. We also build the switch architecture that allows GPUs to connect to CPUs and allows CPUs to connect to GPUs as well as memory and storage. And so, we build these extremely high bandwidth devices that can be positioned between those objects. We manage all that traffic and we get it to its proper location. So that’s our switch business.
But then sometimes that traffic has to go a really long distance. So instead of just going from the CPU to the GPU, which are just a few inches away from each other [on the board], it’s got to go all the way to a storage controller that’s in another data center somewhere else.
And when you’re sending that signal over long distances, sometimes the signal can start to get fuzzy. And so, we have these products called retimers, where they take that fuzzy signal, they crisp it up and make it nice and fresh again. So, you get really good data accuracy.
Jorgensen: So, specifically, are these board solutions?
McCarson: So, we make the chips for all three of those, but for our switches, retimers and our memory controllers, we do not have board solutions. We sell them to the board server manufacturers. They’re designing these huge server blades, and then they will take our chips and put them on their boards.
For our storage solutions, though, we actually do a full board solution. So, someone can go and then socket in our cards, our storage cards, into a big server rack. And if they want to upgrade from an older generation to a newer, they can unsocket the old one and put the new one in. So, at a high level, those are our three main businesses: network switches and retimers, storage controllers, memory controllers.
Jorgensen: And are these used outside of data centers as well?
McCarson: Well, they are, but we have customers that are putting them in huge, monstrous multi-acre sites, AI training and inference data centers. But we also have customers that are taking these things and putting them on premise. So, it could be in a server sitting in the back office of the coffee shop that I’m in. They may have their business analytics running locally here in the coffee shop, and the office may have a small little server box. We may be going straight into one of those OEM boxes all the way to a factory that has its own on-prem data center that might be buying their server rack on Lenovo or HPE or Dell and they go and install those servers themselves. We cover everything in between.
Occasionally, we do have some customers who want to put our products in advanced use cases. Like we currently have some customers that are designing our products into autonomous vehicles, because those are really kind of data centers on wheels, if you would. They’re processing so much information. They’ve got to be so fast. They’ve got cameras and LIDAR systems and everything. So, you know, probably 90-plus percent of our business is traditional large-scale data centers
Jorgensen: The need for compute is out there and demand is going to continue to rise. How are you in terms of your demand and supply? I would imagine demand is high. Are you producing to order or are you seeing customers over-ordering? What’s the product flow from the supply chain standpoint?
McCarson: That’s a really good question. The market is a little confused right now. I think that we see unbelievable amounts of spending in some areas and it’s consuming capital budgets. So, we’re seeing spending pullbacks in other areas.
So overall net spending is certainly going up, but it’s going up in confusing and sometimes unpredictable ways. Furthermore, you’ve got these major data center projects, which would be $12 billion data centers being built. And then they’ve got the plans, they’ve got the product committed, they’ve got everything, and then the electric grid can’t handle it. Or the community tries to get it shut down. And so, companies end up abandoning those projects. But they’re not abandoning the data center because the business model still exists. They’re just relocating it to another location. But when they relocate to another location, now their program is delayed by 12 months.
Does it make sense to build a brand-new data center with 12-month-old technology? Or, as long as your schedule got pushed out 12 months, what were the decisions you made about, ‘well, that’s not ready yet and that’s not ready yet, so let’s go with the old one, but now it’s going to be ready.’
What’s happening sometimes is, you know, a $12 billion data center gets built. All these orders go in. The OEMs start building everything. All the chip providers start building everything. It gets close to time and they say, ‘nope, time out. We’re going to cancel that order. And we’re going to switch to the next generation technology.’ And so I think a lot of the distributors and OEMs are a little bit nervous about that.
Jorgensen: I can confirm distributors are trying to manage anticipated demand.
McCarson: And then you’ve got the NVIDIAs of the world that are moving from being a chip manufacturer to being a board manufacturer to being an entire data center manufacturer. And now they’re building entire complete racks with everything in the entire building is all NVIDIA. And these orders are so big, they’re going into the supply chain and securing capacity. And so, the supply chain is saying no to other people’s orders because they got this huge NVIDIA order come in that’s going to be hard for them to do in time. So that’s creating supply chain ripple effects as well. So, there’s a lot of really strange economic and business practices happening right now. That’s creating, I think, a general anxiety around the data center markets. But yet, the demand for the products continues to be extremely strong.
But I think that there’s a lot of concerns about an AI bubble, not because humans are going to go back from using AI, but it’s just not clear how many of these companies really have a good business model, how many of them being propped up by crazy weird lease deals that they’re getting or, you know, they only get it because NVIDIA is going to fund their building, like the latest one that just came out in the news. So Nvidia is actually buying a data center to lease their own hardware into, so OpenAI can then rent for 10 years from them. Like, okay, so NVIDIA is now in the real estate business of data centers.
Jorgensen: There’s always something in shortage, which only adds to the confusion in the market.
McCarson: I think there’s a bit of confusion for sure. And, you know, I was sharing with you, kind of transparently, my personal view of the ecosystem that wasn’t a Microchip view of the ecosystem. I think if you were to ask for, you know, something floatable, I would say, you know, it’s probably my position is the AI workloads are changing so fast, week over week, with the new frontier models, that it makes it difficult to plan all the hardware three years, four years in advance for something that’s changing on a daily basis. And we’re seeing new bottlenecks show up with memory, storage and networking.
It’s no longer just that you need a faster GPU to develop a better model. And the economics now are so expensive that a lot of companies are worried about what’s the right balance for me in terms of ‘how much inventory should I carry, how far should I pull out my orders? And then how do I deal with all these really large deals that are creating a big supply chain disruption coming through the market?’ That’s probably my official Microchip input on that. But just from a personal level, that was my first question about, ‘hey, you know, these are some of the things we’re seeing in the market that has it described in our heads.’
Jorgensen: Do you outsource your manufacturing, or do you manufacture in-house, or both?
McCarson: We, as a company, do both. We have numerous factories worldwide, both for wafer manufacturing, assembly test, probe, final manufacturing, and we have a whole network of foundries and OSATs that we use as well.
So, we are a heterogeneously designed supply chain where we make sure our manufacturing flow matches the product requirements. So, if a certain technology is needed and we have that technology in house, we default to doing that technology in house.
If we don’t have it, we won’t necessarily go build a new factory to go get it if we can make a good business out of using a foundry ecosystem. So, it’s not even that we have a preference of one versus the other. It really comes down to we have a preference for building amazing products that are priced right with great quality and great features. Ultimately, the customer and the product decide where it’s going to be faster than how it is after.
Jorgensen: Is the memory shortage impacting customers’ ability to build full products or assemblies that eventually go into data centers?
McCarson: We utilize the same memory the rest of the ecosystem does. So, we’re not competing with large memory manufactures. But we do have some of our own proprietary memory products, but they tend to be highly specialized.
Jorgensen: Is that disrupting your supply chain at all or your ability to manufacture your own products?
McCarson: Nothing that’s unique to microchips. I think, you know, we’re seeing the same shortages others see and the same price implications others see as well. So yeah, nothing too crazy there. I think we’re the average in our in our impact compared to the rest of the market.
Jorgensen: Are there other manufacturing related materials and products that are becoming problematic in terms of your own manufacturing?
McCarson: You know, anything like 45 nanometer and below — right now in terms of the technology nodes for that manufacturing is constrained — and I expect that to continue. But right now, there’s so much hyper demand that pretty much everything from blocks to boards to substrates to assembly test capacity to throwers to testers to wafer fabs are all feeling tightly well.
What I am curious to see is, you know, the furious effort to try to build out capacity and build out data centers for meeting this ever-increasing demand. But at the same time, at the software side, there’s a huge amount of work in the ecosystem right now to try and reduce the number of resources required to do some of these AI workloads. At some point, I think there’s going to be a balance that happens where software efficiency becomes the most valuable commodity.
If I can run an AI workload more efficiently at the software level, then that reduces my need for hardware. That becomes a capex savings versus right now, everyone’s spending the capex so they can run the software and run the models. I think there may be something interesting there. So that’s kind of something I’m watching. Right now, the world is mostly focusing on how they get the same models to perform, but cheaper.
But then ultimately, I think even once you take Anthropic and Open AI off of the spending equation, you still have the massive data center. So, I’m starting to see more companies work on architectural innovations to try to bring memory and compute closer together, to reduce latency between transactions, to try to find lighter weight models that can run with less computational requirements.
Jorgensen: Is there anything we missed in our discussion on this topic?
McCarson: I would say I think we have left the era where people associate AI in the GPU. I think we’ve entered the era where people are realizing it takes a village of components to efficiently run a workload. And thanks to NVIDIA building such incredibly powerful GPUs, now the world is shifting towards making sure everything else is in that village of components. And so, I’m expecting to see sort of a Renaissance era of memory innovations, networking innovations, software innovations coming on the heels of this latest GPU-driven economy that we saw for the last almost like 8 years or so. And so it’s going to be an interesting and dynamic environment.