Video
Back to top
Why Manual Test Data Processes are a Brick Wall for Agentic Development
Every tech leader is talking about agentic development and the promise of dramatically speeding up software delivery. But as organizations begin scaling AI-generated code, many are running into the same challenge: their test data processes can't keep up.
In this Q&A, Perforce Delphix experts Matt Yeh (Director of Product Marketing) and Brian Muskoff (VP of Product) discuss what they're seeing in the market. Brian talks through the 4 main ways he sees agentic development initiatives failing, why test data has become a critical bottleneck, and what App Dev leaders need to rethink if they want to successfully scale AI-driven software delivery.
Watch for Brian’s expert insights on:
- What companies are getting right about agentic development.
- Why the AI models aren't the problem.
- Where test data starts slowing things down.
- Four ways agentic development projects get stuck.
- What the bottleneck looks like in the real world.
- Building a stronger data foundation for AI.
- Why trusted, compliant data matters.
- Questions every App Dev leader should ask.
- The key takeaway for scaling agentic development.
Back to top
More Resources
See the key findings from our survey of 500+ enterprise leaders in our 2026 Test Data Management Report for AI-Ready Enterprises:
Explore Perforce Delphix test data management solutions for agentic software delivery:
Back to top
Full Transcript
Matt Yeh
I'm Matt Yeh. I'm a product marketer here at Delphix, and I'm here with Brian Muskoff, who runs our product organization.
At Delphix, we've been talking to a lot of application development leaders about agentic development, and there's, you know, a ton of excitement and pressure to adopt it. But, Brian, as you've shared throughout this last year, the future state of agentic development is not possible if you don't solve for one major bottleneck, and that, of course, is test data.
Brian Muskoff
Good to talk to you again, Matt. And that's absolutely right. It's amazing the progress we've made leveraging AI.
And agentic development's really enabling enterprises to just massively increase their code output, and it's surfacing outdated, manual, kind of unscalable test data processes as an even bigger roadblock. What we like to say in the DevOps world, it's creating even bigger bottlenecks than they are today.
Matt Yeh
Definitely. So, I wanted to sit down with you to discuss what you're seeing go wrong as organizations try to adopt agentic development, what they need to rethink in order to actually be successful with it. And to start, what do you think tech leaders are getting right about agentic development, and what are they actually underestimating? So kind of both things. What's some of the good, and what are some of the areas where they may be going astray?
Brian Muskoff
Sure. Yeah. It's always fun to talk about what's going right. We all often think about what's going wrong. So let's start there. You know, I think they're right to be excited about what's going on in this world of AI and agentic development.
There's not only a ton of potential to dramatically accelerate your software delivery. We're actually seeing real results right now. So a good indicator is GitHub. You know, it's been in the news.
You may have seen it. They're experiencing an order of magnitude explosion in the number of commits in the last couple of years. And so as an industry, we like to talk about hockey sticks, right, of growth. GitHub's actually experiencing that.
And, you know, what that means as an industry, we're kind of underestimating the pressure this places on traditional DevOps infrastructure and processes.
It's scaling so fast. We haven't really designed for that.
And so as many of the GitHub users know all too well, in fact, we use GitHub in our own development organization. There's been a number of GitHub platform outages throughout the last year, primarily just due to the usage demands. And so this extends beyond just source control. It goes up and down your DevOps tool chain. So every development team needs to start to prepare for these second order effects of AI on their own development environments.
Matt Yeh
You said that this is not really a problem with the AI models. What do you mean by that?
Brian Muskoff
Yeah. The models, you know, Anthropic, OpenAI, and so forth, they get all of the headlines. You know? There's a lot of interest at the model level.
And the reality is the models we have today are really good. And agentic development, for the most part, is not blocked. It's not our constraint right now, that they're not smart enough. The reality is that the models are getting real business results today, and we're seeing those results in our own organization.
We're hearing it from our customers. Development efforts that used to take quarters now take months or weeks. And it's just kind of shifting the focus into finding that new constraint. And, really, it's that the model output needs to be validated, secure, and compliant.
You know, if you 10x the output, you still have that same challenge of making sure that you've got high-quality, secure output. And this is what I call the brick wall. You know, we've streamlined that ability to generate code, but the rest of the process, the rest of the DevOps pipeline, is still very manual and ticket driven. And so it just highlights why these DevOps fundamentals have become more important than ever in the agentic world.
Matt Yeh
I wanna get back to this idea of a bottleneck or a brick wall. Like, how does test data specifically become the limiting factor in an agentic development workflow?
Brian Muskoff
Yeah. Good question. If you think about it, just speeding up the ability to generate code, reducing those cycle times from idea to commit, it puts downstream pressure on test environments to keep up. And so it's test environments. You know, best practice is to keep them production-like, compliant, and cost-effective. And the quality of the test results are only as good as the data that they test against.
And many teams have automated the code and infrastructure sides of the stack, but data is still handled through tickets, manual refreshes, manual masking steps, and approvals. And so if a pipeline can run in minutes, but the test data still takes days or weeks, the true delivery speed is still tied to your constraint, your bottleneck. And so you're gonna deliver in days and weeks even though your code commits are much faster.
Matt Yeh
Got it. Yeah. So test data becomes kind of the slow part of your stack.
What does this test data bottleneck kind of look like in the real world? Like, where is agentic development specifically getting stuck?
Brian Muskoff
Yeah. We see organizations getting stuck in at least four areas. The first one, we call provisioning drag. This is where an agent or a developer needs realistic data, but it takes days or even weeks for it to get provisioned, maybe by your IT team or your DBA team. The second one is what we call a compliance cliff. This is where agents are accessing sensitive data that they shouldn't have access to. And so compliance teams get in front of that and shut that work down.
The third, which is really top of mind with all this token spend, is cost overrun.
So agents may be spinning up new test environments, or you may have long-lived test environments. And these just multiply as agent demand drives more and more need for test data. And then the last one's around auditing and an audit black hole. Teams cannot show what data the agent saw, changed, or used, and we really need that visibility end to end to have confidence that we've got the security that we need.
Matt Yeh
In the past, I've heard you invoke this idea of having, you know, a data foundation, a trusted data foundation, if you will. What does a better data foundation for agentic development actually look like? What are some of the components that you kind of see for this as being requirements?
Brian Muskoff
Yeah. There's definitely best practices emerging here. The first one is what I would describe as self-service. So the agents need to have the ability to request data and get to that data without filing tickets. You know, how do we get them kind of preapproved access on demand?
The second one is around virtualized and ephemeral environments. You know, the long-lived, the days of long-lived test environments are over. We need to reduce the infrastructure that is static or standing and find ways to make it ephemeral. It just lives for the time that you need it to execute the test.
Another best practice is around compliance of production-like, trusted data. And so what we found is the best test data is production-like data, but it needs to be trusted. It needs to be realistic, but also desensitized. We can't have PII making its way downstream into these test environments.
And then, you know, kind of tying it back to the issues we talked about earlier, we need to make sure this is all auditable and that changes in data are versioned like code. This gives teams the ability to reproduce what an agent saw and changed, and it gives the auditors confidence that we're staying kind of within the policy guidelines.
Matt Yeh
Yeah. It's interesting. So this idea of compliant and secure and yet also realistic or production-like data aligns to what we found in our twenty twenty six test data management report that we published recently. Data quality was actually the number one test data challenge that respondents, or leaders, reported there.
Brian Muskoff
Yeah. I love that report. And it's a tough balance. It's a tough mandate. You need both compliance and high-quality data to be successful.
And that can only be achieved by having the right process and the right tools.
Matt Yeh
What are some practical questions Application Development and DevOps leaders should ask their teams to assess whether test data is gonna be a blocker for agentic development?
Brian Muskoff
Yeah. I think a simple way to think about it is through a thought experiment. Just ask yourself, if we can generate code five times faster tomorrow, can our test data environments keep up?
Or more specifically, how long does it take to provision or refresh test data? Are developers waiting on infrastructure or database teams?
If you push on these questions and you're not getting answers that you feel good about, it may tell you, hey. This is an area that we should spend more focus on. And technology leadership should also ask whether sensitive data is protected consistently in production and if the organization has visibility into nonproduction data usage and risks.
Matt Yeh
Yeah. So it sounds like the takeaway question for leaders is, you know, is your data foundation ready? Is it trusted?
You know, is it ready for this kind of new wave of agentic development that is changing so much about how teams build, release, and ship new features and applications? Right?
Brian Muskoff
That's exactly right. Yeah. The development teams that get ahead of these problems, they solve for test data automation, data compliance, and data governance practices now, they're the ones that are gonna be best positioned to scale agentic development in the years ahead. You know, we may be seeing 5x or 10x now. It's not unrealistic to think that we could get another order of magnitude on top of that. And in these cases, the human role is really shifting to setting goals, orchestrating agents, and governing outcomes. And, ultimately, they still need to live in the world of delivering real constraints that require good testing, security practices, compliance, and production readiness, but it can be done.