Delivering Synchronized Data for Integration Testing with Perforce Delphix [Demo]

Efficient integration testing is critical for accelerating application releases, but it often faces a significant bottleneck: providing consistent, compliant data across multiple environments. Teams struggle with slow, manual data provisioning processes that fail to deliver data synchronized in time across different databases and file systems. This leads to inaccurate testing, project delays, and an inability to conduct destructive tests without lengthy environment resets.

The Perforce Delphix DevOps Data Platform provides a transformative solution by automating the delivery of synchronized test data. In this comprehensive video demonstration, you will see how Delphix ingests data from various sources — including Oracle, SQL Server, and unstructured files — and creates lightweight, virtualized copies. These virtual databases (VDBs) are grouped together to represent a complete, time-aligned application environment. Watch as we use Jenkins to orchestrate an entire testing workflow, showcasing how Delphix empowers teams to provision, bookmark, and roll back complex environments in minutes.

Key Video Takeaways

This demo illustrates how Delphix streamlines complex testing workflows. You will learn how to:

• Provision synchronized environments: Deliver complete data environments with multiple data sources (Oracle, MSSQL, files) all aligned to a single point in time.
• Automate data operations: Leverage a rich set of APIs to integrate Delphix with CI/CD tools like Jenkins, enabling fully automated testing pipelines.
• Execute destructive tests confidently: Run tests that alter or destroy data, and then instantly roll the entire environment back to a previous state with a single operation.
• Accelerate testing cycles: Use self-service capabilities to create data bookmarks, refresh environments with the latest production data, and branch data for parallel development.
• Reduce infrastructure footprint: Utilize lightweight virtual databases that consume a fraction of the storage required by physical copies, significantly lowering costs.

See Delphix in Action

Ready to eliminate data bottlenecks and accelerate your integration testing? Request a custom demo to see how the Delphix DevOps Data Platform can transform your software development lifecycle.

Demo Delphix for Integration Testing

Full Transcript

My name is Carlos Cuellar. I'm a Principal Solutions Engineer for Perforce Delphix. Today, we're going to be talking about Delphix and how you can use synchronized data for integration testing.

First, let's do a quick recap about what Delphix is. Delphix is a DevOps data platform that will allow you to deliver production data to non-production environments in the form of lightweight copies that we call virtual databases.

Throughout this process, we are also going to give you the ability to discover where you have sensitive information and mask that data with realistic but fictitious data so you can still run and do any development or testing activities, with the added value that we're going to give you referential integrity across multiple data sources.

Once you have created your virtual databases, you're going to be able to deliver them to application teams, development teams, and testing teams. They will be able to consume that data. They will be able to take advantage of a self-service functionality that Delphix provides so they can refresh their own copies with the latest data from production, take backups, and go back in time and roll back to a specific point in time where they want to see data.

Now, let's talk a little bit about what virtualization is and how it works.

Delphix connects to where your data is — usually in your production environment. We rely on backups and ingest those backups. We have the ability to ingest backups from a specific database. We're going to compress those backups. This means that we're non-disruptive, and we're going to reduce the storage footprint with the added value that we're also going to have the ability to ingest incremental backups.

You will have different backups that have been ingested by Delphix that you can use to create copies as of the time that those backups were taken. If you took a backup on Monday, you took another one on Tuesday, and another one on Wednesday, you will have the ability to create copies as of how the data was on Monday, Tuesday, or Wednesday.

Then these lightweight copies that you're going to be able to create, that we call virtual databases, are going to be extremely small because Delphix, when it creates these databases, does not copy block-by-block. This will be very small, and it will give you the ability to execute their operations. This can be done because Delphix will create a timeline for your virtual databases. It will keep track of all the changes that are happening. Later on, you can refresh, you can take a backup, and then you can go back in time and see how the data was before.

The self-service capability will take advantage of these data operations for all virtual databases.

Now, with the added value that Delphix self-service does provide something that we call VDB Groups, which will allow you to not only execute these operations on a single virtual database; you can virtualize and create copies of multiple applications. In this example, you have a mobile application, CRM, credit application, and payment application. You will be able to ingest those applications into Delphix, create virtual copies from those applications, and then group them together into a single object that we call a VDB Group. That will allow you to create bookmarks and take backups of that application at one specific point in time, so those backups will be consistent across the board. You will be able to execute operations like refreshing with the latest data at one point in time and go back to a specific point in time for all of them.

This is going to be extremely powerful because it's not only giving you copies of your application or a single database. Now we can give you full copies of your environments with these data operations available for you.

In this demo, what we're going to be seeing is an example with two applications. We have these two applications that have been ingested in Delphix, and we have already created virtual copies from these two applications that we're going to use in the SIT environment.

It's a demo application that consists of one Oracle virtual database, and we have a CRM application that consists of a SQL Server virtual database. We also have a set of files that have been virtualized as well that we need for the application to work. We have these two applications, we have combined them into a single VDB Group, and now that's going to enable us to execute anything that we want in terms of data operations.

Delphix is an API-first application, so we can translate everything that we can see on the dashboard with APIs.

This is the workflow that we're going to follow in our demo today. First, we're going to create a bookmark, so it is effectively taking a backup for all of our objects that we're working with: the demo application and the CRM application. Then we will run our first test that will effectively add data across all three systems, so across both the demo and CRM applications. We will create a bookmark after running that first test, and this is going to be very important because we're going to use this data that we added.

Then, we're going to run our second test. This second test is going to be a destructive test, where we will be dropping tables in both SQL Server and Oracle and removing some files. We're going to create another bookmark after the second test. Then, we're going to roll back to how the data was right after the first test. The reason why is because we need that data that we added in that particular test to be able to execute our third test. We need those new records that we added to both our demo and CRM applications. We're going to need everything that we added so we can execute this third test, which will update and modify some values. Then once this is completed, we're going to create a final bookmark, which is another backup, and we are going to complete our process.

With that being said, now I'm going to be switching to Delphix, and I'll show you here. We have already configured our VDB Group that consists of all the objects that we're going to be using in this demo.

We have three VDBs. We have one that is for the demo application, which is an Oracle virtual database. Then we have our SQL Server virtual database and Files virtual object. These two are for the CRM application. We have effectively created virtual databases for all the systems that we need for our SIT environment.

Now with this, we will have the opportunity to execute our automated process. We have chosen Jenkins to orchestrate and execute everything through APIs. You can see how we can easily execute everything from any orchestrator by leveraging all the APIs that we provide.

Now I'm going to go to my pipeline, and I'm going to start executing this process. From here, we're going to be able to see how it moves from one phase to the other.

Every single step is going to be executing something here. This bookmark step is going to create a bookmark in Delphix. It will take a backup, and you will see it in the Delphix graphical user interface. This is where you see it. Here, we're going to see all the bookmarks that we are creating, all the backups that we need between every phase. Then we're going to have all the test use cases that are also going to show all the changes that we're doing across all of our virtual databases.

Okay, so now our first bookmark was completed. Then we moved forward and executed all the tests that we needed for the first test use case. Here we can see that we just added some files, we added a new table into our virtual database that we have for the CRM application, and finally, we added new records for our demo application. Here, we can showcase how we started with 950 records in one table and 30 in another, and then we added five and five.

After that, we are creating another bookmark. This is the one that we will reuse for our third test because this is effectively a backup after adding all these values, adding all these records to both our databases, and adding files to our virtual file system. That bookmark after the first test has been created.

Then we move forward to run our second test, which is a destructive test where we are removing files from our virtual file system, dropping a table to our virtual SQL Server database, and we're also removing and dropping a table in our virtual Oracle database. After that second test was completed, we are also creating a new bookmark, another backup after test number two.

Now that the bookmark we had to create after the second test has been created and is fully completed, we move forward to the rollback step where we are going back to present the data as it was after we ran our first test. That's the data state that we need to see before we run our third test. Here we are doing it to the single VDB Group that consists of both applications, the CRM plus the demo application, in this SIT environment.

Now our rollback operation has completed, so we got to the point where we can see the data that we need to execute our third test. Our third test is going to involve changing data in both the CRM application and the demo application that we have. For instance, here in our demo application, we are adding just a few records. We have recovered our table, and we updated some of those records as well. We can see here that we changed the Club ID for some of the players that we have added. For our CRM application, we also recovered the table that we needed for our application to work, and we added five new records. For the files that we need as well for the CRM application, we recovered those files and we added some lines to those files.

After we have successfully completed this third test, then we are creating a new bookmark, which is a backup, just to have a picture of how everything looked after running all of our tests. Our automated process has completed.

We went through three different test use cases that we ran, and we were able to provision the data that we needed — full copies of both the demo and CRM application into this SIT environment. We were able to take backups before we ran any test and present the data in the state that we needed. In the third test, we needed the data as it was right after the first test, and this is exactly what we did. This is what Delphix provided us.

If we go back here to the Delphix application, we can also confirm that this is our group, and here is where we have all the bookmarks that we created. Effectively, these are backups that we can later reuse to present the data and see how it was after every single test that we executed.

That's everything from me. Thank you very much.