Best Practices from Galactic Bricklayers
As an adage, “Rome wasn’t built in a day” is quite pervasive, despite often being halved for brevity. The omitted modifier – “They were laying bricks every hour” – is a sentiment not lost on the game developers at Cloud Imperium Games (CIG). That’s because CIG is building a digital environment every bit as cosmopolitan, mercantile, and eminent as the sprawling Italian capital.
In short, they're building a whole universe.
Star Citizen, the long-anticipated space simulation game, is scaling its development efforts to an unparalleled degree. Under the guidance of Chris Roberts, the esteemed game designer has helmed the growing development efforts since 2012, with a final vision that pairs nominally disparate game genres into one, including first-person shooter (FPS), massive multiplayer online (MMO), and co-operative campaigns.
It’s this pedigree and sheer scope that has beckoned supporters across the globe, enthusiastic to receive the final iteration, to donate more than $140 million toward development efforts. As the largest crowd-funded project in history, the ambitions of Star Citizen are directly correlated with the support provided by their financial backers.
“We do things different than normal game publishing,” says CIG Director of IT, Mike Jones, citing the influx of end-user monetary support, “We don’t answer to bankers.”
But with such support from the soon-to-be denizens of Star Citizen’s universe, there comes the burden of anticipated returns. Prospective gaming inhabitants have already purchased virtual ships and virtual accessories for in-game use, and CIG has worked diligently to provide playable game elements along the development cycle, lest the citizens revolt and commence the fall of Rome.
Ok, hyperbole aside, it’s true that CIG relies on the single source of truth in Perforce Helix to manage their workflows. Below are just a few ways that CIG brings their burgeoning galaxies to life, including speedy server architectures, efficient stream branches, scalable workspaces, and fine-grained security throughout.
Scale. Scale. Scale.
An ever-expanding universe is a tough horse to reign in. CIG’s distributed development teams operate from a main repository stream consisting of 338,000 files and nearly six terabytes of data. What’s more, the total sync of all Helix Ops streams total 23 million files and 90 terabytes data.
A non-optimal server system will buckle under a high degree of interaction and workflow, overloading server specs. That downtime interrupts development cycles and sends development benchmarks into disarray. But with Helix, server systems continue to provide the CIG network with an exceptional data rate.
So much so, that CIG upgraded studios to a gigabit fiber to accommodate the mass of data being moved. “We have a lot of changes coming in. The amount of activity on [the Helix] server is impressive,” says Jones.
The Helix Versioning Engine expertly supports CIG’s highly distributed and collaborative teams, too, and the processes that CIG DevOps teams rely on.
"You used to be in a box – and you didn't tell anybody what you're working on," says CIG's Network Services Manager, Paul Vaden, adding "We're very open in our development effort," citing waves of outreach, including multimedia demos and playable content of Star Citizen that has been continuously delivered to gaming fans throughout development.
"A lot of companies will wait; we put ours out as soon as it is ready," says Vaden, with CIG revealing their latest demo at Gamescom 2016 in August. "We tend to expose our work sooner than others."
This approach has afforded CIG the latitude to keep growing Star Citizen while simultaneously providing concurrent backers with immersive content in the lead up to their final deployment. This means developers can keep iterating persistent server environments, variable simulation categories, and over 100 unique star systems while crowd-source capital continues to climb.
Fine Grained Access Control
CIG is doing something so ambitious that the stakes are equally high. With teams spread across the world, relying on 24/7 work cycles and distributed personnel, Helix provides a secure platform during development. Helix fine-grained security controls access to data down to the file level, and as a result, projects are protected against unforeseen bugs and late-term glitches that can mitigate the success of a game release.
"We have developers all over the world, including many contractors. Perforce allows us to define very granular access to the game, resulting in faster turnaround and better control," says Vaden. "We limit a great deal of what developers have access to, otherwise copies of your game get out before it's ready."
This is a critical comfort, as stable access control is essential when content is released at various project milestones; for example, "Star Citizen had a Gamescom stream and we limited access to that stream because we couldn't afford to break our build stream last minute."
Helix gives build managers a single source of truth for who has what and where they have it. "I can see in one place who has access to which files– I don't have to jump around looking for them."
Branches with Brains
Securing financing and an immutable data security tool is essential to supporting your development projects. But Helix is there, too, when you commence the brass tacks of building your award-winning titles. Star Citizen is no exception.
Beyond asset clarity, Helix also offers bolstered operational efficiency for DevOps teams via development streams.
Proudly called “branches with brains,” Helix streams technology helps synchronize all the contributions from multiple offices yet allows different stakeholders to pare down the overall working set as needed via virtual streams.
Design artists -- less utilized in certain areas of software development yet essential in gaming – are able to lockdown their non-mergeable binary files. This feature prevents artists from editing files already in use by another artist, ensuring all teams avoid time/productivity loss, making Helix an unparalleled in gaming version control.
"Helix is eminently easier to administer, to setup," says Jones. "It supports the DevOps workflows required for us to deliver a better product."
The hefty development environments that CIG navigates are propped up firmly by Helix to ensure an optimal server architecture.
CIG development teams can execute day-to-day operations with exceeding confidence, relying on built-in backup components, ready at a moment's notice, to replace primary components without shutting the system down.
A variety of hot spares and cold standbys allow admins to safeguard against system failures, integrate vital components, and push scheduled data backups without service interruptions.
Stable enterprise scaling, development efficiencies, and an industry-specific set of platform features; these are just a few reasons Helix is an optimal solution. "It's pretty obvious why Perforce is so prevalent in the gaming industry," says Vaden.
Everything CIG does within the growing Star Citizen universe is supported by Helix, according to Jones: "If Perforce wasn't around, we'd be hard-pressed to find a better version control solution.”
Cloud Imperium Games is committed to Helix versioning over the long haul. That’s because, for a project already years in the making, developers need a stable, scalable versioning platform to reach the developmental finish line.
Add in fine-grained security features and better operational efficiency, and you have the first layer of benefits Helix supports.
To learn more about the innovative strategies CIG uses to buttress their expansive universe, check out our Game Development Best Practices eBook to glean game development insight from the industry’s most well-respected studios, including Epic Games, CD PROJEKT RED, and The Chinese Room.