Azure DevOps for Unreal Engine game development

Microsoft · December 11, 2023

In this article, we will cover Git features such as LFS and sparse checkout, as well as how to properly configure Unreal Engine so that multiple users can work on the same level at the same time. We will also show how to use Anchorpoint to make Git accessible to non-technical people like artists.

Prerequisites

You need access to an Azure account. You can either use an existing one or sign up for a free Azure account.
You need Unreal Engine and a proper content example, which you can get from Epic Games Store
Anchorpoint, as a Git GUI for non-technical people. It’s a paid service. If you don’t want to use it, you can also use the Git CMD, but make sure you have Git LFS installed.

What is so Special about Game Development?

Although the workflows are very similar to software development, game development has two additional challenges that software development does not.

Heavy Binary Files

Textures, 3D models, audio, etc. are all binary files that cannot be diffed and merged properly. It also makes no sense to diff them because many changes, like applying a filter to an audio clip, will change the entire file. Most of us know how well Git works with text files, and how many of the benefits of Git (branching, merging, text diffs) are not applicable to binaries.

It involves Artists

It involves non-technical people for whom it’s very hard to understand the concept of Git. They are visual people and cannot think in abstractions. This results in a lot of support work for the developer in the team, who instead of writing game logic, has to fix broken states in Git. To overcome these issues, we need to make heavy use of Git LFS to store binaries, file locking to prevent conflicts because binaries cannot be merged, and a simple Git desktop application that speaks the language of non-technical people.

Git LFS

Git Large File Storage is an extension that stores binary files in a dedicated space (usually on a block storage). In the Git repository itself, it stores a small pointer file that points to the original file. Unfortunately, it cannot be used out of the box, and some configuration is required. If you don’t want to use a GUI, you need to make sure that Git LFS is installed and configure your Git config. Git LFS is also capable of locking binary files to prevent conflicts.

A GUI

Artists should never work with the command line. They need a GUI. In this article, we will use Anchorpoint because it was made for non-technical people, has an Azure DevOps integration, and comes with a faster file locking system. It is a paid service. If you don’t want to use Anchorpoint, I will also show the proper Git commands to achieve the same effect.

Setting up the Project

Our Git repository will be stored on Azure DevOps because it scales pretty well. We uploaded commits of 75GB with no issues at all. To create projects in Azure DevOps, make sure you have an active organization.

Unreal Engine comes with a built-in Git plugin, which you definitely should not use. For years, it’s been marked in beta stage. All the Git commands will be done via Anchorpoint or the command line.

In Unreal Engine, we will be using a feature called World Partition. As mentioned above, you need to lock files to prevent conflicts. If we want to work on an open-world game, only one person can work on the level file because it’s locked. That is definitely slowing down the development process. World Partition splits the level file into small files so that locking the entire level is not necessary. Multiple people can work on the same level, and our commit size is smaller because these subfiles are way smaller than the whole level file.

Your project in Unreal Engine A game development project consists not only of the game engine, but also of the DCC (digital content creation) files created in Blender, Photoshop, Substance Designer, or any other content creation application. We want to put them into our Git repository as well, to have everything in one place.

The project structure that includes the assets, created in a DCC such as Blender

Connecting Anchorpoint and Azure DevOps

Open Anchorpoint and go to the projects page by creating a new tab. Scroll down to the list of Integrations and click on “Connect Application” and follow this procedure:

Choose the Azure DevOps integration and click on “Connect”.
Your web browser will open, and you have to allow Anchorpoint to work with Azure DevOps.

A popup in Anchorpoint will ask you to do three things:

Select your Azure DevOps organization. Anchorpoint can only create projects and add members in a specific organization.
Enter your Azure DevOps credentials. They are required so that Anchorpoint can push and pull from Azure DevOps repositories.
Check OAuth Policies. You need to enable “Third-party application access via OAuth” for your organization.

Connect Azure DevOps to Anchorpoint

Click on “Connect”. If you don’t see this integration, just restart the application.

You will get a consent screen which you have to accept

Follow the instructions of this popup.

Create the Git repository with Git LFS

In Anchorpoint, Git LFS will be configured automatically. You don’t need to do anything. Just go back to the projects page and click on “New project”. A wizard will start. On the first page, click on “Git Repository” and fill in this information:

Browse to the folder of your project, which includes the Unreal Engine files and the DCC files.
Under “Template for .gitignore”, select “Unreal Engine”.
Under Remote Settings, select “New Azure DevOps Repository”.

Click on “Continue” to enter the name of the project. In the last step, you will be able to invite members to your project. These members will also be invited to Azure DevOps. Make sure they also have an Azure DevOps account.

Click on the “New Project” button

Browse to the folder with your Unreal files and make sure the “Template for .gitignore” and “Remote Settings” are set up correctly. Then click “Continue”.

The last step is to invite the members you want to collaborate with. Leave this blank if you don’t want to collaborate with them. Each member must also have an Azure DevOps account.

Using the Command Line

If you want to use the command line, perform a normal Git initialization procedure and add the .gitignore and .gitattributes files in the root folder of your repository. We also need to add some lines to the Git config to improve its compatibility with Azure DevOps:

git config http.version HTTP/1.1
git config lfs.activitytimeout 600
git config core.longPaths 1

Push your files to Azure DevOps

In Anchorpoint, click on the “Timeline” and then on “Changed Files”. Simply enter a message and press “Push”. It will handle the staging and committing process under the hood. Your history is empty. Click “Changed Files” and add a meaningful message. Then hit the “Push” button and everything will be uploaded to DevOps.

Using the Command Line

When you are using the command line, you have to do these things manually.

Stage your files

git add .

Make a commit with the message “Initial commit”

git commit -m "Initial commit"

Create the remote branch (you only have to do this once)

git branch -M main

git remote add origin <repository-url>

Push the files to Azure DevOps

git push -u origin main

Working in Unreal Engine

When you modify files in Unreal Engine, you need to ensure that nobody else is working on these files to prevent conflicts. This is where file locking comes in. Anchorpoint locks files for others immediately after you modify them. It uses its own file locking system, which operates in real time.

You will see that Anchorpoint has detected the file change. Changed files are automatically locked for other members.

Git LFS File Locking

If you want to use the command line, you can still use the native Git LFS file locking. Keep in mind that the file is locked on the other side only after executing a Git fetch command. First of all, you need to set up your .gitattributes file and add the --lockable flag to the end of each file. For example:

.uasset filter=lfs diff=lfs merge=lfs -text --lockable

Locking and Unlocking Files

To lock a file in the command line, simply use this command:

git lfs lock <filename>

To unlock a file, use:

git lfs unlock <filename>

To see all files that are locked, use:

git lfs locks

To unlock files from other people who have forgotten to unlock them, use:

git lfs unlock <filename> --force

Participating as Another Member

Every member needs to have an Azure DevOps account and, if Anchorpoint is used, an Anchorpoint account. Once you have added members to a project in Anchorpoint, they will receive 2 emails. One from Azure DevOps, with an invitation to your organization, and one from Anchorpoint, with a download link to the application.

Using Git Sparse Checkout to Only Clone Particular Folders of the Git Repository

If you have artists for assets who are creating rocks, trees, buildings, etc., they may not need the engine folder with the Unreal Engine project. If you have developers, they most likely don’t need the DCC files. Git, by nature, downloads everything, but there is a way to avoid that.

Join the Git Repository and Pick the Folders You Want to Download in Anchorpoint

If other members open up Anchorpoint, they will be invited to your workspace. Then, they have to click on your project name and press the big blue button “Join Git repository”. Then they have to disable the checkbox “Download Everything”. Once they have joined the repository, they will see a folder tree of your project with little cloud icons. They just need to click on them and then press “Download”.

When you are invited to join an Anchorpoint project as another member, you will be asked to join the Git repository.

You will be able to download all project files from Azure DevOps. Make sure that “Download everything” is unchecked. If you are doing this for the first time, you will also need to connect Anchorpoint and DevOps by entering your DevOps credentials.

Select the folder you want to download. In this case, it’s the Engine folder. Anchorpoint performs a sparse checkout under the hood.

Sparse Checkout using the Command Line

In the command line, you have to use these commands. To checkout the repository without downloading any content of a subfolder, use:

git clone <repository-url> --sparse

To download the “engine” folder only, use:

git sparse-checkout add engine

This will download the engine folder, including all its subfolders.

Conclusion

Thanks to the monorepo movement, new features like sparse checkout in Git make it more scalable, and with Git LFS it can be used in game development. Especially for indie and AA development, it is a good choice because you benefit from the huge Git ecosystem.

The post Azure DevOps for Unreal Engine game development appeared first on Azure DevOps Blog.

View the full article

Sign In