Continuous Deployment of Unity Games to Steam

Continuous Deployment

Automated deployment has long become standard in software development, yet its use in game development is still rare. In particular, automation via continuous integration processes is now indispensable for most software projects. First, a simple summary of what Continuous Delivery is all about. While CI is about automating builds, tests and source code merging, continuous deployment is about automating all the way to releases in the production environment. In the following we will give you a quick overview of how we implemented a continuous deployment process for our game Panic Mode. Panic Mode is developed using the Unity Engine and is released on the steam platform.

Automating the build

Before any deployment processes can be automated, the first requirement is to have a automatic build of the game available. We experimented with a few options regarding building the game. In the end we decided to go with the Unity Cloud Build Service, which allows access to an out-of-the-box solution provided by Unity itself. In addition to offering a easy setup for simple CI pipelines, it also provides a full-fledged REST-API which can be used for more complex integrations. Other options would have been dedicated Build-Servers or using Docker-Images. The Cloud Build Service offers the possibility to automatically start a build on commits in preconfigured branches, but this is limited to statically defined branches. Because we are making use of feature branches, this is not sufficient for our requirements.

The solution in this case is the provided REST-API, which allows us to dynamically configure Build-Targets in the service. In this case using the API to trigger the builds also gives us full control over the build-lifecycle. To achieve this, a script was written that takes care of configuring a build-target, triggering a new build, monitoring its state until finally a completed build artifact can be fetched. A small snippet of the script can be seen below, which takes care of starting the new build on a previously configured build target. Once the build is done the complete artifact is fetched through the API and made available for following CI jobs. In case of a release, we are also making use of parallel CI jobs to start builds for all our target platforms (Linux, OSX, Windows), which later can be aggregated to a release. Because we are making use of the given REST-API, we can control anything from Target-Platforms to special flags for showcase builds through our own Gitlab pipeline.

        
      
uri = URI . parse (" https :// build -api. cloud . unity3d .com/api/v1/ orgs /"
+ ORG_ID + "/ projects /" + PROJECT_ID
+ "/ buildtargets /" + buildTargetId
+ "/ builds ")
request = Net :: HTTP :: Post . new(uri)
request . content_type = " application / json "
request [" Authorization "] = " Basic " + AUTH_KEY
request . body = JSON . dump ({
" clean " = > true ,
" delay " = > 30
})
req_options = {
use_ssl : uri . scheme == " https ",
}
response = Net :: HTTP . start ( uri. hostname , uri.port , req_options ) do | http |
http . request ( request )
end

Automating the release

Once the gitlab pipeline is configured to automatically create builds through the cloud build service, we have everything we need to create new releases. Instructions on how to set up the steam releases without automation can be found in the official documentation from steam. In our case the automation again is done by yet another gitlab job, which can be seen below. This job is based on the cm2network/steamcmd:root docker image, which gives access to the SteamCMD binary and enables connecting to steam servers to create and upload new builds. On this container it is then possible to execute our deployment script, which takes care of multiple things:

Extracts all compressed builds from previous build jobs
Extract a fitting version-name either from tag or commit hash
Replace a bunch of placeholders in the .vdf files required for steam deployments
Attempt to log into the steamCmd Interface
Fetch a code for steam guard verification
Actually log in and execute the run_app_build steam command

One of the most difficult steps in achieving this automation, is in getting your docker container authenticated for accessing steam. This issue lies in steam guard being required for company accounts of steam. Because each docker container is a completely fresh system, steam thinks it’s a new system each time. To work around this issue, we created a simple tool that uses a IMAP reader in GO to parse through our emails and fetch the required steam guard code. This code is automatically send once a login attempt from a new machine is done and can be passed to the command line interface.

The rest of the script can also be seen below, most of which is taking care of different placeholders in the build-files. By making use of a lot of placeholders, which can be configured either through global variables for the gitlab project or uniquely for each job, it is easily possible to target different steam releases with the same script. The full release script as well as an example for the app_build.vdf required to create the build can be seen below.

We decided to make the tool to fetch the code from the email public, feel free to check it out on github.

The Regex should look something like this:
account accountNameHere:(?:[\s]+)([\S]+)(?:[\s]+)

Gitlab Job:

.steam_release_template: &steamReleaseTemplate
  image: cm2network/steamcmd:root
  # steamcmd/steamcmd is not used because it does not provide bash
  stage: release
  script:
    - chmod +x ci/steamDeploy.sh
    - ./ci/steamDeploy.sh
  variables:
    STEAM_BRANCH_NAME: ""  # Default to not set build live
  artifacts:
    name: 'SteamArtifacts'
    when: always
    expire_in: 7 day
    paths:
      # Upload the updated .vdf files as well as all manifest files generated
      - "ci/steam/*"

Release Script:

        
      
echo "Starting steam deployment"

# For easier debugging, print everything being executed
set -x

# These directories are used by the build script, see ci/steam/*.vdf
initialDir=$(pwd)

BUILD_DIR="${initialDir}/builds"
BUILD_LOG_DIR="${initialDir}/buildoutput"
mkdir -p "$BUILD_DIR"
mkdir -p "$BUILD_LOG_DIR"


buildFile="${initialDir}/ci/steam/app_build.vdf"

# Extract the cloud builds from the previous jobs
apt-get update
apt-get -y install unzip

for f in $(find . -type f -iname "*_build.zip")
do
	fileName=${f//"./"/}
	filePath=${fileName//.zip/}
	filePath="${BUILD_DIR}/${filePath}/PanicMode"
	echo "Extracting build from $fileName to $filePath"
	mkdir -p "${filePath}"
	unzip "$fileName" -d "$filePath"
	# We dont need to send everyone our test-results :)
	rm -rf "${filePath:?}/Editor"
done

# In addition to checking cloudbuild artifacts, also check dockerbuild artifacts
if [ -d "${initialDir}/Build" ] ; then
  cd "${initialDir}/Build" || exit
  for f in $(find . -mindepth 1 -maxdepth 1 -type d -iname "*")
  do
    fileName=${f//"./"/}
    srcPath="${initialDir}/Build/${fileName}"
    dstPath="${BUILD_DIR}/${fileName}_build/PanicMode"
    echo "Copying build from $srcPath to $dstPath"
    mkdir -p "${dstPath}"
    cp -r "$srcPath/." "$dstPath"
  done
fi


# Fetch the variables to set for the current environment
versionName=${CI_COMMIT_TAG}
if [ -z "$versionName" ]
then
      versionName=$CI_COMMIT_SHA
else
      versionName=${versionName//"version/"/}
fi



# Replace some placeholders with values for the current environment
# This allows us to use absolute paths which are more resilient
echo "Updating <BUILD_NAME> to '${versionName}'"
echo "Updating <STEAM_BRANCH> to '${STEAM_BRANCH_NAME}'"
echo "Updating <BUILD_DIR> to '${BUILD_DIR}'"
# TODO: Settings this somehow does not work, and even fails the build completley without a message
# so instead for now logs are printed in the ci/steam directory, and uploaded with the vds as proof up release
echo "Updating <BUILD_LOG_DIR> to '${BUILD_LOG_DIR}'"

echo "Setting up Steam APP ID ${STEAM_APP_ID} with depots: ${STEAM_WINDOWS_DEPOT_ID} (Windows), ${STEAM_MAC_DEPOT_ID} (Mac), ${STEAM_LINUX_DEPOT_ID} (Linux)"

# Steam target config (For exmaple demo vs usual branch)

BUILD_DIR="${initialDir}/builds"
BUILD_LOG_DIR="${initialDir}/buildoutput"

cd "${initialDir}/ci/steam/" || exit
for f in $(find . -type f -iname "*.vdf")
do
	fileName=${f//"./"/}
	echo "Updating placeholders in file '${fileName}'"
	sed -i  "s|<BUILD_NAME>|${versionName}|" "${fileName}"
  sed -i  "s|<STEAM_BRANCH>|${STEAM_BRANCH_NAME}|" "${fileName}"
  sed -i  "s|<BUILD_DIR>|${BUILD_DIR}|" "${fileName}"
  sed -i  "s|<BUILD_LOG_DIR>|${BUILD_LOG_DIR}|" "${fileName}"
  sed -i  "s|<APP_ID>|${STEAM_APP_ID}|" "${fileName}"
  sed -i  "s|<WINDOWS_DEPOT_ID>|${STEAM_WINDOWS_DEPOT_ID}|" "${fileName}"
  sed -i  "s|<MAC_DEPOT_ID>|${STEAM_MAC_DEPOT_ID}|" "${fileName}"
  sed -i  "s|<LINUX_DEPOT_ID>|${STEAM_LINUX_DEPOT_ID}|" "${fileName}"
done


# https://hub.docker.com/r/cm2network/steamcmd/ Owner ship of steamcmd.sh command is for this user

cd /home/steam/steamcmd || exit
su - steam -c "cd /home/steam/steamcmd && ./steamcmd.sh +login '${STEAM_ACCOUNT?}' '${STEAM_PASSWORD?}' +quit" || STEAM_EXIT=$?
echo "Completed steam login with result $STEAM_EXIT"
cd "$initialDir" || exit

STEAM_GUARD_CODE=""
if [ $STEAM_EXIT -eq 5 ]; then
  echo "Login without steam guard code failed, fetching code..."
  # Wait 3 minutes to make sure the email is actually received in time
  chmod +x ci/imap-extractor
  sleep 180
  STEAM_GUARD_CODE=$(./ci/imap-extractor "ci/imap-config.json")
  echo "Fetched steam guard code '$STEAM_GUARD_CODE'"
fi

echo "Attempt doing steam build with file $buildFile"

STEAM_EXIT=0
cd /home/steam/steamcmd || exit
su - steam -c "cd /home/steam/steamcmd && ./steamcmd.sh +login '${STEAM_ACCOUNT?}' '${STEAM_PASSWORD?}' ${STEAM_GUARD_CODE} +run_app_build '${buildFile}' +quit" || STEAM_EXIT=$?
echo "Completed steam command with $STEAM_EXIT"
cd "$initialDir" || exit
exit $STEAM_EXIT

app_build.vdf with place-holders:

"appbuild"
{
	"appid"	"<APP_ID>"
	"desc" "Build <BUILD_NAME>" // description for this build
	"buildoutput" "." // build output folder for .log, .csm & .csd files, relative to location of this file
	"contentroot" "<BUILD_DIR>" // root content folder, relative to location of this file
	"setlive" "<STEAM_BRANCH>" // branch to set live after successful build, non if empty
	"preview" "0" // to enable preview builds
	"local"	""	// set to file path of local content server

	"depots"
	{
		"<WINDOWS_DEPOT_ID>" "depot_windows.vdf"
		"<MAC_DEPOT_ID>" "depot_mac.vdf"
		"<LINUX_DEPOT_ID>" "depot_linux.vdf"
	}
}

Benefits

Through the setup explained above we are able to fully automate the process of building and pushing our game to the steam servers. Although updating the live build is not possible with this, because steam restricts API accesses to that branch. However, it is possible to automatically update development branches on steam with this workflow. Updating the live branch is then only a matter of a few clicks.

For panic mode, this enables us to do two things:

We configure an automated nightly build for internal testing purposes
We configured a preview branch, which game developers can update with new features manually

Both of these features allow us to quickly integrate changes into the deployed game and quickly access and test changes on all of our supported platforms. This means no installation of any build tools is required to create a release, and every team member is able to do so easily. Automated builds are also more consistent and don’t require special handling of any operating systems, as the cloud build service already handles those.

Make sure to check out our latest automated release on steam! You can buy the game in early access on steam and be sure to look out for our next major update releasing in the coming months.

Continuous Deployment of Unity Games to Steam