A code-first approach to versioning Octopus Deploy

When using Octopus Deploy as your deployment server, you've probably used its user interface to update its configuration in step with the requirements of the code which is being deployed. Hopefully you've also configured scheduled SQL backups so that all your hard work is saved in case of disaster.

Occasionally, however, you might find yourself in a scenario where you really want to roll back Octopus configuration to a particular past version - for instance, an application deployment which didn't go well and needed to be rolled back (for which you had to make Octopus changes). Trawling through database backups might be a painful way of doing this. Isn't there just a way of snapshotting Octopus into a source control repository somewhere?

Octopus Deploy logo

Code-first

As you can see by reading this, a lot of people have been asking the same question, as right now it's not possible to simply "output" the state of Octopus as e.g. JSON or XML. As Octopus is an "API first" tool (all the stuff you are doing via the user interface can also be done via the Octopus REST API) it is recommended that you instead use an "IaC" approach to code-up your configuration first, then use some mechanism to push these changes into Octopus. As C# is my hammer of choice, I used the Octopus.Client library wrapped up in a simple console application. When the time comes to push the code changes into Octopus, I have a TeamCity project which will pull the desired Octopus configuration branch from GitHub and execute the console application.

Downsides

Ideally you want your code->Octopus process to be idempotent, i.e. you can run your changes multiple times without ending up with duplicated variables, process steps, etc. In practise this means a lot of boilerplate code to "tear down" your existing Octopus configuration before building it back up again. The article I mentioned earlier proposes an "Octopus.Client.Declarative" library which sits on top of Octopus.Client and allows you to "declare" the desired state of Octopus, rather than having to imperatively tear down and build up again. Unfortunately this library doesn't exist yet.

Just worry about the things that change

To make this approach pragmatic, rather than attempting to code up *everything*, I've just coded up the things that tend to change frequently - non-sensitive variables, library packages and deployment process steps. I've made the assumption that everything else is largely stable and already there (project, channel, lifecycle, environments, roles, deployment targets, sensitive variables, community step templates.)

Code samples

There are some good samples here, but not as many samples of C# Octopus.Client as I expected, which meant a bit of trial and error. I've put some of my samples here to hopefully save you some time :)

Setting up variables:

using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Octopus.Client;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    class SetUpVariables
    {
        private static readonly IList<string> SensitiveVariableNames = new List<string>
            {
                "MySensitiveVariableName1",
                "MySensitiveVariableName2"
            };

        internal static async Task Do(IOctopusAsyncClient client, string variableSetId)
        {
            var variableSet = await client.Repository.VariableSets.Get(variableSetId);

            ClearExistingNonSensitiveVariables(variableSet);

            await AddNonSensitiveVariables(client, variableSet);

            await client.Repository.VariableSets.Modify(variableSet);
        }

        private static void ClearExistingNonSensitiveVariables(VariableSetResource variableSet)
        {
            var nonSensitiveVariables = variableSet.Variables.Where(v => !SensitiveVariableNames.Contains(v.Name)).ToList();
            foreach (var nonSensitiveVariable in nonSensitiveVariables)
            {
                variableSet.Variables.Remove(nonSensitiveVariable);
            }
        }

        private static async Task AddNonSensitiveVariables(IOctopusAsyncClient client, VariableSetResource variableSet)
        {
            var environment1Id = (await client.Repository.Environments.FindByName("Environment1")).Id;
            var environment2Id = (await client.Repository.Environments.FindByName("Environment2")).Id;

            variableSet.AddOrUpdateVariableValue("MyNonSensitiveVariableName", "MyNonSensitiveVariableValue",
                new ScopeSpecification {{ScopeField.Environment, new ScopeValue(environment1Id, environment2Id) }});
        }
    }
}

Setting up library packages:

using System;
using System.Threading.Tasks;
using Octopus.Client;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    class SetUpLibrary
    {
        internal static async Task Do(IOctopusAsyncClient client, string octopusUrl, string apiKey)
        {
            await EnsureLibraryPackage(client, octopusUrl, apiKey, "MyLibraryPackageName", "MyLibraryPackageName.nupkg");

        }

        private static async Task EnsureLibraryPackage(IOctopusAsyncClient client, string octopusUrl, string apiKey, string packageName, string fileName)
        {
            ResourceCollection<PackageFromBuiltInFeedResource> packages = await client.Repository.BuiltInPackageRepository.ListPackages(packageName);
            foreach (var package in packages.Items)
            {
                await client.Repository.BuiltInPackageRepository.DeletePackage(package);
            }
            Console.WriteLine($"Cleared all existing {packageName} library packages");

            var filePath = AppDomain.CurrentDomain.BaseDirectory + "\\LibraryPackageFiles\\" + fileName;
            Console.WriteLine($"Starting to push library package {fileName}");
            PackagePusher.Do(octopusUrl, apiKey, filePath);
            Console.WriteLine($"Finished pushing library package {fileName}");
        }
    }
}

Note that I had some issues with the Octopus.Client library package pushing methods, so I had to fall back to calling the REST API directly:

using System;
using System.IO;
using System.Net;
using System.Text;

namespace OctopusConfiguration
{
    // Adapted from https://github.com/OctopusDeploy/OctopusDeploy-Api/blob/master/Octopus.Client/LINQPad/Push%20Package%20to%20Built-In%20Repository.linq
    static class PackagePusher
    {
        private const int MillisecondsPerSecond = 1000;
        private const int SecondsPerMinute = 60;
        private const int TimeoutInMinutes = 2;

        internal static void Do(string octopusUrl, string apiKey, string packageFilePath)
        {
            var packageUrl = octopusUrl + "/api/packages/raw?replace=false";

            var webRequest = (HttpWebRequest) WebRequest.Create(packageUrl);
            webRequest.Accept = "application/json";
            webRequest.ContentType = "application/json";
            webRequest.Method = "POST";
            webRequest.Headers["X-Octopus-ApiKey"] = apiKey;
            webRequest.Timeout = TimeoutInMinutes * SecondsPerMinute * MillisecondsPerSecond;

            using (var packageFileStream = new FileStream(packageFilePath, FileMode.Open))
            {
                var requestStream = webRequest.GetRequestStream();

                var boundary = "----------------------------" + DateTime.Now.Ticks.ToString("x");
                var boundarybytes = Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");
                webRequest.ContentType = "multipart/form-data; boundary=" + boundary;
                requestStream.Write(boundarybytes, 0, boundarybytes.Length);

                var headerTemplate =
                    "Content-Disposition: form-data; filename=\"{0}\"\r\nContent-Type: application/octet-stream\r\n\r\n";
                var header = string.Format(headerTemplate, Path.GetFileName(packageFilePath));
                var headerbytes = Encoding.UTF8.GetBytes(header);
                requestStream.Write(headerbytes, 0, headerbytes.Length);
                packageFileStream.CopyTo(requestStream);
                requestStream.Write(boundarybytes, 0, boundarybytes.Length);
                requestStream.Flush();
                requestStream.Close();
            }

            var response = webRequest.GetResponse();
            response.Close();
        }
    }
}

Setting up deployment process steps:

using System;
using System.Threading.Tasks;
using Octopus.Client;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    class SetUpDeploymentProcesses
    {
        internal static async Task Do(IOctopusAsyncClient client, ProjectResource project)
        {
            var deploymentProcesses = await client.Repository.DeploymentProcesses.Get(project.DeploymentProcessId);
            var iisStopCommunityStep = await client.Repository.ActionTemplates.FindByName("IIS AppPool - Stop");
            var iisStartCommunityStep = await client.Repository.ActionTemplates.FindByName("IIS AppPool - Start");

            ClearExistingDeploymentSteps(deploymentProcesses);

            deploymentProcesses.Steps.Add(SetUpPowershellScriptDeploymentProcesses.CreateExamplePowershellDeploymentStep());
            deploymentProcesses.Steps.Add(SetUpIISAppPoolDeploymentProcesses.CreateStopIISAppPoolDeploymentStep(iisStopCommunityStep));
            deploymentProcesses.Steps.Add(SetUpPackageDeploymentProcesses.CreateExamplePackageDeploymentStep());
            deploymentProcesses.Steps.Add(SetUpIISAppPoolDeploymentProcesses.CreateStartIISAppPoolDeploymentStep(iisStartCommunityStep));

            await client.Repository.DeploymentProcesses.Modify(deploymentProcesses);
            Console.WriteLine("Finished adding deployment process steps");
        }
      
        private static void ClearExistingDeploymentSteps(DeploymentProcessResource deploymentProcesses)
        {
            deploymentProcesses.Steps.Clear();
            Console.WriteLine("Cleared all existing deployment process steps");
        }
     
    }
}
using System.Collections.Generic;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    class SetUpPowershellScriptDeploymentProcesses
    {    
        internal static DeploymentStepResource CreateExamplePowershellDeploymentStep()
        {
            return CreatePowerShellScriptDeploymentStep("Example Powershell deployment step", "Write-Host \"Hello, World!\"",
                "Role1,Role2");          
        }

        private static DeploymentStepResource CreatePowerShellScriptDeploymentStep(string stepName, string script, string roles)
        {
            return new DeploymentStepResource
            {

                Name = stepName,
                Condition = DeploymentStepCondition.Success,
                StartTrigger = DeploymentStepStartTrigger.StartAfterPrevious,
                Actions = { new DeploymentActionResource
                {
                    Name = stepName,
                    ActionType = "Octopus.Script",
                    Properties =
                    {
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.RunOnServer",
                            new PropertyValueResource("false")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.Syntax",
                            new PropertyValueResource("Powershell")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.ScriptSource",
                            new PropertyValueResource("Inline")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.ScriptBody",
                            new PropertyValueResource(script))
                    }
                } },
                Properties =
                {
                    new KeyValuePair<string, PropertyValueResource>("Octopus.Action.TargetRoles", roles)
                }
            };
        }    
    }
}
using System.Collections.Generic;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    // ReSharper disable once InconsistentNaming
    class SetUpIISAppPoolDeploymentProcesses
    {
        // ReSharper disable once InconsistentNaming
        internal static DeploymentStepResource CreateStopIISAppPoolDeploymentStep(ActionTemplateResource actionTemplateResource)
        {
            return new DeploymentStepResource
            {
                Name = "IIS AppPool - Stop",
                Condition = DeploymentStepCondition.Success,
                StartTrigger = DeploymentStepStartTrigger.StartAfterPrevious,
                Actions = { new DeploymentActionResource
                {
                    Name = "IIS AppPool - Stop",
                    ActionType = "Octopus.Script",
                    Properties =
                    {
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.ScriptBody",
                            new PropertyValueResource(actionTemplateResource.Properties["Octopus.Action.Script.ScriptBody"].Value)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.Syntax",
                            new PropertyValueResource("PowerShell")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Template.Id",
                            new PropertyValueResource(actionTemplateResource.Id)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Template.Version",
                            new PropertyValueResource(actionTemplateResource.Version.ToString())),
                        new KeyValuePair<string, PropertyValueResource>("AppPoolCheckDelay",
                            new PropertyValueResource("10000")),
                        new KeyValuePair<string, PropertyValueResource>("AppPoolCheckRetries",
                            new PropertyValueResource("20")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.RunOnServer",
                            new PropertyValueResource("false")),
                        new KeyValuePair<string, PropertyValueResource>("AppPoolName",
                            new PropertyValueResource("#{ApplicationPoolNameVariable}"))
                    }
                } },
                Properties =
                {
                    new KeyValuePair<string, PropertyValueResource>("Octopus.Action.TargetRoles", "Role1,Role2")
                }
            };
        }

        // ReSharper disable once InconsistentNaming
        internal static DeploymentStepResource CreateStartIISAppPoolDeploymentStep(ActionTemplateResource actionTemplateResource)
        {
            return new DeploymentStepResource
            {
                Name = "IIS AppPool - Start",
                Condition = DeploymentStepCondition.Success,
                StartTrigger = DeploymentStepStartTrigger.StartAfterPrevious,
                Actions = { new DeploymentActionResource
                {
                    Name = "IIS AppPool - Start",
                    ActionType = "Octopus.Script",
                    Properties =
                    {
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.ScriptBody",
                            new PropertyValueResource(actionTemplateResource.Properties["Octopus.Action.Script.ScriptBody"].Value)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Script.Syntax",
                            new PropertyValueResource("PowerShell")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Template.Id",
                            new PropertyValueResource(actionTemplateResource.Id)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Template.Version",
                            new PropertyValueResource(actionTemplateResource.Version.ToString())),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.RunOnServer",
                            new PropertyValueResource("false")),
                        new KeyValuePair<string, PropertyValueResource>("AppPoolName",
                            new PropertyValueResource("#{ApplicationPoolNameVariable}"))
                    }
                } },
                Properties =
                {
                    new KeyValuePair<string, PropertyValueResource>("Octopus.Action.TargetRoles", "Role1,Role2")
                }
            };
        }
    }
}
using System.Collections.Generic;
using Octopus.Client.Model;

namespace OctopusConfiguration
{
    class SetUpPackageDeploymentProcesses
    {
        internal static DeploymentStepResource CreateExamplePackageDeploymentStep()
        {
            return CreateDeployPackageDeploymentStep("Example package deploy", "MyPackageName", "#{WebsiteDirectoryVariable}",
                true, "Octopus.Features.CustomDirectory", false, false, string.Empty, "Role1,Role2");
        }
      
        private static DeploymentStepResource CreateDeployPackageDeploymentStep(string stepName, string packageId, string installDirectory,
            bool purgeDirectory, 
            string enabledFeatures, bool runConfigTransforms, bool substituteVariables, string variableSubstituteFiles,
            string targetRoles)
        {
            return new DeploymentStepResource
            {
                Name = stepName,
                Condition = DeploymentStepCondition.Success,
                StartTrigger = DeploymentStepStartTrigger.StartAfterPrevious,
                Actions = { new DeploymentActionResource
                {
                    Name = stepName,
                    ActionType = "Octopus.TentaclePackage",
                    Properties =
                    {
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.EnabledFeatures", new PropertyValueResource(enabledFeatures)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.DownloadOnTentacle", new PropertyValueResource("False")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.FeedId", new PropertyValueResource("feeds-builtin")),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.PackageId", new PropertyValueResource(packageId)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.CustomInstallationDirectory", new PropertyValueResource(installDirectory)),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.CustomInstallationDirectoryShouldBePurgedBeforeDeployment", new PropertyValueResource(purgeDirectory.ToString())),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.Package.AutomaticallyRunConfigurationTransformationFiles", new PropertyValueResource(runConfigTransforms.ToString())),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.SubstituteInFiles.Enabled", new PropertyValueResource(substituteVariables.ToString())),
                        new KeyValuePair<string, PropertyValueResource>("Octopus.Action.SubstituteInFiles.TargetFiles", new PropertyValueResource(variableSubstituteFiles))
                    }
                } },
                Properties =
                {
                    new KeyValuePair<string, PropertyValueResource>("Octopus.Action.TargetRoles", targetRoles)
                }
            };
        }
    }
}

Happy Octopussing!


By James at 13 Aug 2018, 15:27 PM


Comments

Post a comment