Create Golden Paths for your development teams with Datadog App Builder

Many organizations are working to enhance the developer experience for teams maintaining highly complex architectures and platforms supported by intricate internal processes. Platform engineering for Golden Paths seeks to address this challenge by providing self-service tools, capabilities, and processes to help engineers start new projects in a more standardized, less mistake-prone way.

Project scaffolding tools like nunjucks support a user-friendly approach for templatizing project creation to help you create Golden Paths. You can use Datadog App Builder in conjunction with nunjucks to create an extensible, self-service UI for spinning up projects from Golden Path templates.

In this post, we’ll discuss some of the key advantages of Golden Path project scaffolding, explore how to use Datadog App Builder to create a self-service UI for scaffolding new projects, and show how you can use App Builder to easily make projects like this one discoverable across your organization.

Simplify project setup by defining Golden Paths

The complexity of the DevOps technical ecosystem and its plethora of deployment patterns and requirements can introduce friction for development teams. When developers own infrastructure, monitoring, and operations logic—in addition to application code—using templated infrastructure and configurations can help them create new projects more efficiently. This kind of standardized project scaffolding can help distributed teams collapse knowledge silos, streamline onboarding, and make it easier to get services working together. Golden Path templates offer a framework for organizations to scaffold projects by maintaining prepackaged compositions of well-integrated code and infrastructure capabilities. Golden Path templates typically provide:

• A repository template that engineers can quickly branch off of and get all the key resources and configurations they need
• A pipeline that can build and deploy the templated repo, including all deployment manifests (such as Helm charts, Kustomize files, etc.)
• Default observability settings for traces, logs, custom metrics, and more

By defining best practices for project initialization and making it easy for developers to follow them, Golden Paths promote development velocity, standardization, and process optimization. You can help engineers at your org adopt Golden Paths by offering a workflow for creating them in your organization’s internal developer portal (IDP), alongside the software catalog and other DevEx resources.

Next, we’ll walk through how you can use Datadog App Builder in conjunction with your project templates to create your own Golden Paths tool.

Create project templates with Datadog App Builder

By using Datadog App Builder, you can automate the process of branching off of an existing template to create a new project for your engineers. Let’s create an App that implements the following steps:

• Download a project template from GitHub
• Generate new project files from the template
• Deploy the completed project back into GitHub, ready for development

To bootstrap this App, we’ll use nunjucks, a popular open source library for project templating. You can use nunjucks to create Golden Paths that include key template assets, such as:

• Infrastructure and observability manifests
• Testing frameworks
• Documentation templates
• Code formatting guidelines

You can create templates with nunjucks in a few different ways, including the CLI, a UI, and API. Templates live in GitHub and can be generated from a few lines of code and a set of input parameters using {{variable_name}} templating syntax throughout your template repository.

For the first step, we’ll create an App that executes a series of GitHub API requests to fetch the template repo’s tree object and files, and then runs a script to write all these files into a local working directory according to the hierarchy specified by the tree.

For the second step, we'll build a UI using App Builder that users can submit all their project details through. We’ll add forms for the project info (repo template path, project name, description, and owner) and destination (repo, path, and slug), as shown in the following screenshot.

Next, we’ll customize the form to match the parameters of our project template. We can update fields to add dropdowns, pre-fill data based on the user, or validate user input. Once we’re done with the form, we’ll pass in the form inputs to the template_variables field on the trigger workflow action.

Finally, we’ll connect our App to our existing project template by setting our Github connection and filling in the template_repo field.

Now, we have a complete app that lets team members generate a new repo or PR using templating logic at the touch of a button. The workflow adapts to templates based on the template_variables field and can therefore be standardized and reused across templates. We can further customize the logic by cloning the embedded Workflow to add notifications, approvals, or additional integration calls to cloud services, such as Terraform, internal deployment systems, and more.

Make your templates discoverable

Let’s say your organization has several templates with different versions that are owned by disparate teams. We can make the app even more useful by adding a layer to the UI that enables users to sort and filter the template list to find templates without knowing the specific repository URL offhand.

To improve discoverability, we’ll add the app to Self-Service Actions, which provides a central hub for developers to discover and take actions such as spinning up a new S3 bucket or scaffolding a new microservice.

Improve your organization’s developer experience

By defining organization-wide best practices for project scaffolding and making it easier for developers to follow them, Golden Paths help your development teams ship code faster and with fewer hiccups. Workflows and App Builder can help you implement Golden Paths for your organization by building a simple self-service interface for creating new templated projects. The Golden Paths app we’ve demonstrated in this post can help your teams more easily access templates and ensure that engineers can spin up new projects more efficiently.

Datadog App Builder is generally available for all Datadog customers. For more help getting started with project scaffolding, see the dedicated blueprints for both Github and Gitlab.

If you’re brand new to Datadog, sign up for a free trial.