Skip to main content

Deployment configuration reference (Dagster+)

Dagster+ feature

This feature is only available in Dagster+.

Dagster+ Hybrid deployments use several configuration files to define how code locations are built, deployed, and run. These files are typically scaffolded by dg plus deploy configure and live in your project or workspace root.

FilePurpose
build.yamlDocker image build settings (registry and build directory)
container_context.yamlInfrastructure-specific runtime configuration (Kubernetes, ECS, Docker)
pyproject.tomlProject metadata including code location name, agent queue, and image
note

In older deployments not managed by the dg CLI, a single dagster_cloud.yaml file was used instead of these three files. For details, see Legacy dagster_cloud.yaml format.

File locations

my-project/
├── build.yaml
├── container_context.yaml # optional
├── Dockerfile
├── pyproject.toml
├── src/
│ └── my_project/
│ ├── __init__.py
│ └── definitions.py
└── uv.lock

build.yaml

The build.yaml file defines Docker image build settings for Hybrid deployments. It contains only two fields:

# build.yaml
registry: 764506304434.dkr.ecr.us-east-1.amazonaws.com/my-image
directory: ./
PropertyDescriptionFormatDefault
registryThe Docker registry to push the built image tostring (Docker registry URL)
directoryThe path to the directory containing the Dockerfilestring (path).

The registry field specifies where Docker images are pushed during CI/CD. The directory field is useful when your Dockerfile or pyproject.toml is in a subdirectory rather than the project root.

Workspace and project merging

When both workspace-level and project-level build.yaml files exist, settings are merged with project-level values taking precedence. A common pattern is to set registry at the workspace level and directory at the project level:

# workspace-level build.yaml
registry: 764506304434.dkr.ecr.us-east-1.amazonaws.com/my-image
# project-level build.yaml
directory: ./projects/data-eng

container_context.yaml

The container_context.yaml file defines infrastructure-specific runtime configuration for code server and run containers. The available settings depend on which Hybrid agent platform you use.

All platforms support a shared top-level env_vars field:

# container_context.yaml
env_vars:
- DATABASE_NAME=staging
- DATABASE_PASSWORD # pulled from agent environment
PropertyDescriptionFormat
env_varsEnvironment variables for all containers. Use KEY=VALUE to set a value, or KEY alone to pull from the agent's environmentlist[string]

The platform-specific settings are nested under a key matching your agent platform. For example, a Kubernetes deployment might look like:

# container_context.yaml
k8s:
namespace: my-namespace
env_secrets:
- my-secret

For the full list of available settings, see the configuration reference for your agent platform:

pyproject.toml

For projects managed with the dg CLI, several deployment settings are configured in pyproject.toml under the [tool.dg.project] section. These settings determine how the project is identified and routed during deployment.

# pyproject.toml
[tool.dg.project]
root_module = "my_project"
code_location_name = "my-project"
agent_queue = "special-queue"
image = "my-registry/my-image:latest"
PropertyDescriptionFormat
root_moduleRequired. The root Python module containing Dagster definitionsstring
code_location_nameThe name for this code location as it appears in the Dagster UI. Defaults to the project directory namestring
agent_queueRoutes this code location to a specific agent queue. Useful when running multiple agentsstring
imageA pre-built Docker image to use for this code location, skipping the build step entirely. This is an alternative to configuring a registry in build.yamlstring (image reference)
tip

If you use dg.toml instead of pyproject.toml, the same settings go under [project] instead of [tool.dg.project].

Legacy dagster_cloud.yaml format

Expand for the legacy dagster_cloud.yaml reference

In older Dagster+ deployments not managed by the dg CLI, a single dagster_cloud.yaml file (sometimes also called build.yaml) was used to define all deployment configuration in one place. This format combines code location metadata, build settings, and container context into a single file.

warning

This format is supported for backwards compatibility. New projects should use the dg CLI with the separate build.yaml, container_context.yaml, and pyproject.toml files described above.

Structure

The file contains a single top-level locations key with a list of code location entries:

# dagster_cloud.yaml
locations:
- location_name: data-eng-pipeline
code_source:
package_name: example_etl
build:
directory: ./
registry: localhost:5000/docker_image
container_context:
k8s:
env_vars:
- database_name
- database_username=hooli_testing
env_secrets:
- database_password
agent_queue: special-queue
- location_name: ml-pipeline
code_source:
module_name: example_ml
image: my-registry/my-image:latest

Settings per location

PropertyDescriptionRequired
location_nameThe name of the code location as it appears in the Dagster UIYes
code_sourceHow to find Dagster definitions (see below)Yes
buildDocker build settings: directory and registryNo
working_directoryDirectory to load code from, if different from the project rootNo
executable_pathPath to the Python executable to useNo
imagePre-built Docker image (alternative to build)No
container_contextInfrastructure-specific runtime configuration (same schema as container_context.yaml)No
agent_queueRoutes to a specific agent queueNo

Code source

The code_source must contain exactly one of:

PropertyDescription
code_source.package_namePython package name containing Dagster definitions
code_source.module_namePython module path containing Dagster definitions
code_source.python_filePath to a Python file containing Dagster definitions

Defaults

You can set default values that apply to all locations using the defaults key:

# dagster_cloud.yaml
defaults:
build:
registry: my-registry/my-image
container_context:
k8s:
namespace: dagster
locations:
- location_name: data-eng
code_source:
package_name: example_etl
- location_name: ml-pipeline
code_source:
module_name: example_ml