In the world of Salesforce development and administration, managing and migrating metadata, process definitions, and configurations between different Salesforce environments is a critical task. Whether you’re working on a Salesforce Sandbox, a Production org, or using Salesforce’s Data Cloud, knowing how to transfer these elements efficiently is key to maintaining a smooth development lifecycle.
In this blog post, we’ll walk through the best practices, tools, and steps involved in migrating your Data Cloud metadata, process definitions and configurations between Salesforce orgs.
What is Data Cloud Metadata and Why Does It Matter?
Before diving into the specifics of migration, let’s define what we mean by Data Cloud metadata. In Salesforce, metadata refers to the underlying configuration and structure that makes up your Salesforce environment—like objects, fields, and custom settings. When working with the Salesforce Data Cloud, metadata can also include data models and process definitions.
Process definitions in Data Cloud refer to the workflows and processes involved in managing and integrating data within the Data Cloud platform. For example, datastreams, CI’s, segments – all are process definitions.
As developers and admins, it’s critical to manage and migrate both metadata as well as process definitions between orgs to maintain consistency, track changes, and ensure proper functionality across environments. A seamless migration process minimizes downtime and ensures that everything works as expected in your production org.
Prerequisite(s) for migrating Data Cloud metadata & process definitions
Before you begin the migration process, make sure you have the following:
- Access to Source and Target Orgs: You need sufficient permissions in both the source Salesforce org and the target org (Sandbox or Production).
- Salesforce CLI (Command Line Interface): Salesforce CLI is a powerful tool for managing Salesforce environments, including metadata migration. Ensure it is installed and configured.
- Salesforce Change Sets or CI/CD Pipeline: If you’re using Salesforce’s out-of-the-box Change Sets, or a custom CI/CD pipeline, you’ll need access to these tools as well.
- Metadata Coverage: Verify that your source and target orgs are in sync in terms of metadata compatibility. Some Salesforce features may not be available across all environments.
There are several methods available for migrating metadata and process definitions from the sandbox org.
Two of which are listed below:
- Using Salesforce CLI (Based on metadata api)
Salesforce CLI provides a flexible and scriptable way to extract metadata from an org and deploy to another org. You can create a SFDX project (with manifest) and use CLI to retrieve and deploy Data cloud metadata via DevOps datakits.
- Using Change Sets (UI-based)
For a more visual, point-and-click approach, Salesforce Change Sets can be used. Here’s how:
- In your target org, navigate to Setup->deployment settings and set the “Allow Inbound changes” = True
- In your source org, navigate to Setup → Outbound Change Sets.
- Create a new Change Set and add component “datapackagekitdefinition”
- Select the devOps datakit that you created.
- Click on “View/Add dependencies” and add all the dependencies to the changeset.
- Upload the Change Set to the target org (Sandbox or Production).
Note: Salesforce Change Sets work best when moving metadata between orgs within the same Salesforce instance.
Let us look at the steps involved in moving data cloud metadata and configurations using CLI.
Steps to Move Data Cloud Metadata and Configurations using CLI
Step 1: Create an SFDX Project
Before creating the SFDX project, ensure the following prerequisites are met:
- Download and install VScode
- Install Salesforce DX CLI
- Install Salesforce Extension Pack for Visual Studio Code
You can then create a project with the standard SFDX folder structure, with manifest.
In the sfdx-project.json file, make sure sfdcLoginURL is set to the correct URL. When working with Sandbox org, sfdcLoginURL can be set to https://test.salesforce.com . For production org, sfdcLoginURL can be set to https://login.salesforce.com
Step 2: Authorize your source and target orgs in SFDX
Both source and target orgs need to be authorized in your SFDX project.
You can authorize multiple orgs, but only one of the authorized orgs can be set as a default org.
In this blog, we will show how you can migrate Data cloud metadata, process definitions as well as Platform configurations (Flow, Reports which have references to Data cloud metadata like DMO’s) from one sandbox org to another sandbox org.
Source Sandbox Org is SandboxTDX1 and Target Sandbox Org is SandboxTDX2.
Step 3: Ensure Dataspaces with same prefix exist in both source and target orgs
Any Dataspace which is newly created in a Sandbox org needs to be manually created in your target sandbox or production org. Dataspace and prefix cannot be migrated automatically across orgs via any of the mechanisms described in this blog. Make sure the dataspace from which you want to migrate metadata and process definitions in source org also exists in target org. If it doesn’t, please login to your target org and create the dataspace manually by going to Data cloud setup -> Dataspaces.
Dataspaces in Source Org (SandboxTDX1):
Dataspaces in Target Org (SandboxTDX2):
4. Identify the Data Cloud Metadata and process definitions to migrate
The first step is identifying exactly which components/features/process definitions need to be migrated. Features/process definitions could include the following but is not limited to:
- Datastreams, DLO’s,Custom DMO’s and mappings
- Calculated Insights
- Data Transforms
- Identity Resolution Rules
- Data Actions
- Data Graphs
- Segments
It is important to plan and keep a note of all interdependencies between above components which need to be migrated. Example: A segment may be dependent on a CI, a CI may be dependent on a DLO to DMO mapping.
Let us consider that you have added a new CRM connection in your source sandbox org SandboxTDX1 with the alias “TDXCRMorg” as shown in the snapshot below:
Also, you have added a S3 connection with the name “S3TDX” as shown in the snapshot below:
There are two datastreams created for the above two connections. Both reads Account data, one from the CRM org and other from a csv file placed in an S3 bucket.
The data from both these streams is joined and written into a custom output DLO “AccountTDXOutputDLO” via a Batch Transform as you can see below.
In addition to output DLO and Batch transform, there is also a new segment created in SandboxTDX1.
Additionally, a Flow is created which has a reference to the Individual DMO in the North America dataspace.
There is a report which is created on Individual DMO in North America dataspace.
Now, we have a bunch of Data cloud metadata (Datastreams, DLO, transforms, segments) as well as Platform metadata (Flow, Reports) to be migrated to target Sandbox Org.
Step 5: Create a DevOps Datakit in your source org
- Go to Data cloud Setup -> Developer Tools -> Datakits.
- Click on New and select the datakit type as “DevOps”.
- Select the dataspace for which you are creating the devops datakit i.e. North America
- Specify the datakit name i.e. TDXDevDatakit
- Click on Save
- Add the metadata and process definitions to the datakit
Please note that the components that you add to the devops datakit (North America dataspace) in source org will be deployed into exactly the same dataspace (North America) in the target org.
After you’ve created a devops datakit and added all metadata and process definitions that you want to migrate, it is very important to review and save the deployment sequence of components in the datakit, which is described in the next step.
Step 6: Define & Save the deployment sequence of components in a devops datakit
After you have added all components, metadata and process definitions into the datakit, click on “Publish sequence”, edit the sequence to arrange the components in the correct order of dependencies to ensure the components are deployed in the correct order in the target org and then save the sequence.
This is a very important step and requires careful review to ensure the right order. If the order is not defined correctly, this could lead to deployment failures in the target org.
Example: If the Segment is dependent on a CI, then CI needs to be placed before Segment in the deployment order.
By default, the order is automatically generated based on the created date of the component, with the component with the earliest created date as the first component.
Step 7: Download Manifest (i.e. package.xml) for the datakit
After your datakit is ready, Download the manifest (i.e. package.xml) for the datakit from the UI as shown in the snapshot below.
This package.xml contains all the metadata entities related to a datakit and you no longer need to create this manually.
Step 8: Pull the devops datakit from source org via CLI
You can use the package.xml downloaded from the UI in your SFDX project and run the retrieve command to retrieve the datakit metadata into your SFDX project. Make sure your source org (i.e. SandboxTDX1) is set as the default org in your SFDX project.
Command to retrieve the datakit metadata:
sf project retrieve start --manifest manifest/package.xml
This command will retrieve the datakit from source org into your SFDX project.
Step 9: Integrate your SFDX project with Github
You can integrate your SFDX project with github and maintain the datakit single source of truth in the github repository.
All retrieved data kit metadata files in your SFDX project can be uploaded to Github.
Step 10: Push the devops datakit into the target org via CLI
Set the default org in your SFDX project as your Target Sandbox org where you want to move the metadata i..e Sandbox TDX2
Run the following command to push the devops datakit into your target org via CLI.
sf project deploy start --manifest manifest/package.xml
Step 11: Deploy the metadata & process definitions from the devops datakit into the target org
Login to your target org, navigate to DC setup -> Developer tools -> Datakits. You can see the datakit that you had created in your source org is now available in your target org.
Click on the datakit name to view the datakit record home page. Then click on the “Deploy” button to deploy all metadata & process definitions from the datakit into your target org.
After you have clicked on the “Datakit Deploy” button, You can check the status of deployment in the deployment history page.
Step 12: Authorize the new connections(if any) that were deployed in inactive state
If there were any new connectors that were created in your source org and if there were datastreams related to these new connectors that were added to the datakit. In this case, the deployment will fail for datastreams because the connectors will get deployed in an inactive state and would need to be re-authorized or re-authenticated.
After you have successfully re-activated/re-authenticated the connections, you can proceed to the next step. You need to do the same for the S3 connection as well.
Step 13: Re-deploy the metadata & process definitions from the devops datakit into the target org
After the connections are re-authorized and are in active state, click on the “Deploy” button again to resume the deployment of datakit.
Step 14: Track the status of datakit deployment via deployment history
At any time during the migration process, you can check the status of datakit deployment or any component deployment by going to the deployment history page on the datakit UI.
Now, All the Data cloud metadata and process definitions have been successfully deployed into the target org.
Now, let’s take a look at how you migrate the core platform configurations like Flow and Report which have a dependency on Data cloud DMO’s from SandboxTDX1 to SandboxTDX2.
It is very simple. These are not supposed to be added to Datakit and can be retrieved and deployed via metadata api CLI commands directly. Only Data cloud metadata is required to be added to a Datakit.
Conclusion
Migrating Salesforce Data Cloud metadata, process definitions, and configurations between different orgs is an essential skill for both developers and admins. By understanding the available tools—such as Salesforce CLI, Change Sets, and the Metadata API—and following best practices, you can ensure a smooth and reliable migration process.
Whether you’re moving to a new Sandbox for testing or pushing configurations to Production, maintaining a clean, well-structured process will save you time, prevent errors, and ensure that your Salesforce environments are always up-to-date.
Written by
