Sync Out for Snowflake
The Tableau CRM team spent a bunch of time in 2020 working with our partners at Snowflake, and as a result of the collaboration, we came up with a unique feature that will allow Salesforce customers to send raw data to Snowflake for data lake hydration purposes.
Tableau CRM already had the key pieces needed to make this work:
- A mechanism to pull data from Salesforce via the Bulk API
- A write connector to send data to Snowflake.
So our engineers pieced the two features together to give an ability to leverage the data sync schedule, incremental load support, and execution along with a crafty auto DDL (table create and recreate) execution to create Sync Out for Snowflake. Enabled with a few clicks, this is a great way to keep the data in Snowflake refreshed.
Note: In case you prefer a video rather than reading or in conjunction with? Then please check out the Two Minute Connector Series.
Not all customers would want all the raw data in Snowflake, so they can use the same Output Connector to Snowflake to write curated data to targeted tables via Data Prep Recipes.
- The Data Sync for your Local Salesforce Data allows you to pull data from Salesforce into Tableau CRM.
- The Snowflake Output Connector can be used to write data to a Snowflake table of your choice via a Data Prep Recipe; this connector is also how you create the connection for Sync Out to use.
- The new topic here: Our Spring ’21 feature Sync Out for Snowflake lets you write the data to Snowflake based on the execution of Data Sync.
Pro Tip: Data Sync can be scheduled at your desired interval (no lower than 15 min) and you can setup a Recipe to execute based on Data Sync success; so you can use Sync Out and the Output Connector in a unified strategy (some raw data and some curated)
Setting up Sync Out
The setup for Sync Out is quite simple assuming you are familiar with the concepts and features of Data Sync.
- Have your Admin enable the Connector in Analytics Settings by selecting Enable Snowflake output connection.
- Create a Connection in Data Manager using Connect
- Enable Sync Out on the Salesforce Local Data Sync Connection(s) you wish.
- Lastly, when your Data Sync runs on-demand or via schedule, the data is written to Snowflake (A manual run could be scheduled as well). Once you have run the data you can head to Snowflake to see the new tables that have been created.
That’s how you can keep your Salesforce data up to date in Snowflake without the need for a third-party ETL tool.
Written by
Hi Chris, thanks for the post, would this require a Tableau CRM subscription just to access the sync out to snowflake connector?
Nicola, Sync Out is a feature of Tableau CRM therefore you need to be licensed for Tableau CRM to use it.
Hi Chris
Thanks for this. We are lapping it up and thinking of all sorts of use cases for writing data back to salesforce. Data Prep is such a powerful tool for ingesting and cleaning data!
Curious if you have considered using it as a data integration tool? E.g. to pull in an external data source, fix up the data, then write to Salesforce. This would appear to work well when you are writing to single objects – trickier if you are creating related records I expect. In our case we have a staging object that we use to stage our integrations so it should work.
Our scenario is web donations that sit in an S3 table.
Thoughts?
Hi Mike,
Have a look at Chris’ latest blog on writing to Salesforce https://www.salesforceblogger.com/2021/04/14/output-connector-to-salesforce/
Chris, do you think this could be used by the customers of a TCRM OEM partner like us at FinancialForce, would love to chat about this with you and some of my dev team.
Joe, no reason why not it is TCRM functionality. Reach to out to discuss as needed.
Greetings, Chris.
Outstanding post. Quick question: what type of exception is used in the outbound connector to snowflake?
Encryption* damn autocorrect. What type of encryption is used in the outbound connector to snowflake
Philip, we don’t publicly disclose security/trust details out of practice. So I can’t tell you the “type”. We do have customer self-service on our accreditations which allow you to see the details of our core requirements including TLS 1.2 or greater for data transfer encryption. Keep in mind we enforce at a product level not a feature by feature so we can’t take something to market without adhering to the set standards and levels.
https://compliance.salesforce.com/en/services/salesforce-services
Hi Chris
Follow up question, is it a limitation that formula fields created on SF Objects don’t sync out to snowflake?
Very unfortunate that it required TCRM subscription to get your data across to Snowflake. What if I have a Sales/Service cloud subscription and want to move my data to Snowflake?
Hi chris,
Thanks for the article – super helpful
Wanted to confirm a small detail – is this sync-out feature able to handle schema changes?
Thanks in advance!