How to copy ADF pipeline from one subscription to another and change the config

  • Hello,

    I have a pipeline which is a chain of several sprocs, activities etc... My ADF uses user assigned managed identity of the relevant subscription, let's call it ArchiveUser that accesses several Azure sql servers and azure blob storages.

    Pipeline in Dev

    Dev subscription

    ArchiveUser set up in dev subscription

    Azure sql servers in dev subscription

    Azure blob storage in dev subscription

    My question is how to copy and customize those settings for another subscription?

    We have 4 subscriptions Dev, test, stage and prod.

    Each of these subscriptions is meant to have the same pipeline with the exception of pointing to relevant resources in their appropriate subscription.

    In traditional SSIS, there is a .DTSConfig which contains the config variables for these resources.

    Thank you!

  • If you are just talking about the ADF (assuming databases, etc. already exist in other subscriptions) I would look here for Data Factory CI/CD https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery.  You will need your Dev ADF tied to a Git repo which is a good idea anyway.

    You can also Export the ARM template of the ADF and then deploy that in each subscription and supply the different parameters such as connection strings to the resources in each subscription.  The parameters can be stored in a parameters file for each environment.'

    If you need to deploy all the resources (database, storage account, etc.) then you should investigate Bicep (my preference) or Terraform.

  • Scott,

    Thank you for your reply.  We plan to use ADF CI/CD so will definitely investigate further for implementation.

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply