Using variables in SSIS packages can make your ETL process more dynamic and respond to different scenarios during runtime.
Not only we can use variables, we also have many different ways to set values for our variables. This is all good news for developers. But like many things in real life, when we have too many choices, we can get carried away and overcome by our enthusiasm. This is especially true in terms of variables in SSIS packages.
5 different ways to set values for variables
In general, we have the following 5 different ways to set values for variables. The question we need to ask is what is the order of precedence during runtime when we are using all these 5 different ways to set values for a variable.
- In the Variable Editor in the Value field: although varCubeName is a String type, we do not need to use the double quotes to enclose the value Cube Name in Value.
- In the Variable Expression Editor: variable values can also be set in the Expression field. We must enclose the value in double quotes. This is true even for Integer type variable. Notice that the value in the Value field has changed from Cube Name in Value to Cube Name in Expression. Once I entered an expression, both the Value and the Expression field has become the same.
- In the package configuration file: I always prefer an XML configuration file, in which I set varCubeName to a value of Cube Name in Package Configuration.
- In many control flow tasks, such as an Execute SQL Task, and some data flow transformation tasks. for the purpose of this blog, I picked the Execute SQL Task to set the value for varCubeName to Cube Name from Execute SQL Task during runtime.
- In Script Task, I can also modify the value of varCubeName. For the purpose of this blog, I set it to Cube Name in Script Task in a scrip task.
- To summarize, here are the values I set for the variable varCubeName using 5 different ways:
- Cube Name in Value: in the Variable Editor in the Value field
- Cube Name in Expression: in the Variable Editor in the Expression field
- Cube Name in Package Configuration: in the package configuration file
- Cube Name from Execute SQL Task: in the Execute SQL Task
- Cube Name in Script Task: in a scrip task
4 and 5 are essentially the same, so I’ll just test it one at a time. In my first test, I disabled the scrip task because I only want to test 1, 2, 3, and 4. I want to see what the value for my variable varCubeName really is at runtime. I enabled the package configuration so I can throw #3 in the midst.
I used two script tasks with the MessageBox function to show me the value at runtime, both before and after the Execute SQL Task.
“Cube Name in Expression” took precedence over all other values
Before the Execute SQL Task…
After the Execute SQL Task…
The value is always Cube Name in Expression. It’s like the Execute SQL Task had never even executed.
“Cube Name in Package Configuration” took precedence over Cube Name in Value, but gives way to “Cube Name from Execute SQL Task”
Now we know that Cube Name in Expression has the highest precedence order. Let’s remove it and only leave the Cube Name in Value, as shown in the following screenshot.
Before the Execute SQL Task…
After the Execute SQL Task…
This time we see that the Cube Name in Value which is set in the Variable Editor in the Value field never showed up. Cube Name in Package Configuration and Cube Name from Execute SQL Task both overwrite the Cube Name in Value.
Enabling the Scrip Task produces similar results, in which “Cube Name in Script Task” has the same precedence order as the “Cube Name from Execute SQL Task”.
Conclusion on the precedence order
- Cube Name in Expression: the value is set in the Variable Editor in the Expression field. The value set in this way has the highest precedence order and overwrites the values in all other ways.
- Cube Name from Execute SQL Task: the value is set in the Execute SQL Task. It has the second highest precedence order. Cube Name in Script Task has the same precedence order.
- Cube Name in Package Configuration: the value is set in the package configuration file. It can only overwrites the variable’s default value.
- Cube Name in Value: the value is set in the Variable Editor in the Value field. This value is usually called the default value, because it can be overwritten by all the proceeding ways during runtime.
Lessons learned
During the SSIS package deployment, your DBA decided to include all the variables along with all the connection strings as the property/value pairs in the package configuration files, and somehow your packages are not working the way you had expected.
During development, your packages are not working the way you would expect and you decided to debug your variables and found out that the variable value at runtime was incorrect.
The above two scenarios happened to me in the past and they have inspired this blog.
The lessons I’ve learned are:
- During deployment, do not include variables property/value pair in the package configuration file. Variables should only be handled in the SSIS packages.
- If variable expression is sufficient, use variable expression only.
- If you need to combine different ways to set variable values at runtime, only use these two combinations.
- Default value + Execute SQL Task, or
- Default value + Script Task
Note
- All screenshots are from Integration Services 2012.