Execute Process Task: How SSIS loads executable files at runtime

Question

Execute Process Task: How SSIS loads executable files at runtime

reelnoncents

SSChasing Mays

Points: 632
More actions
October 25, 2017 at 6:58 am

#329276

Please help me understand how SSIS loads executable files at runtime. Does it copy the file to RAM at runtime to run it? I'd like to know if many jobs running on th same schedule, running the same executable file, will potentially lock resources to this file. My personal tests haven't been fully consistent and would like to know this before deploying to production.
Thank you in advance.

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply

Phil Parkin SSC Guru Points: 246962 More actions · Answer 1

reelnoncents - Wednesday, October 25, 2017 6:58 AM
Please help me understand how SSIS loads executable files at runtime. Does it copy the file to RAM at runtime to run it? I'd like to know if many jobs running on th same schedule, running the same executable file, will potentially lock resources to this file. My personal tests haven't been fully consistent and would like to know this before deploying to production.
Thank you in advance.

My understanding is that it does not 'load' anything. It just runs the executable. Now imagine five people running the same executable ... depending on the exe, that may or may not be an issue & the same should be true of an Execute Process task.

The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin

reelnoncents SSChasing Mays Points: 632 More actions · Answer 2

Thank you for your response Phil. It helps in my search. Do you know what the dependent variable may be on the exe? The exe in question is a custom console application I've built to handle universal processes amongst our many clients. This exe can potentially be called at the same time by multiple SQL agent jobs depending on how we implement it. One possibility is having a unique copy of this exe for each client, but that wouldn't be optimal. I'd ideally have one exe copy on our system that each client calls. Do you know the progamming concept I should look out for to avoid the possibilty of locking this single source file? Thanks again, in advance.

ZZartin SSC-Dedicated Points: 30920 More actions · Answer 3

It really depends on what the application is doing. At the basic level all execute process is doing is starting an OS process that runs whatever command you gave it and there's certainly nothing preventing an executable from being loaded and run multiple times in parallel. What is the limiting factor is what exactly the process is doing, for example is it using some shared resource like writing to a shared log file that will crash if multiple instances try to use it? Is it parameterized so each client can call it with appropriate inputs for what they're doing without conflicts?

Phil Parkin SSC Guru Points: 246962 More actions · Answer 4

reelnoncents - Wednesday, October 25, 2017 8:40 AM
Thank you for your response Phil. It helps in my search. Do you know what the dependent variable may be on the exe? The exe in question is a custom console application I've built to handle universal processes amongst our many clients. This exe can potentially be called at the same time by multiple SQL agent jobs depending on how we implement it. One possibility is having a unique copy of this exe for each client, but that wouldn't be optimal. I'd ideally have one exe copy on our system that each client calls. Do you know the progamming concept I should look out for to avoid the possibilty of locking this single source file? Thanks again, in advance.

I'm no expert in this, but my expectation is that while EXEs do get locked at file level during execution, that does not mean that they cannot be called (and executed) from process 2 while already executing process 1.
I suggest that you set up a few Agent jobs and test it ... should not take too long. Add in a delay (in the EXE), if necessary, to make sure there is an overlap in the calls.

The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin

reelnoncents SSChasing Mays Points: 632 More actions · Answer 5

ZZartin - Wednesday, October 25, 2017 9:06 AM
It really depends on what the application is doing. At the basic level all execute process is doing is starting an OS process that runs whatever command you gave it and there's certainly nothing preventing an executable from being loaded and run multiple times in parallel. What is the limiting factor is what exactly the process is doing, for example is it using some shared resource like writing to a shared log file that will crash if multiple instances try to use it? Is it parameterized so each client can call it with appropriate inputs for what they're doing without conflicts?

This is exactly what I needed. I ultimately found a couple other sources that stated the same, including a peer. It is parameterized, and they all will run within their own client environments so I believe I should technically be OK. Many thanks for your response ZZartin.

reelnoncents SSChasing Mays Points: 632 More actions · Answer 6

Phil Parkin - Wednesday, October 25, 2017 9:14 AM

reelnoncents - Wednesday, October 25, 2017 8:40 AM
Thank you for your response Phil. It helps in my search. Do you know what the dependent variable may be on the exe? The exe in question is a custom console application I've built to handle universal processes amongst our many clients. This exe can potentially be called at the same time by multiple SQL agent jobs depending on how we implement it. One possibility is having a unique copy of this exe for each client, but that wouldn't be optimal. I'd ideally have one exe copy on our system that each client calls. Do you know the progamming concept I should look out for to avoid the possibilty of locking this single source file? Thanks again, in advance.

I'm no expert in this, but my expectation is that while EXEs do get locked at file level during execution, that does not mean that they cannot be called (and executed) from process 2 while already executing process 1.
I suggest that you set up a few Agent jobs and test it ... should not take too long. Add in a delay (in the EXE), if necessary, to make sure there is an overlap in the calls.

[/quote

Thank you for your advice, Phil. Always astonished at how you're typically the first person to respond to any of my posts. Model worker, you 🙂 Have a good one!