Problem Definition

Like most DBAs, I have a lot of predictable maintenance tasks that I’m required to monitor. For example, there are changes, specifically data loads, which happen on a 'quarterly' basis, as defined by the institution where I am employed.    In this article I will describe an SSIS Package, that I have created, that once created into a job and run the first time, will schedule its next run before it completes.

Package Image

The red boxes with numbers correspond to the described Package Steps below.

Package Steps

This section describes the above image of an SSIS package and the details that are necessary to understand what is happening in each step.

Package Variables

*Note that good practice calls for a naming standard to name variables; for example the variables should have str before the strings, and int before the integer variable.

Base Data Load (Step 1)

I won’t be going into the details of this load, other than a general description, because the details are specific for each data load. 

In this case, the SQL statement calls a stored procedure that runs against a set of tables to generate the list of classes that were offered at the college over the last 2 years, as well as a few other details that will allow students to make an informed choice about when the classes they want to take are likely to be available.

Step Details

Setting up of next run variables (Step 2)

In this case, the next run date can be easily generated by looking at “Quarter” table, and determining when the next quarter’s ‘run’ should happen. For example, SQL Server generating next quarter’s run date out of the quarter table.

select Substring(Convert(varchar(30), qtrStartDate, 112),1,8) 
from CCSSM_REP.dbo.Quarter_Metadata 
where qtrIndex = (select qtrIndex+1 from CCSSM_REP.dbo.Quarter_Metadata where qtrCurrentQtr = 1)

As I am only showing a part of a single quarter’s information as an example,  assume past and future quarter information exists. The T-SQL code grabs the next line’s qtrStartDate, that exist in the table following the current quarter per the qtrCurrentQtr column.

All that is needed is a valid ‘future’ date SQL statement that returns  the next date you want to run the job.  For example, this SQL could easily be replaced with SQL that gives you the next Sunday date so that it handles weekly loads.

Step Details 

*Step Images that involve scripting have associated images to show the package variables used.

Script Details

This step sets the package variables.  This is where you’ll make your changes to fit your job name and schedule name.  Also the date generating SQL (strSQL) will need to be modified to fit your needs.  The key variable details are strSQL, JobName, and ScheduleName.

        public void Main()
        {
         String strSQL ="select Substring(Convert(varchar(30), qtrStartDate, 112),1,8) from CCSSM_REP.dbo.Quarter_Metadata where qtrIndex = (select qtrIndex+1 from CCSSM_REP.dbo.Quarter_Metadata where qtrCurrentQtr = 1)";
         Dts.Variables["strSQL"].Value = strSQL;
         Dts.Variables["JobName"].Value = "EdPlan_Fill_Extended_Info";
         Dts.Variables["ScheduleName"].Value = "Quarterly_Run_Edplan_Fill_ExtendedInfo";
         Dts.TaskResult = (int)ScriptResults.Success;
        }

Execution of variable SQL to get next run date (Step 3)

This step takes the strSQL variable created in Step 2,  executes it against the database, and returns the value to the user variable ScheduleDate.

Step Details

Result Set

Setting of schedule lookup variable (Step 4)

To determine if there is an existing schedule for this job, this step sets a variable with the proper SQL statement based on variables set in Step 2, to be executed against the database.  This section does not need to be modified.

Step Details

Script Details 

        public void Main()
        {
         // Get next Run date.
            String strSQL = "SELECT COUNT(*) FROM msdb.dbo.sysschedules where name ='" + Dts.Variables["ScheduleName"].Value + "'";
         Dts.Variables["strSQL"].Value = strSQL;
         Dts.TaskResult = (int)ScriptResults.Success;
        }

Execution of schedule lookup SQL (Step 5)

This Section takes the strSQL variable and executes it against the database and returns the value to the user variable ScheduleDate.

Result Set

Success Connectors

On the out-flow of this Step there are precedence constraints added to the success pathways to dictate the flow, the dependant variable is UpdateSchedule. This will allow the variability to either call an update schedule stored procedure or a create schedule stored procedure 

Schedule Exists

There exists a current schedule by this name in the database.

Schedule Does Not Exist

No schedule was found with this name in the database.

Set of SQL to Create or Update Schedule (Step 6)

Update Schedule SQL Step Details

Update Schedule SQL Script Details

The basic SQL for this came from Microsoft. http://msdn.microsoft.com/en-us/library/ms187354(v=SQL.100).aspx; I modified it to suit my specific purposes.  This step does not need to be modified.

       public void Main()
        {
            String strJobScheduleSQL;
            strJobScheduleSQL = "exec msdb.dbo.sp_update_jobschedule @job_name ='" + Dts.Variables["JobName"].Value + "', @name='" + Dts.Variables["ScheduleName"].Value + "', @freq_type = 1, @active_start_date = '" + Dts.Variables["ScheduleDate"].Value + "'";
            Dts.Variables["sqlSchedule"].Value = strJobScheduleSQL;
            Dts.TaskResult = (int)ScriptResults.Success;
        }

Create Schedule SQL Step Details

Create Schedule SQL Script Details

The basic SQL for this came from Microsoft http://msdn.microsoft.com/en-us/library/ms366342(v=sql.100).aspx; I modified it to suit my specific purposes.  This step does not need to be modified.

        public void Main()
        {
            String strJobScheduleSQL;
            strJobScheduleSQL = "EXEC msdb.dbo.sp_add_schedule @schedule_name = '" + Dts.Variables["ScheduleName"].Value + "', @freq_type = 1, @active_start_date = '" + Dts.Variables["ScheduleDate"].Value + "' EXEC msdb.dbo.sp_attach_schedule @job_name = '" + Dts.Variables["JobName"].Value + "', @schedule_name = '" + Dts.Variables["ScheduleName"].Value + "'";
            Dts.Variables["sqlSchedule"].Value = strJobScheduleSQL;
            Dts.TaskResult = (int)ScriptResults.Success;
        }

 Schedule Creation/Modification (Step 7)

Update Schedule Execute Step

This step executes hte SQL generated in the previous step against the database.

 Create Schedule Execute Step

This is really the same step as the update; both use the same variable but it is repeated here for clarity, to match the package image.

Mail Job Results (Step 8) 

Each mail message has slightly different verbiage, which tells me where the failure happened, or which branch the success took.

Step Details

Example 1 - Load Failure

Example 2 - Update Success

This is the end of the package details.

Last Task

To make all of this functional, you will need to deploy this script to your SQL server.  There are several ways to do this, which I won’t discuss here.  After the package is deployed to your server, create a job that references the package, and then run the job.  If the schedule is non-existent a job schedule will be created, or if it does exist it will be updated.

Limits

These are limitations of this process.

1. The user needs to add mail messages for failure of either update or insert of schedule.
2. This is not covered, but the administrator could add some more input parameters that could set the run for a specific time if required.

Other Considerations

1. By making the package even more generic and passing the needed values into the package when the job runs, this process could become even more versatile. 
2. By changing the MessageSourceType to a variable, and then passing the message in and using each scripting step along the way to adjust the message to what is happening in that step, would allow me to remove all but two mail steps; success and failure.
3. Merging the Create and Update parts of step 7 would consolidate a functional duplicate.

Summary

What has been shown is a methodology for jobs that can self-schedule, based on a calculable date, and in this case handles a periodic data load.  Once one understands the basic flow, it’s really a pretty simple idea as demonstrated by the following list:

• Do Load
• Set variable for job name, schedule name, and SQL to generate next run date
• Execute the previous step's sql and place value in variable
• Set variable for Job Schedule Lookup
• Execute the previous step's sql and place value in variable
• A) if Schedule exists create update SQL and place in variable
• B) if Schedule does not exist create Schedule create SQL and place in variable
• Depending on if the Schedule exists execute the proper SQL statement
• Send mail message of results

So far, this methodology has worked well and hasn’t failed yet.  With the minor addition of mail messages to #7, I should always get a message telling me the results of the run.

So there you have it, a self-scheduling job that handles a data load.