Monday 29 April 2013

SSIS Child Errors in Parent Package

If you have a parent package that is looping through a set of files and calling child packages and want the parent package to continue even if the child package fails, there is a little work involved in making this happen.
In this example I have a parent package that loops through a collection of files & calls the appropriate child package (out of 2 child packages). I want to abort/exit the child package without aborting the parent package also. As soon as 1 error is found, I need to exit the child package and move the file to an error directory so the parent loop won’t be interrupted.

I created a master package and two child packages to demonstrate how I would do this. I will walk through the master set up and one of the child packages. Each child package is identical. I am going to explain how I would call each child package also. I will also cover the error issue.

image

      Above is the Master or Parent package. I have a For Each Loop loping through a set of files. The first item in the For Each Loop is a Sequence Container. This is used as an expression anchor. This allows you to decide which child package to run based on the value of the expressions. The expressions just check the file name. Based on the file name it will run the appropriate child package. After the child packages there is a script task that checks for the error file that may have been create by the child packages. It also deletes the error file and moves the data file that had the error to a backup folder.

image
      Above are the variables in the master package. Notice we are trying to make the package as dynamic as possible. Nothing is hard coded. If any of these items change it is a simple change to a variable to update the package. This can be done from configuration files also. The variable strFileName is going to be filled with the For Each Loop. Everything else is as shown.

image

      The For Each Loop in the master package is set up to use the variables in the expressions of the collection. Notice it is only getting the name of the file. The folder and extension are in the variables. The file name is mapped to the variable strFileName

image
      Above is the child package configuration. I have placed all the same variables in the child package and used parent configurations to pass all of the values to the child package.

image
      The child package consists of only one data flow. The input file name is set dynamically with an expression on the connection string mapped to the variables. The expression is:

@[User::strInputFolder] +  @[User::strFileName] +  @[User::strFileExt]

This combines all of the proper variables to give the package the location of the current file found by the for each loop in the master package.

    Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
        ‘
        ‘Create script variables to hold package variable values
        Dim strFileExt As String = Me.Variables.strFileExt
        Dim strChildErrorFile As String = Me.Variables.strChildErrorFile
        Dim strInputFolder As String = Me.Variables.strInputFolder
        Dim badfile As String = strInputFolder + strChildErrorFile + strFileExt

        ‘create file to indicate an error I
        Dim fs As FileStream = File.Create(badfile)
        fs.Close()

        ‘
    End Sub

    On the source in the data flow I have set all of the errors and truncation to redirect to the error output. This will cause the script tasks to run. The code in the script task is above. I also set the read only variables to strChildErrorFile, strFileExt, and strInputFolder.

This code will create a file that the master package will be able to see to tell there was an error. Notice I am using a file stream to create the file and I close the FileStream. If you do not do this you will get an error in the master package saying the file is in use in another process.

If you want the file to be an all or none load, set up a transaction on the Child package.

    Public Sub Main()

        ‘create script variable to hold package variable values
        Dim strBackupFolder As String = Dts.Variables(“strBackupFolder”).Value
        Dim strChildErrorFile As String = Dts.Variables(“strChildErrorFile”).Value
        Dim strFileExt As String = Dts.Variables(“strFileExt”).Value
        Dim strFileName As String = Dts.Variables(“strFileName”).Value
        Dim strInputFolder As String = Dts.Variables(“strInputFolder”).Value

        ‘create error file and backup file full file names
        Dim strErrorFile As String = strInputFolder + strChildErrorFile + strFileExt
        Dim strInputFile As String = strInputFolder + strFileName + strFileExt
        Dim strBackupFile As String = strBackupFolder + strFileName + strFileExt

        ‘check for the child error file, move bad file if it exist
        If File.Exists(strErrorFile) Then
            File.Move(strInputFile, strBackupFile)
            File.Delete(strErrorFile)
        End If

        Dts.TaskResult = ScriptResults.Success
    End Sub

    Back in the master package there is a script task after the child package that will check for the existence of the bad file. This code is in the left pane.
The top section is just saving all of the package variables in to VB script varables.

The second section is combining some of these variables to get the file names.
The If Then section is checking for the existence of the error file that might have been created by the child package. If the error file does exist it moves the input file to the bad files folder and deletes the error file.
This allows the master package to go to the next file in the loop and allows the next child package to create a new error file if needed.

No comments:

Post a Comment