Common SSIS Problems and Solutions
Problem:Loop over a list of files & load each one
Tasks Required: Foreach Loop, Data Flow Task
Solution:
Configure the Foreach Loop to loop over any particular directory of files. The loop should be configured to output
to a given variable. Map the given variable to a connection manager by using expressions.
Problem:
Conditionally executing tasks
Solution:
Double-click the precedence constraint and set the Evaluation property to Expression and Constraint.
Type the condition that you want to evaluate in the Expression box.
Problem:
Pass in variables when scheduling or running a package
Solution:
Use the /SET command in the DTExec command line or change the Property tab in the Package Execution Utility to have the property path like:
\Package.Variables[User::VariableName].Properties[Value]
Problem:
Move and rename the file at the same time
Tasks Required:
File System Task
Solution:
Set the File System task to rename the file and point to the directory you'd like to move the file to. This enables you to rename and move the file in the same step.
Problem:Loop over an array of data in a table & perform a set of tasks for each row
Tasks Required: Execute SQL Task, Foreach Loop
Solution:
Use an Execute SQL Task to load the array and send the data into an object variable. Loop over the variable in a Foreach Loop by use an ADO Enumerator.
Problem:
Perform an incremental load of data
Tasks Required: 2 Execute SQL Tasks, Data Flow Task
Solution:
Have the 1st Execute SQL Task retrieve a date from a control table of when the target table was last loaded and place that into a variable.
In the Data Flow Task, create a date range on your query using the variable. Then, update the control table using a 2nd Execute SQL Task to specify when the table was
last updated.
Problem:
Perform a conditional update & insert
Components Required: Data Flow Task, Conditional Split, Lookup Transform or Merge Join, OLE DB Command Transform
Solution:
Use the lookup Transform or Merge Join to detemine if the row exists on the destination and ignore a failed match. If the row yields blank
on the key, then you know the row should be inserted into the target (by Conditional Split). Otherwise, the row is a duplicate or an update. Determine if the row
is an update by comparing the source value to the target value in the Conditional Split. The update can be done by an OLE DB Command Transform or by loading the data
into a staging table.
Problem:Loop over a list of files & load each one
Tasks Required: Foreach Loop, Data Flow Task
Solution:
Configure the Foreach Loop to loop over any particular directory of files. The loop should be configured to output
to a given variable. Map the given variable to a connection manager by using expressions.
Problem:
Conditionally executing tasks
Solution:
Double-click the precedence constraint and set the Evaluation property to Expression and Constraint.
Type the condition that you want to evaluate in the Expression box.
Problem:
Pass in variables when scheduling or running a package
Solution:
Use the /SET command in the DTExec command line or change the Property tab in the Package Execution Utility to have the property path like:
\Package.Variables[User::VariableName].Properties[Value]
Problem:
Move and rename the file at the same time
Tasks Required:
File System Task
Solution:
Set the File System task to rename the file and point to the directory you'd like to move the file to. This enables you to rename and move the file in the same step.
Problem:Loop over an array of data in a table & perform a set of tasks for each row
Tasks Required: Execute SQL Task, Foreach Loop
Solution:
Use an Execute SQL Task to load the array and send the data into an object variable. Loop over the variable in a Foreach Loop by use an ADO Enumerator.
Problem:
Perform an incremental load of data
Tasks Required: 2 Execute SQL Tasks, Data Flow Task
Solution:
Have the 1st Execute SQL Task retrieve a date from a control table of when the target table was last loaded and place that into a variable.
In the Data Flow Task, create a date range on your query using the variable. Then, update the control table using a 2nd Execute SQL Task to specify when the table was
last updated.
Problem:
Perform a conditional update & insert
Components Required: Data Flow Task, Conditional Split, Lookup Transform or Merge Join, OLE DB Command Transform
Solution:
Use the lookup Transform or Merge Join to detemine if the row exists on the destination and ignore a failed match. If the row yields blank
on the key, then you know the row should be inserted into the target (by Conditional Split). Otherwise, the row is a duplicate or an update. Determine if the row
is an update by comparing the source value to the target value in the Conditional Split. The update can be done by an OLE DB Command Transform or by loading the data
into a staging table.
No comments:
Post a Comment