Workflow Recovery

Task Recovery Strategies

Each task in a workflow has a recovery strategy. When the Integration Service recovers a workflow, it recovers tasks based on the recovery strategy:
¨ Restart task. When the Integration Service recovers a workflow, it restarts each recoverable task that is
configured with a restart strategy. You can configure Session and Command tasks with a restart recovery
strategy. All other tasks have a restart recovery strategy by default.
¨ Fail task and continue workflow. When the Integration Service recovers a workflow, it does not recover the
task. The task status becomes failed, and the Integration Service continues running the workflow.
Configure a fail recovery strategy if you want to complete the workflow, but you do not want to recover the task.
You can configure Session and Command tasks with the fail task and continue workflow recovery strategy.
¨ Resume from the last checkpoint. The Integration Service recovers a stopped, aborted, or terminated
session from the last checkpoint. You can configure a Session task with a resume strategy.

 

Session Task Strategies

When you configure a session for recovery, you can recover the session when you recover a workflow, or you can
recover the session without running the rest of the workflow.
When you configure a session, you can choose a recovery strategy of fail, restart, or resume:

¨ Resume from the last checkpoint.The Integration Service saves thesession state of operationand
maintains target recovery tables.If the session aborts, stops, or terminates, the Integration Service uses the saved recovery information to resume the session from the point of interruption.
When you validate a session, the Workflow Manager verifies that the transformations are configured to produce repeatable and deterministic data.The session is not valid if you configure recovery to resume from the last checkpoint, but the transformations are not configured for repeatable and deterministic data.

You cannot configure a session with a resume strategy if it uses mapping variables.

¨ Restart task. The Integration Service runs the session again when it recovers the workflow.When you recover with restart task, you might need to remove the partially loaded data in the target or design a mapping to skip the duplicate rows.

¨ Fail task and continue workflow. When the Integration Service recovers a workflow, it does not recover the
session. The session status becomes failed, and the Integration Service continues running the workflow.
Configure the recovery strategy on the Properties page of the Session task.

 

Resuming Sessions

When you configure session recovery to resume from the last checkpoint, the Integration Service creates
checkpoints in $PMStorageDir to determine where to start processing session recovery. When the Integration
Service resumes a session, it restores the session state of operation, including the state of each source, target,
and transformation. The Integration Service determines how much of the source data it needs to process.

When the Integration Service resumes a session, the recovery session must produce the same data as the original session.The session is not valid if you configure recovery to resume from the last checkpoint, but the session cannot produce repeatable data.

You can configure a mapping to enable transformations in the session to produce the same data between the session and recovery run. When a mapping contains a transformation that never produces repeatable data, you can add a transformation that always produces repeatable data immediately after it.

 

When you recover a session from the last checkpoint, the Integration Service restores the session state of
operation to determine the type of recovery it can perform:
¨ Incremental. The Integration Service starts processing data at the point of interruption. It does not read or
transform rows that it processed before the interruption. By default, the Integration Service attempts to perform
incremental recovery.
¨ Full. The Integration Service reads all source rows again and performs all transformation logic if it cannot
perform incremental recovery. The Integration Service begins writing to the target at the last commit point. If
any session component requires full recovery, the Integration Service performs full recovery on the session.

 

You can recover a workflow if the Integration Service can access the workflow state of operation. The workflow state of operation includes the status of tasks in the workflow and workflow variable values. The Integration Service stores the state in memory or on disk, based on how you configure the workflow:
¨ Enable recovery. When you enable a workflow for recovery, the Integration Service saves the workflow state of operationin a shared location. You can recover the workflow if it terminates, stops, or aborts. The workflow does not have to be running.
¨ Suspend. When you configure a workflow to suspend on error, the Integration Service stores the workflow state of operationin memory. You can recover the suspended workflow if a task fails. You can fix the task error and recover the workflow.

 

State of Operation

When you recover a workflow or session, the Integration Service restores the workflow or session state of operation to determine where to begin recovery processing. The Integration Service stores the workflow state of operation in memory or on disk based on the way you configure the workflow. The Integration Service stores thesession state of operationbased on the way you configure the session.

 

Rules and Guidelines for Session Recovery

recovery功能有很多限制条件（尤其是Session的resume from the last checkpoint选项），很多种情况都不能recovery或者recovery后会出现前后数据不一致等其他问题。（Pipeline Partitioning也是限制条件过多，要注意的事项细节太多）