Purpose of the pull request

close #13133

Like this :Flyter Caching

In machine learning workflow, if some tasks will be caching, the workflows will be executed faster

How to determine whether a task has been cached when the cache is executed, that is, how to determine whether a task can use the running result of another task?

For the task identified as Cache Execution, when the task starts, a cache key will be generated, and the key is composed of the following fields and hashed:

• task definition: the id of the task definition corresponding to the task instance
• task version: the version of the task definition corresponding to the task instance
• task input parameters: including the parameters passed in by the upstream node and the global parameter, the parameters referenced by the parameter list of the task definition and the parameters used by the task definition using ${}
• environment configuration: the actual configuration content of the environment configuration under the environment name, that is, the actual configuration content in the security - environment management

If the task with cache identification runs, it will find whether there is data with the same cache key in the database,

• If there is, copy the task instance and update the corresponding data
• If not, the task runs as usual, and the task instance data is stored in the cache when the task is completed

If you do not need to cache, you can right-click the node to run Clear cache in the workflow instance to clear the cache, which will clear the cache data of the current input parameters under this version.

Brief change log

Front end

• Add useCache flag to all task plugin except the logical component

• Add handleRemoveTaskInstanceCache method to task instance clear cache

Backend

• Add removeTaskInstanceCache API to clear task instance cache
• Add TaskCacheUtils to manage the cache key
• Add checkIsCacheExecution before dispatcher.dispatch task to worker
• Add TaskCacheEventHandler to the handle cache task instead dispatch task
• Add saveCacheTaskInstance after the cache task successfully run

Database

• add is_cache into table t_ds_task_definition、t_ds_task_definition_log、t_ds_task_instance
• add cache_key to t_ds_task_instance

Doc

• Add Cache Execution parameters introduction
• Add faq about task cache

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md