Skip to content

Paused pipeline gets stuck in STOPPING #5697

@ryzhyk

Description

@ryzhyk
  • Start a transaction from a connector (e.g., delta)
  • Pause the pipeline before the connector commits
  • Call /stop

The checkpoint is blocked due to the ongoing transaction, which will never complete since the connector and the whole pipeline are paused; however the /resume call is rejected in this state because the pipeline is stopping. The only way to terminate the pipeline is to force-stop it.

A minimal solution would be to allow /resume while stopping. Not sure if the limitation is in the manger or the pipeline itself, so assigning to both @snkas and myself.

Metadata

Metadata

Assignees

Labels

APIDistributed system APIsconnectorsIssues related to the adapters/connectors crate

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions