Welp, you're right. Sounds like a design flaw.
Perhaps it's made this way bc only step 5 knows if it should continue to additional step 6, and only step 6 knows if it passes criteria already or needs to go to step 7, etc.
Hmm. The way it overall knows it's finished is when there are no incomplete subtasks. But the criteria for propagating subtasks is at the subtask level.
I probably can't actually change whole structure, but I can recommend moving a "commit" around.
Just a wild guess but could a race condition between a subtask marking itself complete and your “supervisor” process that checks if no subtasks are incomplete.
So when subtask 3 for example marks itself complete if your “supervisor” checks before your subtask 4 is added it would see all subtasks complete. A bandaid could be only marking a subtask complete after the next subtask is inserted.
Sounds like you’re using a DB to track the jobs. Is that because your workers are distributed?
I'm hoping for the bandaid problem. I'm getting ready to pop open the code now.
It's a DB... I think it's a DB because it's interfacing with web pages that could be open in different locations by multiple people at once.
I'm not a native computer programmer so I wouldn't know what another solution would look like. I just take the code as it was created by others and assume it's the best solution and just try to find the bandaid problems.
