Workflows review 2nd iteration

[jost-s]

There are more parts of the workflows which need review.

• Incoming ops are immediately written to Action and Entry tables in DHT database, before validation has completed. [Workflows] Add action/entries to DHT DB only after successfully validating incoming OP #3724
• Ops are partly integrated during app validation. Ops with a "sys dependency" (= that reference another action like updates and deletes) are eventually integrated in the integration workflow which is triggered after app validation.
• There's a complex mechanism around sys dependencies as well which are kept in the dht_ops table. Likely it isn't required any longer as app validation processes and integrates all ops in sequence, so that ops with a dependency will only be integrated after the dependency has been integrated. Related PR Always integrate rejected ops immediately, and relax the timestamp sys validation rule #3277
• It's unclear what the integration workflow does and if all of it is still relevant. A clear separation of workflows would mean that app validation concludes evaluation of validity of an op (validation status) without integrating it. Integration of ops as well as sending validation receipts and any other follow-up action would be processed in the integration workflow.
• Actions written to the source chains should automatically be published. At the moment they are written to the DHT database, but not published.

[artbrock]

It's unclear what the integration workflow does and if all of it is still relevant.

It looks like you added some of what it does later in this bullet. But yes, it is still relevant. We want the validation workflow to be complete as soon as the validation status fully determined (valid / rejected).

However, there are follow-up async actions that should not be a part of the validation workflow, such as sending validation receipts and/or warrants. So some workflow must follow validation, whether it is called "integration" or not can be discussed.

Also, I want to note that a timestamp for the moment of final validation / integration is vital to the integrity of gossip which should be just asking for "what's new since time x" as your last integration timestamp you've already gossiped with them. This is something we should confirm is happening.

[jost-s]

It looks like you added some of what it does later in this bullet. But yes, it is still relevant. We want the validation workflow to be complete as soon as the validation status fully determined (valid / rejected).

I'm referring to the status quo. There are 4 database updates it performs and it's not obvious what they're for.

Upon checking now I see that the integration workflow triggers a validation receipt workflow. That should be part of the review too.