[SPIKE] Better dev UX with simulations

[alpe]

Description

🚧 WIP - early version

Registration example

func (am AppModule) WeightedOperations(simState module.SimulationState) []simtypes.WeightedOperation {
    ak, bk := am.accountKeeper, am.keeper
    reg := simsx.NewSimsRegistryAdapter(&simsx.BasicSimulationReporter{}, ak, bk, simState.TxConfig)
    weight := simsx.ParamWeightSource(simState.AppParams)
    reg.Add(weight.Get("msg_send", 100), simulation.MsgSendFactory(bk))
    reg.Add(weight.Get("msg_multisend", 10), simulation.MsgMultiSendFactory(bk))
    return reg.ToLegacyWeightedOperations()
}

Message factory example

func MsgSendFactory(bk keeper.Keeper) simsx.SimMsgFactoryFn[*types.MsgSend] {
	return func(ctx context.Context, testData *simsx.ChainDataSource, reporter simsx.SimulationReporter) ([]simsx.SimAccount, sdk.Msg) {
		from := testData.AnyAccount(reporter, simsx.WithSpendableBalance())
		to := testData.AnyAccount(reporter, simsx.ExcludeAccounts(from))
		coins := from.LiquidBalance().RandSubsetCoins(reporter, simsx.WithSendEnabledCoins(ctx, bk))
		return []simsx.SimAccount{from}, types.NewMsgSend(from.AddressBech32, to.AddressBech32, coins)
	}
}

or mixed with old sims setup

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• included the correct type prefix in the PR title
• confirmed ! in the type prefix if API or client breaking change
• targeted the correct branch (see PR Targeting)
• provided a link to the relevant issue or specification
• reviewed "Files changed" and left comments if necessary
• included the necessary unit and integration tests
• added a changelog entry to CHANGELOG.md
• updated the relevant documentation or specification, including comments for documenting Go code
• confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed all author checklist items have been addressed
• reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

[coderabbitai]

Important

Review Skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[raynaudoe]

Hi @alpe, happy to use and give some feedback to this new feature!
Question first for some context, this PR aims to replace the WeightedOperation with this new MsgFactory for each msg in each module?

[alpe]

Hi @alpe, happy to use and give some feedback to this new feature! Question first for some context, this PR aims to replace the WeightedOperation with this new MsgFactory for each msg in each module?

Yes, this is correct. The WeightedOperation come with some dependencies and are very powerful as they provide direct access to baseapp and support post operations. The use case for modules is much simpler though and can be boiled down to message factories only. This reduces the lines of code needed and complexity a lot.

I would suggest to support the old sims interface for backward compatibility of existing modules.

[raynaudoe]

Hi @alpe, happy to use and give some feedback to this new feature! Question first for some context, this PR aims to replace the WeightedOperation with this new MsgFactory for each msg in each module?

Yes, this is correct. The WeightedOperation come with some dependencies and are very powerful as they provide direct access to baseapp and support post operations. The use case for modules is much simpler though and can be boiled down to message factories only. This reduces the lines of code needed and complexity a lot.

I would suggest to support the old sims interface for backward compatibility of existing modules.

Nice, I do find also that the MsgFactory is much easier to implement and also they run much faster than previous ones.
I've a couple of questions:

• do you think that something similar could be done at the transaction level (if beneficial) ?
• since now testing sending module messages is easier and more "composable" I think it would be worth to choose certain test scenarios and check if this framework fits them. An idea for this would be testing scenarios of chain upgrades, I imagine something like:
  - Run simapp with with <previous> chain version
  - Export its state
  - Load the state into <next> version
  - Run the fuzzer from that state.

I used as an example chain upgrades like something that has the most potential to have "chain halting" bugs but could be anything that scores high in a risk analysis table.

The proposal to have a test at the transaction level I think it could be tricky but I see it beneficial to test out parameters like the unordered one in several scenarios.

[alpe]

Hi @raynaudoe,
good thinking.

• do you think that something similar could be done at the transaction level (if beneficial) ?

The sims framework works on the TX level already. They are submitted via Baseapp.SimDeliver(txEncoder sdk.TxEncoder, tx sdk.Tx). This give a lot of power but it comes with complexity that makes it hard for devs. There is no single sims operation that uses multi msg TX. My goal is to provide a better Dev experience so that we get a wider variation of messages , at this stage

• since now testing sending module messages is easier and more "composable" I think it would be worth to choose certain test scenarios and check if this framework fits them. An idea for this would be testing scenarios of chain upgrades, I imagine something like:

The bigger challenge is to provide the old code to run the sims for the initial state. You can check the DB into the repo but you would have to redo this for every new release as there is no history of state migrations available.
For a proper upgrade test, you want to check state and behaviour. This would be quite hard with sims.

But the good news is, that we have this covered by system tests. They spawn a multi node chain locally with some initial data, execute a chain upgrade proposal and run the new instance (main) on the DB. All as black box tests with the binary. You can adopt them for other chains as well.