Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[berkeley] Don't bootloop at startup for non-seed nodes #15609

Open
wants to merge 4 commits into
base: berkeley
Choose a base branch
from

Conversation

mrmr1993
Copy link
Member

@mrmr1993 mrmr1993 commented May 6, 2024

This PR fixes a bootloop at startup for nodes without the --seed flag, where nodes will detect that they're offline (i.e. receiving no messages) and restart every 5 minutes.

A 'mitigation' for this previously existed, but didn't work as intended for the reasons given in the newly-added comment. This code should eventually be removed.

This is currently untested. I am travelling and have been so far unable to.

@mrmr1993 mrmr1993 requested a review from a team as a code owner May 6, 2024 17:56
@mrmr1993
Copy link
Member Author

mrmr1993 commented May 6, 2024

Confirmed working as intended with the following command:

$ _build/default/src/app/cli/src/mina.exe daemon --config-file genesis_ledgers/mainnet.json --peer-list-url https://storage.googleapis.com/seed-lists/matt_test_seeds.txt
2024-05-06 18:24:07 UTC [Info] Mina daemon is booting up; built with commit "[DIRTY]f9e6c242f748a3aba3cfabc9362d70c8a38222e5" on branch "feature/no-bootloop-before-genesis"
2024-05-06 18:24:07 UTC [Info] Booting may take several seconds, please wait
2024-05-06 18:24:07 UTC [Info] Reading configuration files $config_files
        config_files: [ "/home/mathew/.mina-config/daemon.json", "genesis_ledgers/mainnet.json" ]
2024-05-06 18:24:07 UTC [Warn] Could not read configuration from "/home/mathew/.mina-config/daemon.json"
2024-05-06 18:24:07 UTC [Info] Initializing with runtime configuration. Ledger name: "mainnet"
2024-05-06 18:24:07 UTC [Info] Using the compiled constraint constants
2024-05-06 18:24:07 UTC [Info] Loaded genesis ledger from $ledger_file
        ledger_file: "/home/mathew/.mina-config/genesis/genesis_ledger_accounts_fe217061fe891753eec9838e1f4edc169ff6c7f28ad0e9628f2167e6716f5fec.tar.gz"
2024-05-06 18:24:07 UTC [Info] Cleaning up old epoch ledgers with genesis state $state_hash at locations $staking and $next
        state_hash: "3NKVWi3D9z3bJeG5pTbV32sZDSKVwAfaGi83ToX2pdHGCAudcoYq"
        staking: "/home/mathew/.mina-config/epoch_ledgerc2490d55-513e-3239-c227-b3e867c6bafc"
        next: "/home/mathew/.mina-config/epoch_ledger6634046c-1b6e-3e14-eb42-d5f2bd651bd8"
2024-05-06 18:24:25 UTC [Info] Daemon will use chain id 38ac2bb187ecc81307a4c23e914c202132f91dbb22360f340e15d312e1fddc97
2024-05-06 18:24:25 UTC [Info] Daemon running protocol version 3.0.0
2024-05-06 18:24:25 UTC [Info] Node started before genesis time "2024-05-06T18:35:30.000000Z". Sleeping for "3m"
2024-05-06 18:27:25 UTC [Info] Node started before genesis time "2024-05-06T18:35:30.000000Z". Sleeping for "3m"
2024-05-06 18:30:25 UTC [Info] Node started before genesis time "2024-05-06T18:35:30.000000Z". Sleeping for "2.083m"
2024-05-06 18:32:30 UTC [Info] Starting a new prover process
2024-05-06 18:32:49 UTC [Info] Daemon started process of kind "Prover" with pid 114141
2024-05-06 18:32:49 UTC [Info] Starting a new verifier process
2024-05-06 18:33:06 UTC [Info] Daemon started process of kind "Verifier" with pid 114168
2024-05-06 18:33:06 UTC [Info] Starting a new vrf-evaluator process
2024-05-06 18:33:06 UTC [Info] Daemon started process of kind "Vrf_evaluator" with pid 114229
Environment variable MINA_TIME_OFFSET not found, using default of 0
2024-05-06 18:33:07 UTC [Error] Invalid peer $peer found in peers list
        peer: "<?xml version='1.0' encoding='UTF-8'?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Details>No such object: seed-lists/matt_test_seeds.txt</Details></Error>"
2024-05-06 18:33:07 UTC [Info] Custom child process "libp2p_helper" started with pid 114236
2024-05-06 18:33:07 UTC [Error] libp2p_helper: failed to set log level debug for routing/record: error: No such logger
2024-05-06 18:33:07 UTC [Info] libp2p peer ID this session is $peer_id
        peer_id: "12D3KooWKQTRme83KA1mGpCVsLW5hHYNdxY6pujCZpC2oLtW2tHd"
2024-05-06 18:33:07 UTC [Info] Network not instantiated when node status requested
2024-05-06 18:33:07 UTC [Info] Starting transition router
2024-05-06 18:33:07 UTC [Warn] Node started before the chain start time: waiting 142175 milliseconds before starting participation
2024-05-06 18:33:07 UTC [Warn] GCLOUD_KEYFILE environment variable not set. Must be set to use upload_blocks_to_gcloud
2024-05-06 18:33:07 UTC [Info] Mina daemon is connecting
2024-05-06 18:33:07 UTC [Info] Initializing plugins
2024-05-06 18:33:07 UTC [Info] Not running uptime service, no URL given
2024-05-06 18:33:07 UTC [Info] Stopping daemon after 10620 mins and when there are no blocks to be produced
2024-05-06 18:33:07 UTC [Info] Attempted to turn on snark worker, but snark worker key is set to none
2024-05-06 18:33:07 UTC [Info] Daemon ready. Clients can now connect
2024-05-06 18:33:07 UTC [Info] Created GraphQL server at: http://localhost:3085/graphql
2024-05-06 18:33:07 UTC [Error] libp2p_helper: failed to routing advertise: failed to find any peer in table
2024-05-06 18:33:07 UTC [Warn] starting libp2p up failed: $error
        error: {
  "commit_id": "[DIRTY]f9e6c242f748a3aba3cfabc9362d70c8a38222e5",
  "string": "libp2p error: failed to find any peer in table"
}
$ head -n 5 -- genesis_ledgers/mainnet.json
{
  "genesis": {
    "genesis_state_timestamp": "2024-05-06T18:35:30Z"
  },
  "ledger": {

@@ -464,6 +464,16 @@ let create_sync_status_observer ~logger ~is_seed ~demo_mode ~net
let offline_shutdown_delay = Time.Span.of_min 25. in
let after_genesis =
let genesis_timestamp =
(* Horrifyingly, this completely disregards the timestamp from the
Copy link
Member

@emberian emberian May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is horrifying and indeed explains the mysterious behavior i have been seeing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants