v1.21.1

Release notes

This hotfix release is a recommended update for all network operators and addresses the invalid block production issue initially reported on the Gnosis chain from October 19 to 20.

⚠️ IMPORTANT

Since v1.21.0, the Nethermind executable has been renamed from Nethermind.Runner to nethermind
The Docker images entry points have been updated accordingly.
Please update your setups accordingly, if any.

The obsolete --config xdai option has been deprecated in favor of --config gnosis.
Also, to avoid syncing from scratch, rename the "xdai" directory to "gnosis" in the Nethermind database directory (by default, "nethermind_db").

Changelog

• Bad blocks - potential fix by @MarekM25 in #6212
• Fix on handshake failure not disconnecting connection by @asdacap in #6179

Full Changelog: 1.21.0...1.21.1

v1.21.0

Release notes

⚠️ IMPORTANT

• The Nethermind executable has been renamed from Nethermind.Runner to nethermind
• The Nethermind Launcher executable has been renamed from Nethermind.Launcher to nethermind-launcher
• The Nethermind CLI executable has been renamed from Nethermind.Cli to nethermind-cli

The Docker images entry points have been updated accordingly.
Please update your setups accordingly, if any.

Major highlights

• Unified Nethermind executable names
  As per the above warning — Nethermind executable names have been changed to be the same across all distribution channels. This may be a breaking change, so please pay extra attention and ensure your scripts are adjusted accordingly.

• Removed obsolete xDai configs
  The configuration options --config xdai and --config xdai_archive has been removed in favor of --config gnosis and --config gnosis_archive accordingly.

  ⚠️
  Users still using the obsolete xdai moniker also need to rename the xdai directory to gnosis that is located in the Nethermind database directory (by default, nethermind_db) to avoid syncing from scratch.

• Trie Recovery functionality

  • This new mechanism should recover a database when there is a recognized case with a missing node. It will download necessary data from the network, and after a very short downtime database should be repaired.
  • Please read more at #5861

• Throttling for JSON-RPC calls to ensure the stability of a node under a high-load

  • Together with the Gateway team, which uses plenty of Nethermind clients to serve JSON-RPC for Gnosis, we noticed that when flooding with RPC calls that take long to execute, the call queue gets clogged, causing the client to stall.
  • Added new configuration option: --JsonRpc.RequestQueueLimit with the default value of 500, which causes all requests above that limit to be canceled immediately. This may result in some limitations on requests queued (can be adjusted accordingly to your needs), but in our internal testing, it allowed the node to process successfully twice as many calls because of reduced traffic on a single node.
  • At first, PR applied for eth_* calls and later expanded to all JSON-RPC calls.

• Significant reduction of OldBodies sync time

  • During our internal testing, we found out that one of the configuration options wasn’t properly enabled.
  • Enabling it caused SIGNIFICANT improvements in OldBodies phase. Examples below:
    • Linux AMD machine (high-end) - 3h 45min to 1h 40min
    • Arm AWS Graviton3 machine - 19h to 2h
    • Windows BareMetal machine with limited network bandwidth - From 14h for 65% OldBodies (and crashed) to 3,5h in total

• Improved sync time with better DB tuning option

  • Old Headers sync faster on AMD machine by 18,8% and on ARM by 18,9%
  • Snap Sync faster on AMD machine by 27,8% and on ARM by 52%
  • Old Receipts faster on AMD machine by 12,5% and o ARM by 16,3%

• Cancun progress update

  • Majority of implementation is already merged into master branch
  • Team is transitioning into more testing-oriented approach focusing on devnets and better tooling for internal/external testing of Cancun EIPs

• Support for new Holesky chain

  • This release supports the new testnet in space: Holesky. More details can be found here: https://github.com/eth-clients/holesky
  • To run Nethermind on new chain use:
    • For a Snap Sync node --config holesky.
    • For an archival mode setup --config holesky_archive.

Changelog

New features and Important changes

• Unify Nethermind executable name by @rubo in #5908
• Improve Nethermind executable rename handling by @rubo in #6039
• Trie recovery by @LukaszRozmej in #5861
• Throttle all RPC modules by @MarekM25 in #5957
• Throttle Eth_ module queue by @MarekM25 in #5945
• Remove xDai configuration by @rubo in #5978

Cancun

• Update BLS12-381 and EIP-4844 precompiles by @flcl42 in #5857
• Add shard blob tx gas calculations by @flcl42 in #5596
• Cancun fixes + EIP 4788 by @MarekM25 in #6009
• EIP-6780: SELFDESTRUCT only in same transaction by @smartprogrammer93 in #4704
• Decrease MaxSizeOfTxForBroadcast from 128KB to 4KB by @marcindsobczak in #5871
• [cancun] [eip6780] apply danno's clarification: disable eth burn on preexisting contract when inheritor equals contract address by @smartprogrammer93 in #6006
• Fix length calculations of blob-type txs by @marcindsobczak in #5884
• Update TSTORE and TLOAD opcode values to match Eip1153 updates by @Demuirgos in #5839
• Make Engine *V3s handle V3 requests only by @flcl42 in #5845
• selfdestruct in initcode should destroy by @smartprogrammer93 in #5913
• Rename data gas to blob gas by @flcl42 in #5967

Bug fixes and stability

• Add missing peer types to limits by @benaadams in #5838
• Fix/fix json stats report by @LukaszRozmej in #5831
• Fix/receipt migration not working with compact receipt by @asdacap in #5828
• Perf/smoother peer discovery by @asdacap in #5846
• Fix/disconnect deserialize exception by @asdacap in #5866
• Fix RocksDB dependency issues on macOS x64 by @rubo in #5883
• Optimize, clean up data gas calculations by @flcl42 in #5891
• Fix gas estimations by @emlautarom1 in #5973
• Fix receipt recovery & migration by @emlautarom1 in #5989
• Fix sync pivot not set when dbload by @asdacap in #6080
• Fix old blocks downloaded beyond old barrier by @asdacap in #6059
• SecondsPerSlot in eth_estimateGas by @MarekM25 in #6037
• Fix fast header may miss a retried batch by @asdacap in #6066
• Fix accesList handling on Transactions by @emlautarom1 in #6031
• Return full REVERT error message by @emlautarom1 in #6030
• Fix/missing state root on start by @asdacap in #6055
• [Fix][PivotUpdator] Update receipts and bodies Barriers by @smartprogrammer93 in #6054

Performance

• Reduce Evm Stackspace clearing by @benaadams in #5826
• Adjust snap request threshold by @asdacap in #5842
• Prefer code from code cache for Create by @benaadams in #5854
• Pass ProcessingBranch via in by @benaadams in #5858
• Skip alloc for zero sstore by @benaadams in #5867
• Receipt clean up in batch by @benaadams in #5870
• Use ready to run by @benaadams in #5873
• Feature/disable tx gossip during sync by @LukaszRozmej in #5850
• Evm optimisations by @benaadams in #5877
• Use AllocateUninitializedArray for LargerArrayPool by @benaadams in #5909
• Feature/bodies and receipts request sizer by @asdacap in #5994
• Perf/optimize old bodies memory by @asdacap in #6008
• Fix blob files not enabled by default by @asdacap in #6060

Logging

• Don't use decimal percentages in peers report by @benaadams in #5841
• Only output Changing state if state actually changes by @benaadams in #5840
• Use better processing timer for logging by @benaadams in #5843
• Don't use DarkGray by @benaadams in #5849
• Even more logging improvements. by @LukaszRozmej in #5851
• Remove warning in failed JSON RPC call, when still syncing relevant data by @LukaszRozmej in #5853
• Make missing root hash exception clearer by @asdacap in https://github.com/Nether...

v1.20.4

Release notes

This release supports the new testnet in space: Holesky

Due to the unsuccessful initial launch of the Holesky network, the date has been rescheduled. Below are the updated details:

• Launch date: September 28, 2023, 12:00 UTC
• Epoch time: 1695902400
• Network id: 17000

More details can be found here: https://github.com/eth-clients/holesky

To use Nethermind with this new network, follow these steps:

1. Run Nethermind with new config:

   • For a Snap Synced node, use the flag --config holesky.
   • For an archival mode setup, use the flag --config holesky_archive.

2. Run a consensus client that supports the Holesky network.

Changelog

Holesky

• [Fix][Holesky] timestamps and genesis updates by @smartprogrammer93 in #6112

Full Changelog: 1.20.3...1.20.4

v1.20.3

Release notes

Fixed the configuration issue in v1.20.2 which caused an error in engine JSON-RPC when executed by CL clients:

eth_chainId found but the containing module is disabled for the url 'http://localhost:8551', consider adding module in JsonRpcConfig.AdditionalRpcUrls for additional url, or to JsonRpcConfig.EnabledModules for default url

Please read the full v1.20.2 changelog here.

Changelog

• Change EngineEnabledModules to be comma separated by @kamilchodola in #6057

Thanks @antondlr for reporting the issue.

Full Changelog: 1.20.1...1.20.3

v1.20.2

Release notes

⚠️ This release is replaced with v1.20.3 because of the issue in the holesky.cfg and holesky_archive.cfg which caused issues with proper Engine JSON-RPC usage by CL.

This release is aimed for support of new testnet chain in space: Holesky

A few details about Holesky:

• Launch Date: Sept/15, 2023, 14:00 UTC
• Epoch time: 1694786400
• LTS: Dec/2027, EOL: Dec/2028
• Network ID: 17000

More details can be found here: https://github.com/eth-clients/holesky

To use Nethermind with this new chain, follow these steps:

1. Run nethermind with new config:

   • For a Snap Synced node, use the flag --config holesky.
   • For an archival mode setup, use the flag --config holesky_archive.

2. Run a Consensus Layer Client of your choice which supports holesky network.

3. Enjoy!

Changelog

Holesky

• Add holesky configs and chainspec by @smartprogrammer93 in #6034

Other changes

• Fix PPA package post-install script by @rubo in #6045

Full Changelog: 1.20.1...1.20.2

v1.20.1

Release notes

Fixed the regression in v1.20.0 which caused an error in Prysm:

Could not reconstruct full bellatrix block batch from blinded bodies

No re-sync is required.

Additionally, we added a support for ARM64 PPA package, so now Nethermind can be installed on this architecture as well.

Please read the full v1.20.0 changelog here.

Changelog

• Restore v in tx signature for Geth compatibility by @rubo in #5937

Thanks @nazar-kuzo for reporting the issue.

Full Changelog: 1.20.0...1.20.1

v1.20.0

Release notes

⚠️ This release is replaced with v1.20.1 because of the issue in the eth_getBlockByHash JSON-RPC method which cause problems on Prysm side.

Major highlights

• Logs readability improvements. Now with color support, improved log organization, and the prominent display of vital information related to block production, it's easier than ever to have a comprehensive understanding of the current head of the chain and all necessary details:

  Break down of new logs:

  

  • JsonRPC logs are turned off by default.
  • Users of older versions of Windows may need to set registry entry [HKEY_CURRENT_USER\Console\VirtualTerminalLevel] to DWORD 1 to enable colors if using cmd. If colors are too dark they can be adjusted in your terminal color settings.

• Boosted block processing priority to improve attestation rates during sync

• Improvements in sync (faster Snap Sync, Old Bodies and Old Receipts) + reduced SSD writes requirement

• Full Pruning improvements (better threads management, free disk space check for full pruning, pruning time reduced etc).

• This is the Shapella-ready release for the upcoming hard-fork on Gnosis!
  The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changelog

Gnosis

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848
• Add more Gnosis bootnodes by @MarekM25 in #5910

Cancun

• Add RLP encoding and hashing for Blob txs by @flcl42 in #5593
• Add path to kzg setup file by @deffrian in #5797
• EIP-4844 V6 fixes and improvements by @flcl42 in #5745

Bug fixes and stability

• Improve peer discovery and connectivity by @asdacap in #5846
• Update DotNetty and RocksDB packages (fix issue with installation of Homebrew on MacOS AMD machines) by @rubo in #5883
• Remove v of non-legacy tx signature from RPC response (#5927) by @rubo in #5927
• Safety check for full pruning by @deffrian in #5550
  Example usage:
  Now the client will check available disk space to ensure that machine is capable to successfully execute full pruning. To disable this feature set --Pruning.AvailableSpaceCheckEnabled=false
• Don’t hide precompile contract exception by @asdacap in #5679
• Fixed pivot-related edge case that could lead to node corruption by @MarekM25 in #5724
• Copy timestamp from json by @deffrian in #5731
• Fix stuck state from period offline by @benaadams in #5710
• Fix point evaluation precompile by @deffrian in #5739
• Fixed eip2537 activation by @MarekM25 in #5824
• Quick fix for FullPruner by @marcindsobczak in #5809
• Bugfix - exchangeCapabilites based on ChainSpec by @MarekM25 in #5801
• Fix estimate gas by @deffrian in #5800
• Fix gas fees accounting in TransactionProcessor.Trace by @deffrian in #5743
• Fix KeccaksIterator not working when some keccak zero prefix is not i… by @asdacap in #5780
• Fixed fail on precompile dll import error by @Demuirgos in #5725.
• MaxDegreeOfParallelism defaults for full pruning by @MarekM25 in #5662
  • Introduced few more logs to better navigate through Full pruning process.
  • Revised parameters for --Pruning.FullPruningMaxDegreeOfParallelism parameter
    • -1 - number of logical processors
    • 0 - (DEFAULT) 25% of logical processors. This would reduce overhead of hardware and will give better results of attestations during that process.
    • 1 - In this example, 1 logical processor will be used. Set any value higher than 0
  • Revised parameter --Pruning.FullPruningMemoryBudgetMb and changed the default to 4000 MB

Logging

• Improved colour logs + block tx gas gwei range by @benaadams in #5770
• Move detailed peer logging to debug, add diversity by @benaadams in #5795
• JSON-RPC logs from Info to Debug by @MarekM25 in #5782

Performance

• Boost Block Processor priority to improve attestations during syncing by @benaadams in #5764
• Apply cache miss hint which improves full pruning perf by @asdacap in #5723
• Improve db priorities which improves snap sync perf by @asdacap in #5722
• Apply auto tune to compressed db by @asdacap in #5771
• Perf/valuekeccak snap code by @asdacap in #5689
• Optimise Account access and contract detection patterns in Evm by @benaadams in #5646
• Feature/disable WAL flag which will reduce SSD write requirements during SnapSync phase by @asdacap in #5702
• Perf/keccak with valuekeccak which results with reduced memory and CPU usage by @asdacap in #5715
• Perf/faster block load for receipt by @asdacap in #5708
• Reduce exceptions in Evm by @benaadams in #5727
• Keccaks: Skip unneeded array allocations by @benaadams in #5735
• Less contention for OldReceipts and OldBodies by @benaadams in #5737
• Add option to have multiple network processing thread by @asdacap in #5749
  Example usage:
  In case internet speed is higher than 500Mbps, there is option to set flag --Network.ProcessingThreadCount 32 which may reduce Old Bodies sync phase time
• Less aggressive peer discovery by @benaadams in #5747
• Reuse batch Dictionary and skip empty batches by @benaadams in #5802
• Don't add Keccaks during cache pruning by @benaadams in #5781
• Optimize Snap sync + Sort DB Batches by @benaadams in #5789
• Old Bodies sync performance improvement with blobfiles by @asdacap in #5785
• Add fast test for balance < tx.Value + gas by @benaadams in #5783
• Remove from ConcurrentDict directly rather than building secondary list by @benaadams in #5779
• Improve peering management strategy which results with faster sync time by @asdacap in #5772
• Refactor/Split dispatcher and downloader by @asdacap in #5778
• Reduce datetime calls in peer discovery by factor of 100 by @benaadams in #5751
• More efficient EvmPooledMemory by @benaadams in #5752

Other changes

• refactor state and storage behind one interface by @tanishqjasoria in #5659
• Address build warnings in Nethermind.Serialization.Ssz.Test.csproj by @MarekM25 in #5695
• Add support for specifying address in CLI by @emlautarom1 in #5690
• Move some logic from blocktree into chain level info by @asdacap in #5705
• Refactor/extract block store from blocktree by @asdacap in #5704
• One more Engine API test: Blocks_before_pivots_should_not_be_added by @MarekM25 in #5719
• [README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors by @kayla-henrie in #4278
• Revise README by @rubo in #5728
• Update Nethermind.Crypto.SecP256k1 package by @rubo in #5733
• Update Nethermind.Crypto.Pairings package by @rubo in #5742
• Cleanup/remove keccak key by @asdacap in #5748
• Fix/inconsistently failing tests by @asdacap in #5773
• Added GnosisSpecProvider and gnosis_loads_properly test by @MarekM25 in #5775
• Rename BlockChain -> Bl...

v1.19.3

Release notes

For the Gnosis network operators, this is a mandatory update because of the upcoming Shapella hard-fork. We urge you to update promptly to ensure seamless network functioning post-fork.

The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changes

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848

Full Changelog: 1.19.2...1.19.3

v1.20.0-rc

Release notes

Major highlights

• Logs readability improvements. Now with color support, improved log organization, and the prominent display of vital information related to block production, it's easier than ever to have a comprehensive understanding of the current head of the chain and all necessary details:

  Break down of new logs:

  

  • JsonRPC logs are turned off by default.
  • Users of older versions of Windows may need to set registry entry [HKEY_CURRENT_USER\Console\VirtualTerminalLevel] to DWORD 1 to enable colors if using cmd. If colors are too dark they can be adjusted in your terminal color settings.

• Boosted block processing priority to improve attestation rates during sync

• Improvements in sync (faster Snap Sync, Old Bodies and Old Receipts) + reduced SSD writes requirement

• Full Pruning improvements (better threads management, free disk space check for full pruning, pruning time reduced etc).

• This is the Shapella-ready release for the upcoming hard-fork on Gnosis!
  The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changelog

Gnosis

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848

Cancun

• Add RLP encoding and hashing for Blob txs by @flcl42 in #5593
• Add path to kzg setup file by @deffrian in #5797
• EIP-4844 V6 fixes and improvements by @flcl42 in #5745

Bug fixes and stability

• Safety check for full pruning by @deffrian in #5550
  Example usage:
  Now the client will check available disk space to ensure that machine is capable to successfully execute full pruning. To disable this feature set --Pruning.AvailableSpaceCheckEnabled=false
• Don’t hide precompile contract exception by @asdacap in #5679
• Fixed pivot-related edge case that could lead to node corruption by @MarekM25 in #5724
• Copy timestamp from json by @deffrian in #5731
• Fix stuck state from period offline by @benaadams in #5710
• Fix point evaluation precompile by @deffrian in #5739
• Fixed eip2537 activation by @MarekM25 in #5824
• Quick fix for FullPruner by @marcindsobczak in #5809
• Bugfix - exchangeCapabilites based on ChainSpec by @MarekM25 in #5801
• Fix estimate gas by @deffrian in #5800
• Fix gas fees accounting in TransactionProcessor.Trace by @deffrian in #5743
• Fix KeccaksIterator not working when some keccak zero prefix is not i… by @asdacap in #5780
• Fixed fail on precompile dll import error by @Demuirgos in #5725.
• MaxDegreeOfParallelism defaults for full pruning by @MarekM25 in #5662
  • Introduced few more logs to better navigate through Full pruning process.
  • Revised parameters for --Pruning.FullPruningMaxDegreeOfParallelism parameter
    • -1 - number of logical processors
    • 0 - (DEFAULT) 25% of logical processors. This would reduce overhead of hardware and will give better results of attestations during that process.
    • 1 - In this example, 1 logical processor will be used. Set any value higher than 0

Logging

• Improved colour logs + block tx gas gwei range by @benaadams in #5770
• Move detailed peer logging to debug, add diversity by @benaadams in #5795
• JSON-RPC logs from Info to Debug by @MarekM25 in #5782

Performance

• Boost Block Processor priority to improve attestations during syncing by @benaadams in #5764
• Apply cache miss hint which improves full pruning perf by @asdacap in #5723
• Improve db priorities which improves snap sync perf by @asdacap in #5722
• Apply auto tune to compressed db by @asdacap in #5771
• Perf/valuekeccak snap code by @asdacap in #5689
• Optimise Account access and contract detection patterns in Evm by @benaadams in #5646
• Feature/disable WAL flag which will reduce SSD write requirements during SnapSync phase by @asdacap in #5702
• Perf/keccak with valuekeccak which results with reduced memory and CPU usage by @asdacap in #5715
• Perf/faster block load for receipt by @asdacap in #5708
• Reduce exceptions in Evm by @benaadams in #5727
• Keccaks: Skip unneeded array allocations by @benaadams in #5735
• Less contention for OldReceipts and OldBodies by @benaadams in #5737
• Add option to have multiple network processing thread by @asdacap in #5749
  Example usage:
  In case internet speed is higher than 500Mbps, there is option to set flag --Network.ProcessingThreadCount 32 which may reduce Old Bodies sync phase time
• Less aggressive peer discovery by @benaadams in #5747
• Reuse batch Dictionary and skip empty batches by @benaadams in #5802
• Don't add Keccaks during cache pruning by @benaadams in #5781
• Optimize Snap sync + Sort DB Batches by @benaadams in #5789
• Old Bodies sync performance improvement with blobfiles by @asdacap in #5785
• Add fast test for balance < tx.Value + gas by @benaadams in #5783
• Remove from ConcurrentDict directly rather than building secondary list by @benaadams in #5779
• Improve peering management strategy which results with faster sync time by @asdacap in #5772
• Refactor/Split dispatcher and downloader by @asdacap in #5778
• Reduce datetime calls in peer discovery by factor of 100 by @benaadams in #5751
• More efficient EvmPooledMemory by @benaadams in #5752

Other changes

• refactor state and storage behind one interface by @tanishqjasoria in #5659
• Address build warnings in Nethermind.Serialization.Ssz.Test.csproj by @MarekM25 in #5695
• Add support for specifying address in CLI by @emlautarom1 in #5690
• Move some logic from blocktree into chain level info by @asdacap in #5705
• Refactor/extract block store from blocktree by @asdacap in #5704
• One more Engine API test: Blocks_before_pivots_should_not_be_added by @MarekM25 in #5719
• [README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors by @kayla-henrie in #4278
• Revise README by @rubo in #5728
• Update Nethermind.Crypto.SecP256k1 package by @rubo in #5733
• Update Nethermind.Crypto.Pairings package by @rubo in #5742
• Cleanup/remove keccak key by @asdacap in #5748
• Fix/inconsistently failing tests by @asdacap in #5773
• Added GnosisSpecProvider and gnosis_loads_properly test by @MarekM25 in #5775
• Rename BlockChain -> Blockchain in Merge tests by @MarekM25 in #5806
• Add a master build docker image for each commit on master branch by @kamilchodola in #5777
• Add tests validating a slot by timestamp by @rubo in #5792
• Refactoring of chainspec tests & handle chainspec exceptions by @MarekM25 in #5819
• Transaction processor refactor by @jmederosalvarado in #5799

New Contributors

• @emlautarom1 made their first contribution in #5690
• @kayla-henrie made their first contribution in #4278

**F...

v1.19.2

Release notes

This is an important release for Teku users.

In order to ensure proper functionality with the upcoming Teku release 23.6.0, it is necessary to upgrade Nethermind from version 1.19.x to version 1.19.2. Nethermind 1.19.2 or higher will be compatible with future Teku releases. If you are using a different consensus client or an older Teku version, there is no need to update your node.

This release addresses an issue with engine_exchange capabilities (PR #5787 by @flcl42).

Nethermind versions 1.17 and 1.18 do not require this fix. However, we recommend updating them to the newest version.