Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: generalize blobs to data for extension ready for ils and/or data columns #6693

Open
wants to merge 1 commit into
base: unstable
Choose a base branch
from

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Apr 21, 2024

to avoid redoing the same work on different branches and have a clean way to extend data requirements

Copy link
Contributor

github-actions bot commented Jun 1, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c89554a Previous: 8b6ecc4 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 992.22 us/op 938.75 us/op 1.06
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 48.782 us/op 48.287 us/op 1.01
BLS verify - blst-native 1.1826 ms/op 1.1825 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 2.4553 ms/op 2.4792 ms/op 0.99
BLS verifyMultipleSignatures 8 - blst-native 5.2607 ms/op 5.3882 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 19.326 ms/op 19.694 ms/op 0.98
BLS verifyMultipleSignatures 64 - blst-native 38.140 ms/op 38.632 ms/op 0.99
BLS verifyMultipleSignatures 128 - blst-native 75.575 ms/op 75.759 ms/op 1.00
BLS deserializing 10000 signatures 865.36 ms/op 860.86 ms/op 1.01
BLS deserializing 100000 signatures 8.6961 s/op 8.8004 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.2527 ms/op 1.2651 ms/op 0.99
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.4793 ms/op 1.4332 ms/op 1.03
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.2745 ms/op 2.2299 ms/op 1.02
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.3286 ms/op 3.3273 ms/op 1.00
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.5127 ms/op 5.4543 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 26.930 us/op 27.016 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 104.33 us/op 105.64 us/op 0.99
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 55.037 ms/op 50.345 ms/op 1.09
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 51.923 ms/op 48.830 ms/op 1.06
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 30.240 ms/op 29.092 ms/op 1.04
getSlashingsAndExits - default max 76.785 us/op 89.072 us/op 0.86
getSlashingsAndExits - 2k 260.57 us/op 258.60 us/op 1.01
proposeBlockBody type=full, size=empty 5.2758 ms/op 5.1887 ms/op 1.02
isKnown best case - 1 super set check 307.00 ns/op 298.00 ns/op 1.03
isKnown normal case - 2 super set checks 284.00 ns/op 288.00 ns/op 0.99
isKnown worse case - 16 super set checks 284.00 ns/op 301.00 ns/op 0.94
InMemoryCheckpointStateCache - add get delete 4.4640 us/op 4.3630 us/op 1.02
validate api signedAggregateAndProof - struct 2.4733 ms/op 2.4355 ms/op 1.02
validate gossip signedAggregateAndProof - struct 2.4772 ms/op 2.4335 ms/op 1.02
validate gossip attestation - vc 640000 1.2384 ms/op 1.1844 ms/op 1.05
batch validate gossip attestation - vc 640000 - chunk 32 159.63 us/op 152.42 us/op 1.05
batch validate gossip attestation - vc 640000 - chunk 64 156.36 us/op 134.15 us/op 1.17
batch validate gossip attestation - vc 640000 - chunk 128 150.82 us/op 135.13 us/op 1.12
batch validate gossip attestation - vc 640000 - chunk 256 144.95 us/op 135.43 us/op 1.07
pickEth1Vote - no votes 1.3256 ms/op 1.1063 ms/op 1.20
pickEth1Vote - max votes 10.753 ms/op 9.7620 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot value x2048 22.870 ms/op 20.370 ms/op 1.12
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.163 ms/op 19.802 ms/op 1.17
pickEth1Vote - Eth1Data fastSerialize value x2048 639.95 us/op 513.50 us/op 1.25
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.3514 ms/op 7.5258 ms/op 0.71
bytes32 toHexString 713.00 ns/op 533.00 ns/op 1.34
bytes32 Buffer.toString(hex) 297.00 ns/op 249.00 ns/op 1.19
bytes32 Buffer.toString(hex) from Uint8Array 509.00 ns/op 380.00 ns/op 1.34
bytes32 Buffer.toString(hex) + 0x 283.00 ns/op 266.00 ns/op 1.06
Object access 1 prop 0.20700 ns/op 0.14700 ns/op 1.41
Map access 1 prop 0.15100 ns/op 0.13300 ns/op 1.14
Object get x1000 6.6800 ns/op 6.0840 ns/op 1.10
Map get x1000 7.0010 ns/op 7.0500 ns/op 0.99
Object set x1000 48.669 ns/op 35.508 ns/op 1.37
Map set x1000 33.195 ns/op 24.734 ns/op 1.34
Return object 10000 times 0.32340 ns/op 0.29970 ns/op 1.08
Throw Error 10000 times 3.7844 us/op 3.5739 us/op 1.06
fastMsgIdFn sha256 / 200 bytes 2.5830 us/op 2.2180 us/op 1.16
fastMsgIdFn h32 xxhash / 200 bytes 298.00 ns/op 233.00 ns/op 1.28
fastMsgIdFn h64 xxhash / 200 bytes 292.00 ns/op 265.00 ns/op 1.10
fastMsgIdFn sha256 / 1000 bytes 7.8330 us/op 7.6520 us/op 1.02
fastMsgIdFn h32 xxhash / 1000 bytes 434.00 ns/op 391.00 ns/op 1.11
fastMsgIdFn h64 xxhash / 1000 bytes 376.00 ns/op 349.00 ns/op 1.08
fastMsgIdFn sha256 / 10000 bytes 71.389 us/op 66.804 us/op 1.07
fastMsgIdFn h32 xxhash / 10000 bytes 2.1220 us/op 1.9370 us/op 1.10
fastMsgIdFn h64 xxhash / 10000 bytes 1.3460 us/op 1.2560 us/op 1.07
send data - 1000 256B messages 14.459 ms/op 13.367 ms/op 1.08
send data - 1000 512B messages 19.236 ms/op 19.066 ms/op 1.01
send data - 1000 1024B messages 28.047 ms/op 27.968 ms/op 1.00
send data - 1000 1200B messages 27.819 ms/op 28.272 ms/op 0.98
send data - 1000 2048B messages 35.498 ms/op 32.347 ms/op 1.10
send data - 1000 4096B messages 33.757 ms/op 33.821 ms/op 1.00
send data - 1000 16384B messages 79.533 ms/op 78.672 ms/op 1.01
send data - 1000 65536B messages 248.95 ms/op 221.38 ms/op 1.12
enrSubnets - fastDeserialize 64 bits 1.2560 us/op 1.1220 us/op 1.12
enrSubnets - ssz BitVector 64 bits 396.00 ns/op 351.00 ns/op 1.13
enrSubnets - fastDeserialize 4 bits 176.00 ns/op 151.00 ns/op 1.17
enrSubnets - ssz BitVector 4 bits 428.00 ns/op 375.00 ns/op 1.14
prioritizePeers score -10:0 att 32-0.1 sync 2-0 180.48 us/op 157.00 us/op 1.15
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 161.17 us/op 152.62 us/op 1.06
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 272.42 us/op 271.88 us/op 1.00
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 440.72 us/op 404.83 us/op 1.09
prioritizePeers score 0:0 att 64-1 sync 4-1 676.22 us/op 602.36 us/op 1.12
array of 16000 items push then shift 1.7205 us/op 1.7163 us/op 1.00
LinkedList of 16000 items push then shift 7.9170 ns/op 7.5930 ns/op 1.04
array of 16000 items push then pop 120.25 ns/op 122.57 ns/op 0.98
LinkedList of 16000 items push then pop 7.6150 ns/op 7.1560 ns/op 1.06
array of 24000 items push then shift 2.5178 us/op 2.4802 us/op 1.02
LinkedList of 24000 items push then shift 7.7370 ns/op 7.3270 ns/op 1.06
array of 24000 items push then pop 156.11 ns/op 137.18 ns/op 1.14
LinkedList of 24000 items push then pop 7.5330 ns/op 7.6490 ns/op 0.98
intersect bitArray bitLen 8 6.6750 ns/op 6.7810 ns/op 0.98
intersect array and set length 8 49.972 ns/op 48.769 ns/op 1.02
intersect bitArray bitLen 128 30.863 ns/op 31.000 ns/op 1.00
intersect array and set length 128 695.32 ns/op 738.23 ns/op 0.94
bitArray.getTrueBitIndexes() bitLen 128 2.0550 us/op 1.9830 us/op 1.04
bitArray.getTrueBitIndexes() bitLen 248 2.8970 us/op 3.8110 us/op 0.76
bitArray.getTrueBitIndexes() bitLen 512 6.4590 us/op 7.7020 us/op 0.84
Buffer.concat 32 items 1.0130 us/op 1.0040 us/op 1.01
Uint8Array.set 32 items 1.7210 us/op 1.4950 us/op 1.15
Buffer.copy 1.7680 us/op 1.6940 us/op 1.04
Uint8Array.set - with subarray 2.4790 us/op 2.9030 us/op 0.85
Uint8Array.set - without subarray 1.3220 us/op 1.4940 us/op 0.88
Set add up to 64 items then delete first 2.3572 us/op 2.6886 us/op 0.88
OrderedSet add up to 64 items then delete first 3.5842 us/op 3.8843 us/op 0.92
Set add up to 64 items then delete last 2.6541 us/op 3.0223 us/op 0.88
OrderedSet add up to 64 items then delete last 3.8800 us/op 3.9977 us/op 0.97
Set add up to 64 items then delete middle 2.6394 us/op 2.5951 us/op 1.02
OrderedSet add up to 64 items then delete middle 5.5811 us/op 5.5474 us/op 1.01
Set add up to 128 items then delete first 5.4430 us/op 5.2599 us/op 1.03
OrderedSet add up to 128 items then delete first 8.3169 us/op 7.6838 us/op 1.08
Set add up to 128 items then delete last 5.2926 us/op 5.1576 us/op 1.03
OrderedSet add up to 128 items then delete last 7.8737 us/op 7.9401 us/op 0.99
Set add up to 128 items then delete middle 5.1199 us/op 5.2871 us/op 0.97
OrderedSet add up to 128 items then delete middle 14.495 us/op 14.662 us/op 0.99
Set add up to 256 items then delete first 11.201 us/op 10.655 us/op 1.05
OrderedSet add up to 256 items then delete first 17.248 us/op 16.337 us/op 1.06
Set add up to 256 items then delete last 10.275 us/op 9.7729 us/op 1.05
OrderedSet add up to 256 items then delete last 15.729 us/op 14.910 us/op 1.05
Set add up to 256 items then delete middle 10.390 us/op 9.9254 us/op 1.05
OrderedSet add up to 256 items then delete middle 43.124 us/op 42.476 us/op 1.02
transfer serialized Status (84 B) 1.4850 us/op 1.5230 us/op 0.98
copy serialized Status (84 B) 1.1760 us/op 1.2640 us/op 0.93
transfer serialized SignedVoluntaryExit (112 B) 1.5280 us/op 1.6770 us/op 0.91
copy serialized SignedVoluntaryExit (112 B) 1.2350 us/op 1.3630 us/op 0.91
transfer serialized ProposerSlashing (416 B) 2.1950 us/op 2.3350 us/op 0.94
copy serialized ProposerSlashing (416 B) 2.1690 us/op 2.3290 us/op 0.93
transfer serialized Attestation (485 B) 2.1710 us/op 1.7670 us/op 1.23
copy serialized Attestation (485 B) 1.9120 us/op 1.6200 us/op 1.18
transfer serialized AttesterSlashing (33232 B) 2.7970 us/op 1.7960 us/op 1.56
copy serialized AttesterSlashing (33232 B) 6.1050 us/op 5.3770 us/op 1.14
transfer serialized Small SignedBeaconBlock (128000 B) 2.7360 us/op 2.5500 us/op 1.07
copy serialized Small SignedBeaconBlock (128000 B) 16.385 us/op 17.098 us/op 0.96
transfer serialized Avg SignedBeaconBlock (200000 B) 3.0560 us/op 3.1060 us/op 0.98
copy serialized Avg SignedBeaconBlock (200000 B) 21.468 us/op 22.921 us/op 0.94
transfer serialized BlobsSidecar (524380 B) 3.1680 us/op 3.1200 us/op 1.02
copy serialized BlobsSidecar (524380 B) 203.74 us/op 81.207 us/op 2.51
transfer serialized Big SignedBeaconBlock (1000000 B) 3.0490 us/op 2.8550 us/op 1.07
copy serialized Big SignedBeaconBlock (1000000 B) 146.18 us/op 157.49 us/op 0.93
pass gossip attestations to forkchoice per slot 3.2363 ms/op 3.1808 ms/op 1.02
forkChoice updateHead vc 100000 bc 64 eq 0 526.92 us/op 507.21 us/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 0 3.2264 ms/op 3.1880 ms/op 1.01
forkChoice updateHead vc 1000000 bc 64 eq 0 5.5247 ms/op 5.4507 ms/op 1.01
forkChoice updateHead vc 600000 bc 320 eq 0 3.1620 ms/op 3.0998 ms/op 1.02
forkChoice updateHead vc 600000 bc 1200 eq 0 3.1864 ms/op 3.3082 ms/op 0.96
forkChoice updateHead vc 600000 bc 7200 eq 0 3.7705 ms/op 3.9695 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 1000 11.120 ms/op 11.389 ms/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 10000 11.158 ms/op 11.522 ms/op 0.97
forkChoice updateHead vc 600000 bc 64 eq 300000 15.596 ms/op 15.355 ms/op 1.02
computeDeltas 500000 validators 300 proto nodes 3.6790 ms/op 3.7209 ms/op 0.99
computeDeltas 500000 validators 1200 proto nodes 3.7151 ms/op 3.8218 ms/op 0.97
computeDeltas 500000 validators 7200 proto nodes 3.9412 ms/op 4.1058 ms/op 0.96
computeDeltas 750000 validators 300 proto nodes 5.7885 ms/op 6.5951 ms/op 0.88
computeDeltas 750000 validators 1200 proto nodes 5.7159 ms/op 6.1981 ms/op 0.92
computeDeltas 750000 validators 7200 proto nodes 5.4183 ms/op 5.9281 ms/op 0.91
computeDeltas 1400000 validators 300 proto nodes 9.6420 ms/op 10.811 ms/op 0.89
computeDeltas 1400000 validators 1200 proto nodes 9.5913 ms/op 10.407 ms/op 0.92
computeDeltas 1400000 validators 7200 proto nodes 9.7567 ms/op 10.135 ms/op 0.96
computeDeltas 2100000 validators 300 proto nodes 14.395 ms/op 14.836 ms/op 0.97
computeDeltas 2100000 validators 1200 proto nodes 14.398 ms/op 14.403 ms/op 1.00
computeDeltas 2100000 validators 7200 proto nodes 14.234 ms/op 14.908 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 1.5706 ms/op 1.7732 ms/op 0.89
altair processAttestation - 250000 vs - 7PWei worstcase 2.4048 ms/op 2.6951 ms/op 0.89
altair processAttestation - setStatus - 1/6 committees join 87.560 us/op 88.826 us/op 0.99
altair processAttestation - setStatus - 1/3 committees join 170.07 us/op 184.30 us/op 0.92
altair processAttestation - setStatus - 1/2 committees join 244.75 us/op 246.23 us/op 0.99
altair processAttestation - setStatus - 2/3 committees join 321.60 us/op 334.35 us/op 0.96
altair processAttestation - setStatus - 4/5 committees join 466.13 us/op 469.32 us/op 0.99
altair processAttestation - setStatus - 100% committees join 550.26 us/op 567.62 us/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase 3.4682 ms/op 4.3318 ms/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.716 ms/op 26.430 ms/op 1.05
altair processBlock - 250000 vs - 7PWei worstcase 39.371 ms/op 39.800 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 85.254 ms/op 85.064 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei normalcase 1.6078 ms/op 2.2540 ms/op 0.71
phase0 processBlock - 250000 vs - 7PWei worstcase 25.844 ms/op 26.556 ms/op 0.97
altair processEth1Data - 250000 vs - 7PWei normalcase 279.46 us/op 409.92 us/op 0.68
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.9730 us/op 6.5370 us/op 0.76
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 16.799 us/op 24.866 us/op 0.68
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 6.8030 us/op 9.0260 us/op 0.75
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 4.8230 us/op 6.1080 us/op 0.79
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 68.111 us/op 118.29 us/op 0.58
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 639.61 us/op 706.45 us/op 0.91
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 904.86 us/op 954.76 us/op 0.95
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 882.62 us/op 977.66 us/op 0.90
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.0461 ms/op 2.2173 ms/op 0.92
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.5897 ms/op 1.5585 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.5280 ms/op 3.6733 ms/op 0.96
Tree 40 250000 create 208.59 ms/op 265.84 ms/op 0.78
Tree 40 250000 get(125000) 138.31 ns/op 158.84 ns/op 0.87
Tree 40 250000 set(125000) 628.56 ns/op 760.55 ns/op 0.83
Tree 40 250000 toArray() 14.581 ms/op 21.602 ms/op 0.67
Tree 40 250000 iterate all - toArray() + loop 14.803 ms/op 22.924 ms/op 0.65
Tree 40 250000 iterate all - get(i) 48.487 ms/op 58.627 ms/op 0.83
MutableVector 250000 create 7.3910 ms/op 10.938 ms/op 0.68
MutableVector 250000 get(125000) 6.7410 ns/op 6.3670 ns/op 1.06
MutableVector 250000 set(125000) 209.50 ns/op 220.73 ns/op 0.95
MutableVector 250000 toArray() 3.4844 ms/op 3.8434 ms/op 0.91
MutableVector 250000 iterate all - toArray() + loop 3.6426 ms/op 4.4589 ms/op 0.82
MutableVector 250000 iterate all - get(i) 1.8330 ms/op 1.6040 ms/op 1.14
Array 250000 create 2.9365 ms/op 3.7210 ms/op 0.79
Array 250000 clone - spread 1.5516 ms/op 1.4920 ms/op 1.04
Array 250000 get(125000) 0.41600 ns/op 0.43800 ns/op 0.95
Array 250000 set(125000) 0.44200 ns/op 0.46200 ns/op 0.96
Array 250000 iterate all - loop 109.51 us/op 95.067 us/op 1.15
effectiveBalanceIncrements clone Uint8Array 300000 30.057 us/op 39.069 us/op 0.77
effectiveBalanceIncrements clone MutableVector 300000 128.00 ns/op 132.00 ns/op 0.97
effectiveBalanceIncrements rw all Uint8Array 300000 177.99 us/op 200.81 us/op 0.89
effectiveBalanceIncrements rw all MutableVector 300000 68.883 ms/op 71.587 ms/op 0.96
phase0 afterProcessEpoch - 250000 vs - 7PWei 88.662 ms/op 91.639 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 38.196 ms/op 44.203 ms/op 0.86
altair processEpoch - mainnet_e81889 374.15 ms/op 426.31 ms/op 0.88
mainnet_e81889 - altair beforeProcessEpoch 63.466 ms/op 71.217 ms/op 0.89
mainnet_e81889 - altair processJustificationAndFinalization 11.154 us/op 16.760 us/op 0.67
mainnet_e81889 - altair processInactivityUpdates 5.4068 ms/op 6.7589 ms/op 0.80
mainnet_e81889 - altair processRewardsAndPenalties 35.693 ms/op 44.483 ms/op 0.80
mainnet_e81889 - altair processRegistryUpdates 1.8350 us/op 2.9110 us/op 0.63
mainnet_e81889 - altair processSlashings 332.00 ns/op 472.00 ns/op 0.70
mainnet_e81889 - altair processEth1DataReset 271.00 ns/op 484.00 ns/op 0.56
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.0856 ms/op 1.1177 ms/op 0.97
mainnet_e81889 - altair processSlashingsReset 2.8940 us/op 3.8280 us/op 0.76
mainnet_e81889 - altair processRandaoMixesReset 4.5780 us/op 5.7230 us/op 0.80
mainnet_e81889 - altair processHistoricalRootsUpdate 534.00 ns/op 1.1660 us/op 0.46
mainnet_e81889 - altair processParticipationFlagUpdates 2.0710 us/op 3.4650 us/op 0.60
mainnet_e81889 - altair processSyncCommitteeUpdates 524.00 ns/op 770.00 ns/op 0.68
mainnet_e81889 - altair afterProcessEpoch 88.891 ms/op 100.10 ms/op 0.89
capella processEpoch - mainnet_e217614 1.4453 s/op 2.0809 s/op 0.69
mainnet_e217614 - capella beforeProcessEpoch 253.89 ms/op 337.65 ms/op 0.75
mainnet_e217614 - capella processJustificationAndFinalization 14.734 us/op 42.337 us/op 0.35
mainnet_e217614 - capella processInactivityUpdates 15.441 ms/op 35.901 ms/op 0.43
mainnet_e217614 - capella processRewardsAndPenalties 235.83 ms/op 289.96 ms/op 0.81
mainnet_e217614 - capella processRegistryUpdates 13.040 us/op 14.420 us/op 0.90
mainnet_e217614 - capella processSlashings 408.00 ns/op 735.00 ns/op 0.56
mainnet_e217614 - capella processEth1DataReset 314.00 ns/op 467.00 ns/op 0.67
mainnet_e217614 - capella processEffectiveBalanceUpdates 11.019 ms/op 3.8323 ms/op 2.88
mainnet_e217614 - capella processSlashingsReset 3.7290 us/op 5.3130 us/op 0.70
mainnet_e217614 - capella processRandaoMixesReset 3.6580 us/op 10.147 us/op 0.36
mainnet_e217614 - capella processHistoricalRootsUpdate 319.00 ns/op 1.3510 us/op 0.24
mainnet_e217614 - capella processParticipationFlagUpdates 1.6210 us/op 4.7550 us/op 0.34
mainnet_e217614 - capella afterProcessEpoch 269.82 ms/op 284.38 ms/op 0.95
phase0 processEpoch - mainnet_e58758 408.86 ms/op 426.22 ms/op 0.96
mainnet_e58758 - phase0 beforeProcessEpoch 119.50 ms/op 117.85 ms/op 1.01
mainnet_e58758 - phase0 processJustificationAndFinalization 15.715 us/op 13.725 us/op 1.14
mainnet_e58758 - phase0 processRewardsAndPenalties 24.802 ms/op 21.140 ms/op 1.17
mainnet_e58758 - phase0 processRegistryUpdates 7.5040 us/op 7.3030 us/op 1.03
mainnet_e58758 - phase0 processSlashings 415.00 ns/op 327.00 ns/op 1.27
mainnet_e58758 - phase0 processEth1DataReset 331.00 ns/op 459.00 ns/op 0.72
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 948.78 us/op 938.17 us/op 1.01
mainnet_e58758 - phase0 processSlashingsReset 2.6030 us/op 3.7790 us/op 0.69
mainnet_e58758 - phase0 processRandaoMixesReset 4.1980 us/op 5.2320 us/op 0.80
mainnet_e58758 - phase0 processHistoricalRootsUpdate 366.00 ns/op 624.00 ns/op 0.59
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.8260 us/op 3.8340 us/op 1.00
mainnet_e58758 - phase0 afterProcessEpoch 75.208 ms/op 84.475 ms/op 0.89
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0547 ms/op 1.1501 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9401 ms/op 2.0175 ms/op 0.96
altair processInactivityUpdates - 250000 normalcase 15.639 ms/op 17.222 ms/op 0.91
altair processInactivityUpdates - 250000 worstcase 14.780 ms/op 17.050 ms/op 0.87
phase0 processRegistryUpdates - 250000 normalcase 6.5050 us/op 7.4830 us/op 0.87
phase0 processRegistryUpdates - 250000 badcase_full_deposits 230.93 us/op 293.97 us/op 0.79
phase0 processRegistryUpdates - 250000 worstcase 0.5 118.89 ms/op 137.08 ms/op 0.87
altair processRewardsAndPenalties - 250000 normalcase 40.800 ms/op 37.847 ms/op 1.08
altair processRewardsAndPenalties - 250000 worstcase 35.168 ms/op 37.862 ms/op 0.93
phase0 getAttestationDeltas - 250000 normalcase 6.5772 ms/op 8.1059 ms/op 0.81
phase0 getAttestationDeltas - 250000 worstcase 7.1327 ms/op 10.539 ms/op 0.68
phase0 processSlashings - 250000 worstcase 75.026 us/op 114.35 us/op 0.66
altair processSyncCommitteeUpdates - 250000 127.21 ms/op 145.04 ms/op 0.88
BeaconState.hashTreeRoot - No change 267.00 ns/op 329.00 ns/op 0.81
BeaconState.hashTreeRoot - 1 full validator 129.22 us/op 101.07 us/op 1.28
BeaconState.hashTreeRoot - 32 full validator 1.0985 ms/op 1.0721 ms/op 1.02
BeaconState.hashTreeRoot - 512 full validator 15.453 ms/op 14.460 ms/op 1.07
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 164.83 us/op 161.74 us/op 1.02
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8159 ms/op 2.4683 ms/op 0.74
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 23.781 ms/op 25.247 ms/op 0.94
BeaconState.hashTreeRoot - 1 balances 115.67 us/op 103.76 us/op 1.11
BeaconState.hashTreeRoot - 32 balances 1.1194 ms/op 1.4518 ms/op 0.77
BeaconState.hashTreeRoot - 512 balances 11.210 ms/op 9.8096 ms/op 1.14
BeaconState.hashTreeRoot - 250000 balances 184.45 ms/op 186.78 ms/op 0.99
aggregationBits - 2048 els - zipIndexesInBitList 23.165 us/op 31.803 us/op 0.73
byteArrayEquals 32 53.451 ns/op 58.200 ns/op 0.92
Buffer.compare 32 45.987 ns/op 52.358 ns/op 0.88
byteArrayEquals 1024 1.5809 us/op 1.7325 us/op 0.91
Buffer.compare 1024 56.006 ns/op 62.227 ns/op 0.90
byteArrayEquals 16384 25.158 us/op 27.618 us/op 0.91
Buffer.compare 16384 220.95 ns/op 252.49 ns/op 0.88
byteArrayEquals 123687377 187.11 ms/op 196.27 ms/op 0.95
Buffer.compare 123687377 6.0162 ms/op 8.0961 ms/op 0.74
byteArrayEquals 32 - diff last byte 51.236 ns/op 53.400 ns/op 0.96
Buffer.compare 32 - diff last byte 44.784 ns/op 47.230 ns/op 0.95
byteArrayEquals 1024 - diff last byte 1.5487 us/op 1.6079 us/op 0.96
Buffer.compare 1024 - diff last byte 54.052 ns/op 56.020 ns/op 0.96
byteArrayEquals 16384 - diff last byte 24.692 us/op 25.853 us/op 0.96
Buffer.compare 16384 - diff last byte 232.90 ns/op 250.56 ns/op 0.93
byteArrayEquals 123687377 - diff last byte 188.89 ms/op 193.15 ms/op 0.98
Buffer.compare 123687377 - diff last byte 6.2314 ms/op 7.3294 ms/op 0.85
byteArrayEquals 32 - random bytes 5.2520 ns/op 5.4360 ns/op 0.97
Buffer.compare 32 - random bytes 51.575 ns/op 48.696 ns/op 1.06
byteArrayEquals 1024 - random bytes 5.1860 ns/op 5.2320 ns/op 0.99
Buffer.compare 1024 - random bytes 51.051 ns/op 47.178 ns/op 1.08
byteArrayEquals 16384 - random bytes 5.1940 ns/op 5.1980 ns/op 1.00
Buffer.compare 16384 - random bytes 50.862 ns/op 46.831 ns/op 1.09
byteArrayEquals 123687377 - random bytes 6.6400 ns/op 6.5500 ns/op 1.01
Buffer.compare 123687377 - random bytes 52.250 ns/op 48.330 ns/op 1.08
regular array get 100000 times 43.022 us/op 34.529 us/op 1.25
wrappedArray get 100000 times 33.355 us/op 33.163 us/op 1.01
arrayWithProxy get 100000 times 12.766 ms/op 13.856 ms/op 0.92
ssz.Root.equals 46.700 ns/op 46.477 ns/op 1.00
byteArrayEquals 46.070 ns/op 46.021 ns/op 1.00
Buffer.compare 10.542 ns/op 10.719 ns/op 0.98
shuffle list - 16384 els 6.3131 ms/op 6.6723 ms/op 0.95
shuffle list - 250000 els 91.978 ms/op 97.445 ms/op 0.94
processSlot - 1 slots 13.156 us/op 17.303 us/op 0.76
processSlot - 32 slots 2.5150 ms/op 2.8729 ms/op 0.88
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 36.666 ms/op 34.746 ms/op 1.06
getCommitteeAssignments - req 1 vs - 250000 vc 2.2534 ms/op 2.2612 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 4.2404 ms/op 4.3363 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5045 ms/op 4.5972 ms/op 0.98
findModifiedValidators - 10000 modified validators 267.11 ms/op 281.96 ms/op 0.95
findModifiedValidators - 1000 modified validators 166.69 ms/op 177.21 ms/op 0.94
findModifiedValidators - 100 modified validators 175.98 ms/op 193.84 ms/op 0.91
findModifiedValidators - 10 modified validators 161.56 ms/op 176.08 ms/op 0.92
findModifiedValidators - 1 modified validators 172.82 ms/op 188.97 ms/op 0.91
findModifiedValidators - no difference 152.18 ms/op 165.16 ms/op 0.92
compare ViewDUs 2.8896 s/op 3.0163 s/op 0.96
compare each validator Uint8Array 1.6352 s/op 1.2934 s/op 1.26
compare ViewDU to Uint8Array 1.0884 s/op 1.2452 s/op 0.87
migrate state 1000000 validators, 24 modified, 0 new 530.59 ms/op 559.07 ms/op 0.95
migrate state 1000000 validators, 1700 modified, 1000 new 849.54 ms/op 862.84 ms/op 0.98
migrate state 1000000 validators, 3400 modified, 2000 new 1.0982 s/op 919.95 ms/op 1.19
migrate state 1500000 validators, 24 modified, 0 new 631.32 ms/op 577.46 ms/op 1.09
migrate state 1500000 validators, 1700 modified, 1000 new 853.01 ms/op 819.79 ms/op 1.04
migrate state 1500000 validators, 3400 modified, 2000 new 1.0477 s/op 971.28 ms/op 1.08
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6500 ns/op 4.9400 ns/op 0.94
state getBlockRootAtSlot - 250000 vs - 7PWei 477.35 ns/op 929.97 ns/op 0.51
computeProposers - vc 250000 7.0173 ms/op 8.3208 ms/op 0.84
computeEpochShuffling - vc 250000 89.857 ms/op 94.436 ms/op 0.95
getNextSyncCommittee - vc 250000 117.53 ms/op 148.82 ms/op 0.79
computeSigningRoot for AttestationData 25.078 us/op 23.739 us/op 1.06
hash AttestationData serialized data then Buffer.toString(base64) 1.5267 us/op 1.5761 us/op 0.97
toHexString serialized data 916.26 ns/op 1.4616 us/op 0.63
Buffer.toString(base64) 179.82 ns/op 277.38 ns/op 0.65

by benchmarkbot/action

@g11tech g11tech marked this pull request as ready for review June 1, 2024 15:37
@g11tech g11tech requested a review from a team as a code owner June 1, 2024 15:37
…a columns

rename types and fix references to them

update forkchoice and availability flow

propagate changes to the codebase

lint and tsc

some fixes

cleanup

test run fixes

fix the tests

fix
function notOnOutOfRangeData(beaconBlockRoot: Root): void {
const protoBeaconBlock = chain.forkChoice.getBlock(beaconBlockRoot);
if (!protoBeaconBlock) {
throw new ApiError(400, "Block not in forkChoice");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add beaconBlockRoot to error message

};
},

outOfRangeData(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is a block "out of range data"?

};
type Availability<T> = {availabilityPromise: Promise<T>; resolveAvailability: (data: T) => void};

type CachedBlobs = BlobsInputCache & Availability<BlockInputDataBlobs>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having both CachedBlobs and BlobsCache types seems confusing

preDeneb = "preDeneb",
postDeneb = "postDeneb",
blobsPromise = "blobsPromise",
preData = "preData",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop a comment to note this also means pre deneb

@twoeths
Copy link
Contributor

twoeths commented Jun 10, 2024

a note for reviewers, this PR also adds a new field "dataAvailabilityStatus" to forkchoice

@@ -136,33 +136,29 @@ export class SeenGossipBlockInput {

if (blobKzgCommitments.length === blobsCache.size) {
const allBlobs = getBlockInputBlobs(blobsCache);
resolveAvailability({...allBlobs, blobsSource: BlobsSource.gossip});
const blockData = {...allBlobs, blobsSource: BlobsSource.gossip, fork: cachedData.fork} as BlockInputDataBlobs;
Copy link
Contributor

@twoeths twoeths Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to enhance type to avoid type. cast?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants