{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":121291048,"defaultBranch":"master","name":"genny","ownerLogin":"mongodb","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2018-02-12T19:23:44.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/45120?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1717184475.0","currentOid":""},"activityList":{"items":[{"before":"fa88f1b014405d48ee59c35f3f8db092130a91e1","after":"4d0d4dae6c2fd1cdca7adbcde085d09e32045c11","ref":"refs/heads/dstorch/PERF-5375","pushedAt":"2024-05-31T19:52:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dstorch","name":"David Storch","path":"/dstorch","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/760706?s=80&v=4"},"commit":{"message":"remove workload name from 'GRANDFATHERED_WORKLOADS_WITHOUT_KEYWORDS'","shortMessageHtmlLink":"remove workload name from 'GRANDFATHERED_WORKLOADS_WITHOUT_KEYWORDS'"}},{"before":null,"after":"fa88f1b014405d48ee59c35f3f8db092130a91e1","ref":"refs/heads/dstorch/PERF-5375","pushedAt":"2024-05-31T19:41:15.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dstorch","name":"David Storch","path":"/dstorch","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/760706?s=80&v=4"},"commit":{"message":"PERF-5375 delete NoGoodPlan.yml multi-planning workload","shortMessageHtmlLink":"PERF-5375 delete NoGoodPlan.yml multi-planning workload"}},{"before":"6579412ef0c1caf680bf0999c0b798b63a5e6412","after":null,"ref":"refs/heads/yoonsoo.kim/PERF-5522","pushedAt":"2024-05-31T19:13:07.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"}},{"before":"119e8f7b023e649dc1b59c8791ea86db47320fe8","after":null,"ref":"refs/heads/gh-readonly-queue/master/pr-1225-633ca77f11bb59b806231df2a0710a9390c151b2","pushedAt":"2024-05-31T19:09:39.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"}},{"before":"633ca77f11bb59b806231df2a0710a9390c151b2","after":"119e8f7b023e649dc1b59c8791ea86db47320fe8","ref":"refs/heads/master","pushedAt":"2024-05-31T19:09:38.000Z","pushType":"merge_queue_merge","commitsCount":1,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"PERF-5522 Fix time_series_lastpoint drop collection actor (#1225)\n\n**Jira Ticket:** [PERF-5522](https://jira.mongodb.org/browse/PERF-5522)\n\n### Whats Changed\n\nFix `TimeSeriesLastpoint.yml` drop collection actor so that it does not\nreturn an error for non-existing collection on v6.0. This is a fix for\nBF-33490\n\n### Patch Testing Results\n\nsys-perf patch\n\nhttps://spruce.mongodb.com/version/6659555749889e0007008b72/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\nsys-perf-6.0 patch\n\nhttps://spruce.mongodb.com/version/665956cfb800a30007b5ad40/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC","shortMessageHtmlLink":"PERF-5522 Fix time_series_lastpoint drop collection actor (#1225)"}},{"before":null,"after":"119e8f7b023e649dc1b59c8791ea86db47320fe8","ref":"refs/heads/gh-readonly-queue/master/pr-1225-633ca77f11bb59b806231df2a0710a9390c151b2","pushedAt":"2024-05-31T17:35:21.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"PERF-5522 Fix time_series_lastpoint drop collection actor (#1225)\n\n**Jira Ticket:** [PERF-5522](https://jira.mongodb.org/browse/PERF-5522)\n\n### Whats Changed\n\nFix `TimeSeriesLastpoint.yml` drop collection actor so that it does not\nreturn an error for non-existing collection on v6.0. This is a fix for\nBF-33490\n\n### Patch Testing Results\n\nsys-perf patch\n\nhttps://spruce.mongodb.com/version/6659555749889e0007008b72/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\nsys-perf-6.0 patch\n\nhttps://spruce.mongodb.com/version/665956cfb800a30007b5ad40/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC","shortMessageHtmlLink":"PERF-5522 Fix time_series_lastpoint drop collection actor (#1225)"}},{"before":null,"after":"6579412ef0c1caf680bf0999c0b798b63a5e6412","ref":"refs/heads/yoonsoo.kim/PERF-5522","pushedAt":"2024-05-31T04:41:51.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"},"commit":{"message":"PERF-5522 fix","shortMessageHtmlLink":"PERF-5522 fix"}},{"before":"18209ff809757958d723450bb7a925043d76c9db","after":"a2793568c82372661f9b51d55f189151f5acf531","ref":"refs/heads/storch/PERF-5373","pushedAt":"2024-05-30T18:09:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dstorch","name":"David Storch","path":"/dstorch","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/760706?s=80&v=4"},"commit":{"message":"changes to generated documentation files","shortMessageHtmlLink":"changes to generated documentation files"}},{"before":null,"after":"18209ff809757958d723450bb7a925043d76c9db","ref":"refs/heads/storch/PERF-5373","pushedAt":"2024-05-30T17:55:06.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dstorch","name":"David Storch","path":"/dstorch","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/760706?s=80&v=4"},"commit":{"message":"PERF-5373 Fix NoSuchField.yml so that it hits 'max works' as intended","shortMessageHtmlLink":"PERF-5373 Fix NoSuchField.yml so that it hits 'max works' as intended"}},{"before":"f3a92900cb904295d339294a1667c16ced9c81f7","after":"b4e3a85bf3f9720b619e73ee0f3ff662aed933b8","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-30T08:18:37.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"Update toolchain ID for the proper vcpkg toolchain","shortMessageHtmlLink":"Update toolchain ID for the proper vcpkg toolchain"}},{"before":"e9f8dc25c70b2e2ef7affec6c7b86e6090e04615","after":"f3a92900cb904295d339294a1667c16ced9c81f7","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-23T07:14:48.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"DEVPROD-7330 Don't use toolchain with intermediate build and debug artifacts","shortMessageHtmlLink":"DEVPROD-7330 Don't use toolchain with intermediate build and debug ar…"}},{"before":"49358819a02d74db3f0ef50df90990bfccbdfa87","after":null,"ref":"refs/heads/yoonsoo.kim/PERF-5440","pushedAt":"2024-05-22T19:42:01.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"}},{"before":"633ca77f11bb59b806231df2a0710a9390c151b2","after":null,"ref":"refs/heads/gh-readonly-queue/master/pr-1219-9e238c885f87118fe60f5d0dfff18e2680fe53ac","pushedAt":"2024-05-22T19:39:37.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"}},{"before":"9e238c885f87118fe60f5d0dfff18e2680fe53ac","after":"633ca77f11bb59b806231df2a0710a9390c151b2","ref":"refs/heads/master","pushedAt":"2024-05-22T19:39:36.000Z","pushType":"merge_queue_merge","commitsCount":1,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"PERF-5440 Fix created indices for ascending sort key queries in TimeSeriesLastpoint.yml workload (#1219)\n\n**Jira Ticket:** [PERF-5440](https://jira.mongodb.org/browse/PERF-5440)\n\n### Whats Changed\n\n- Fix wrong index creation for ascending time sort key queries\n- Repeat optimized last point queries100 times instead of 10 to be able\nto collect on-cpu profiling\n- Use `CrudActor` instead of `RunCommand` to execute the aggregates\n- Use `OnlyActiveInPhases`\n\n### Patch Testing Results\n\n\nhttps://spruce.mongodb.com/version/663feee5e0ec0400072fd1cb/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\n\n\n\n\n","shortMessageHtmlLink":"PERF-5440 Fix created indices for ascending sort key queries in TimeS…"}},{"before":null,"after":"633ca77f11bb59b806231df2a0710a9390c151b2","ref":"refs/heads/gh-readonly-queue/master/pr-1219-9e238c885f87118fe60f5d0dfff18e2680fe53ac","pushedAt":"2024-05-22T17:57:54.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"PERF-5440 Fix created indices for ascending sort key queries in TimeSeriesLastpoint.yml workload (#1219)\n\n**Jira Ticket:** [PERF-5440](https://jira.mongodb.org/browse/PERF-5440)\n\n### Whats Changed\n\n- Fix wrong index creation for ascending time sort key queries\n- Repeat optimized last point queries100 times instead of 10 to be able\nto collect on-cpu profiling\n- Use `CrudActor` instead of `RunCommand` to execute the aggregates\n- Use `OnlyActiveInPhases`\n\n### Patch Testing Results\n\n\nhttps://spruce.mongodb.com/version/663feee5e0ec0400072fd1cb/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\n\n\n\n\n","shortMessageHtmlLink":"PERF-5440 Fix created indices for ascending sort key queries in TimeS…"}},{"before":"90efa9c0f7f518978d79bfa337f9212c79665cf3","after":"49358819a02d74db3f0ef50df90990bfccbdfa87","ref":"refs/heads/yoonsoo.kim/PERF-5440","pushedAt":"2024-05-22T15:06:52.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"},"commit":{"message":"Address PR comments","shortMessageHtmlLink":"Address PR comments"}},{"before":"6afbd31ff4a624c6b8ff24b8ccef06477b86355b","after":"89918e21e37100e4a8e7daa4629a6cd2cca48969","ref":"refs/heads/yoonsoo.kim/PERF-5441","pushedAt":"2024-05-22T14:42:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"},"commit":{"message":"Address PR comment","shortMessageHtmlLink":"Address PR comment"}},{"before":"958150ef92e0e6ed5a57729e01e4f13bd56b144d","after":"e9f8dc25c70b2e2ef7affec6c7b86e6090e04615","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-22T09:00:10.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"DEVPROD-7330 Don't use toolchain with intermediate build and debug artifacts","shortMessageHtmlLink":"DEVPROD-7330 Don't use toolchain with intermediate build and debug ar…"}},{"before":"34ece6d9bb27ac8b59253f39c3b1c7be9e3a7281","after":"958150ef92e0e6ed5a57729e01e4f13bd56b144d","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-22T08:57:12.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"DEVPROD-7330 Don't use toolchain with intermediate build and debug artifacts\n\nRemove strange CMake paths and point to the real installed Ninja path for OSX","shortMessageHtmlLink":"DEVPROD-7330 Don't use toolchain with intermediate build and debug ar…"}},{"before":"43fa3fb0c084c339284044b97fab39bba3f6f9f4","after":"34ece6d9bb27ac8b59253f39c3b1c7be9e3a7281","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-22T08:26:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"Remove strange CMake paths and point to the real installed Ninja path for OSX","shortMessageHtmlLink":"Remove strange CMake paths and point to the real installed Ninja path…"}},{"before":"54fda90382e1ac36e868f413dd0833ffd5a488df","after":"43fa3fb0c084c339284044b97fab39bba3f6f9f4","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-22T07:04:36.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"DEVPROD-7330 Don't use toolchain with intermediate build and debug artifacts","shortMessageHtmlLink":"DEVPROD-7330 Don't use toolchain with intermediate build and debug ar…"}},{"before":"b25f3d381cac594f282e14f1b9c2cf9fbde31fa1","after":"54fda90382e1ac36e868f413dd0833ffd5a488df","ref":"refs/heads/james.littlejohn/DEVPROD-7330_ignore_buildtrees","pushedAt":"2024-05-22T05:47:11.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"thessem","name":"James Littlejohn","path":"/thessem","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/973593?s=80&v=4"},"commit":{"message":"DEVPROD-7330 Don't use toolchain with intermediate build and debug artifacts","shortMessageHtmlLink":"DEVPROD-7330 Don't use toolchain with intermediate build and debug ar…"}},{"before":null,"after":"0559d9488eb87f30de08af7c95d7992ba4055c4b","ref":"refs/heads/jseyster/hacky-bson-transforms-workload","pushedAt":"2024-05-21T18:14:37.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"jseyster","name":"Justin Seyster","path":"/jseyster","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2365502?s=80&v=4"},"commit":{"message":"Quick-and-dirty workload for testing VM-native implementation of BSON-to-BSON transforms","shortMessageHtmlLink":"Quick-and-dirty workload for testing VM-native implementation of BSON…"}},{"before":"dbb3df240b8cbd67e3e1ba217f0289965e1a0741","after":"6afbd31ff4a624c6b8ff24b8ccef06477b86355b","ref":"refs/heads/yoonsoo.kim/PERF-5441","pushedAt":"2024-05-21T17:24:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yun-soo","name":"Yoonsoo Kim","path":"/yun-soo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/78384057?s=80&v=4"},"commit":{"message":"Fix build variants to run","shortMessageHtmlLink":"Fix build variants to run"}},{"before":"fe92b7152eb2d7bb1f1c2dfe760a413cf8872a5c","after":null,"ref":"refs/heads/dependabot/pip/src/lamplib/requests-2.31.0","pushedAt":"2024-05-21T05:55:48.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"}},{"before":null,"after":"30de4447068898b7a1bc305a9d13522adf54fd35","ref":"refs/heads/dependabot/pip/src/lamplib/requests-2.32.0","pushedAt":"2024-05-21T05:55:44.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"---\nupdated-dependencies:\n- dependency-name: requests\n dependency-type: direct:production\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"---"}},{"before":"20d905261971ad424c9bf6f263c33bf4c83ab5da","after":"f9b9ebcb78363aa98cab6bdac771bf0a604bc8b7","ref":"refs/heads/william.qian/perf-report","pushedAt":"2024-05-20T20:32:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"wqian94","name":"William L Qian","path":"/wqian94","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1192680?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"9e238c885f87118fe60f5d0dfff18e2680fe53ac","after":null,"ref":"refs/heads/gh-readonly-queue/master/pr-1221-65e1858730d4cd43882fa61b62f60847a8ee61b1","pushedAt":"2024-05-20T20:01:30.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"}},{"before":"65e1858730d4cd43882fa61b62f60847a8ee61b1","after":"9e238c885f87118fe60f5d0dfff18e2680fe53ac","ref":"refs/heads/master","pushedAt":"2024-05-20T20:01:30.000Z","pushType":"merge_queue_merge","commitsCount":1,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"Perf 5459 compare disable sbe plan cache (#1221)\n\n\n\n**Jira Ticket:** [PERF-5459](https://jira.mongodb.org/browse/PERF-5459)\n\n### Whats Changed\n\nI've added two new Genny benchmarks:\n* $group on an empty collection, intended to show a case where the time\ntaken by stage-builders matters.\n* $eq with a varying array, intended to show a case where the Classic\nplan cache has a much better hit rate than the SBE plan cache.\n\nI've spot checked it locally to make sure it does what we want. I\nchecked the number of plan cache entries in each scenario:\n* empty $group\n* master: 5 cache entries (one per query shape -- there are no\nparameters varying here)\n * draft PR: 0 cache entries (because there are no indexes)\n* $eq array\n * master: 1000 cache entries (one per parameter value)\n* draft PR: 1 cache entry (showing Classic's auto-parameterization\nworking well)\n\nI also looked at the histograms locally, just to spot check. (Don't\nbelieve the actual numbers, because I didn't build with `--release`.)\nThings I noticed are:\n* empty $group\n * master\n * The median latency is negative in many cases??\n* There's consistently one outlier per query shape. (Maybe this is the\nfirst request, which does the query planning?)\n* Despite the SBE plan cache, the average latency does increase with the\nsize of the query text.\n * draft PR\n* For each query shape the average latency is higher than on master.\n* There's still consistently one outlier. (It can't be that only the\nfirst request did query planning, because there are 0 cache entries.)\n* $eq array\n * master\n* There's still one outlier. (It can't be due to planning, because there\nare 1000 cache entries. It must be some other kind of warm-up.)\n * Very roughly, one query took 7ms and the rest took 5ms.\n * draft PR\n* Now there are two \"outliers\": 7ms and 5ms, while the rest took 2ms.\n* (We know there's only one cache entry, and IIRC it would require\nplanning twice to become active. So the 7ms and 5ms must be the two\nrequests that did the planning, which makes sense because the 5ms\nmatches the typical value on master, which is planning every time.)\n\nThe histograms are all generated with `genny_venv/bin/python\nsrc/workloads/contrib/analysis/test_result_summary.py -m timers.dur -a\n\".*agg.*\"`.\n\n\n\n\n### Patch Testing Results\n\nI've started two individual patch builds:\n* new benchmark on draft pr:\nhttps://spruce.mongodb.com/version/66452b0162b06200078e3f27/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n* new benchmarks on base commit:\nhttps://spruce.mongodb.com/version/66452b3ae0a31d00071c6ea4/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\nJust to make sure it basically works, and that I've selected the right\ntasks. If that works I'll do a multipatch to get more trustworthy\nnumbers, or I'll address any PR feedback first.\n\n\n\n\n\n### Histograms I ran locally\n\n\nempty $group, master:\n```\nGroupQuery01Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3135 \n median : -0.1903 \n mode : 0.3056 \n stddev : 0.02719 \n [min, max]: ['0.3024', '1.108']\n histogram:\n [0,0): ************************************************************... (992)\n [0,0): ****** (6)\n [0,0): (0)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery04Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3363 \n median : 0.3329 \n mode : 0.3317 \n stddev : 0.02574 \n [min, max]: ['0.3265', '1.079']\n histogram:\n [0,0): ************************************************************... (993)\n [0,0): **** (4)\n [0,0): * (1)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery08Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3856 \n median : -0.118 \n mode : 0.3832 \n stddev : 0.02769 \n [min, max]: ['0.3747', '1.215']\n histogram:\n [0,0): ************************************************************... (995)\n [0,0): *** (3)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery16Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.4898 \n median : -0.01565\n mode : 0.4814 \n stddev : 0.04695 \n [min, max]: ['0.4755', '1.899']\n histogram:\n [0,1): ************************************************************... (995)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2]: * (1)\n\n\nGroupQuery32Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.6548 \n median : 0.1489 \n mode : 0.6443 \n stddev : 0.05987 \n [min, max]: ['0.6386', '2.507']\n histogram:\n [1,1): ************************************************************... (997)\n [1,1): ** (2)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,3]: * (1)\n```\n\nempty $group, draft PR:\n```\nGroupQuery01Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.4015 \n median : -0.1025 \n mode : 0.3966 \n stddev : 0.01979 \n [min, max]: ['0.3888', '0.8884']\n histogram:\n [0,0): ************************************************************... (966)\n [0,0): ************************** (26)\n [0,0): **** (4)\n [0,1): * (1)\n [1,1): * (1)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery04Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.5456 \n median : 0.04166 \n mode : 0.5391 \n stddev : 0.01639 \n [min, max]: ['0.5319', '0.9104']\n histogram:\n [1,1): ************************************************************... (878)\n [1,1): ************************************************************... (114)\n [1,1): **** (4)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery08Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.7421 \n median : 0.2377 \n mode : 0.7292 \n stddev : 0.01712 \n [min, max]: ['0.7244', '1.074']\n histogram:\n [1,1): ************************************************************... (741)\n [1,1): ************************************************************... (235)\n [1,1): ************** (14)\n [1,1): ****** (6)\n [1,1): (0)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery16Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.130 \n median : 0.6253 \n mode : 1.128 \n stddev : 0.02557 \n [min, max]: ['1.097', '1.510']\n histogram:\n [1,1): ************************************************************... (485)\n [1,1): ************************************************************... (397)\n [1,1): ************************************************************... (84)\n [1,1): ************************** (26)\n [1,1): ** (2)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2]: * (1)\n\n\nGroupQuery32Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.917 \n median : 1.406 \n mode : 1.927 \n stddev : 0.06044 \n [min, max]: ['1.828', '2.360']\n histogram:\n [2,2): ************************************************************... (155)\n [2,2): ************************************************************... (289)\n [2,2): ************************************************************... (266)\n [2,2): ************************************************************... (158)\n [2,2): ******************************************************* (55)\n [2,2): ************************************* (37)\n [2,2): **************** (16)\n [2,2): ***** (5)\n [2,2): ******** (8)\n [2,2): ****** (6)\n [2,2): *** (3)\n [2,2): * (1)\n [2,2): (0)\n [2,2): (0)\n [2,2]: * (1)\n```\n\n$eq array, master:\n```\nMatchQuery02Indexes.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 5.116 \n median : 4.616 \n mode : 5.101 \n stddev : 0.09576 \n [min, max]: ['4.913', '7.244']\n histogram:\n [5,5): ************************************************************... (234)\n [5,5): ************************************************************... (719)\n [5,5): ********************************************* (45)\n [5,6): * (1)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,7): (0)\n [7,7): (0)\n [7,7): (0)\n [7,7): (0)\n [7,7]: * (1)\n```\n\n$eq array, draft PR:\n```\nMatchQuery02Indexes.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.771 \n median : 1.751 \n mode : 1.817 \n stddev : 0.2151 \n [min, max]: ['1.648', '7.319']\n histogram:\n [2,2): ************************************************************... (997)\n [2,2): * (1)\n [2,3): (0)\n [3,3): (0)\n [3,4): (0)\n [4,4): (0)\n [4,4): (0)\n [4,5): (0)\n [5,5): (0)\n [5,5): * (1)\n [5,6): (0)\n [6,6): (0)\n [6,7): (0)\n [7,7): (0)\n [7,7]: * (1)\n```\n\n\n\n\n","shortMessageHtmlLink":"Perf 5459 compare disable sbe plan cache (#1221)"}},{"before":null,"after":"9e238c885f87118fe60f5d0dfff18e2680fe53ac","ref":"refs/heads/gh-readonly-queue/master/pr-1221-65e1858730d4cd43882fa61b62f60847a8ee61b1","pushedAt":"2024-05-20T18:21:37.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"github-merge-queue[bot]","name":null,"path":"/apps/github-merge-queue","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/9919?s=80&v=4"},"commit":{"message":"Perf 5459 compare disable sbe plan cache (#1221)\n\n\n\n**Jira Ticket:** [PERF-5459](https://jira.mongodb.org/browse/PERF-5459)\n\n### Whats Changed\n\nI've added two new Genny benchmarks:\n* $group on an empty collection, intended to show a case where the time\ntaken by stage-builders matters.\n* $eq with a varying array, intended to show a case where the Classic\nplan cache has a much better hit rate than the SBE plan cache.\n\nI've spot checked it locally to make sure it does what we want. I\nchecked the number of plan cache entries in each scenario:\n* empty $group\n* master: 5 cache entries (one per query shape -- there are no\nparameters varying here)\n * draft PR: 0 cache entries (because there are no indexes)\n* $eq array\n * master: 1000 cache entries (one per parameter value)\n* draft PR: 1 cache entry (showing Classic's auto-parameterization\nworking well)\n\nI also looked at the histograms locally, just to spot check. (Don't\nbelieve the actual numbers, because I didn't build with `--release`.)\nThings I noticed are:\n* empty $group\n * master\n * The median latency is negative in many cases??\n* There's consistently one outlier per query shape. (Maybe this is the\nfirst request, which does the query planning?)\n* Despite the SBE plan cache, the average latency does increase with the\nsize of the query text.\n * draft PR\n* For each query shape the average latency is higher than on master.\n* There's still consistently one outlier. (It can't be that only the\nfirst request did query planning, because there are 0 cache entries.)\n* $eq array\n * master\n* There's still one outlier. (It can't be due to planning, because there\nare 1000 cache entries. It must be some other kind of warm-up.)\n * Very roughly, one query took 7ms and the rest took 5ms.\n * draft PR\n* Now there are two \"outliers\": 7ms and 5ms, while the rest took 2ms.\n* (We know there's only one cache entry, and IIRC it would require\nplanning twice to become active. So the 7ms and 5ms must be the two\nrequests that did the planning, which makes sense because the 5ms\nmatches the typical value on master, which is planning every time.)\n\nThe histograms are all generated with `genny_venv/bin/python\nsrc/workloads/contrib/analysis/test_result_summary.py -m timers.dur -a\n\".*agg.*\"`.\n\n\n\n\n### Patch Testing Results\n\nI've started two individual patch builds:\n* new benchmark on draft pr:\nhttps://spruce.mongodb.com/version/66452b0162b06200078e3f27/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n* new benchmarks on base commit:\nhttps://spruce.mongodb.com/version/66452b3ae0a31d00071c6ea4/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC\n\nJust to make sure it basically works, and that I've selected the right\ntasks. If that works I'll do a multipatch to get more trustworthy\nnumbers, or I'll address any PR feedback first.\n\n\n\n\n\n### Histograms I ran locally\n\n\nempty $group, master:\n```\nGroupQuery01Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3135 \n median : -0.1903 \n mode : 0.3056 \n stddev : 0.02719 \n [min, max]: ['0.3024', '1.108']\n histogram:\n [0,0): ************************************************************... (992)\n [0,0): ****** (6)\n [0,0): (0)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery04Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3363 \n median : 0.3329 \n mode : 0.3317 \n stddev : 0.02574 \n [min, max]: ['0.3265', '1.079']\n histogram:\n [0,0): ************************************************************... (993)\n [0,0): **** (4)\n [0,0): * (1)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery08Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.3856 \n median : -0.118 \n mode : 0.3832 \n stddev : 0.02769 \n [min, max]: ['0.3747', '1.215']\n histogram:\n [0,0): ************************************************************... (995)\n [0,0): *** (3)\n [0,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery16Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.4898 \n median : -0.01565\n mode : 0.4814 \n stddev : 0.04695 \n [min, max]: ['0.4755', '1.899']\n histogram:\n [0,1): ************************************************************... (995)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2]: * (1)\n\n\nGroupQuery32Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.6548 \n median : 0.1489 \n mode : 0.6443 \n stddev : 0.05987 \n [min, max]: ['0.6386', '2.507']\n histogram:\n [1,1): ************************************************************... (997)\n [1,1): ** (2)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,2): (0)\n [2,3]: * (1)\n```\n\nempty $group, draft PR:\n```\nGroupQuery01Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.4015 \n median : -0.1025 \n mode : 0.3966 \n stddev : 0.01979 \n [min, max]: ['0.3888', '0.8884']\n histogram:\n [0,0): ************************************************************... (966)\n [0,0): ************************** (26)\n [0,0): **** (4)\n [0,1): * (1)\n [1,1): * (1)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery04Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.5456 \n median : 0.04166 \n mode : 0.5391 \n stddev : 0.01639 \n [min, max]: ['0.5319', '0.9104']\n histogram:\n [1,1): ************************************************************... (878)\n [1,1): ************************************************************... (114)\n [1,1): **** (4)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery08Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 0.7421 \n median : 0.2377 \n mode : 0.7292 \n stddev : 0.01712 \n [min, max]: ['0.7244', '1.074']\n histogram:\n [1,1): ************************************************************... (741)\n [1,1): ************************************************************... (235)\n [1,1): ************** (14)\n [1,1): ****** (6)\n [1,1): (0)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1]: * (1)\n\n\nGroupQuery16Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.130 \n median : 0.6253 \n mode : 1.128 \n stddev : 0.02557 \n [min, max]: ['1.097', '1.510']\n histogram:\n [1,1): ************************************************************... (485)\n [1,1): ************************************************************... (397)\n [1,1): ************************************************************... (84)\n [1,1): ************************** (26)\n [1,1): ** (2)\n [1,1): *** (3)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): * (1)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,1): (0)\n [1,2]: * (1)\n\n\nGroupQuery32Accum.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.917 \n median : 1.406 \n mode : 1.927 \n stddev : 0.06044 \n [min, max]: ['1.828', '2.360']\n histogram:\n [2,2): ************************************************************... (155)\n [2,2): ************************************************************... (289)\n [2,2): ************************************************************... (266)\n [2,2): ************************************************************... (158)\n [2,2): ******************************************************* (55)\n [2,2): ************************************* (37)\n [2,2): **************** (16)\n [2,2): ***** (5)\n [2,2): ******** (8)\n [2,2): ****** (6)\n [2,2): *** (3)\n [2,2): * (1)\n [2,2): (0)\n [2,2): (0)\n [2,2]: * (1)\n```\n\n$eq array, master:\n```\nMatchQuery02Indexes.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 5.116 \n median : 4.616 \n mode : 5.101 \n stddev : 0.09576 \n [min, max]: ['4.913', '7.244']\n histogram:\n [5,5): ************************************************************... (234)\n [5,5): ************************************************************... (719)\n [5,5): ********************************************* (45)\n [5,6): * (1)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,6): (0)\n [6,7): (0)\n [7,7): (0)\n [7,7): (0)\n [7,7): (0)\n [7,7]: * (1)\n```\n\n$eq array, draft PR:\n```\nMatchQuery02Indexes.aggregate summary:\n timers.dur (measured in nanoseconds, displayed in milliseconds):\n count : 1000 \n average : 1.771 \n median : 1.751 \n mode : 1.817 \n stddev : 0.2151 \n [min, max]: ['1.648', '7.319']\n histogram:\n [2,2): ************************************************************... (997)\n [2,2): * (1)\n [2,3): (0)\n [3,3): (0)\n [3,4): (0)\n [4,4): (0)\n [4,4): (0)\n [4,5): (0)\n [5,5): (0)\n [5,5): * (1)\n [5,6): (0)\n [6,6): (0)\n [6,7): (0)\n [7,7): (0)\n [7,7]: * (1)\n```\n\n\n\n\n","shortMessageHtmlLink":"Perf 5459 compare disable sbe plan cache (#1221)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEWVpzgQA","startCursor":null,"endCursor":null}},"title":"Activity · mongodb/genny"}