Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more benchmark tests and seamless immutable #299

Open
3 of 8 tasks
nlepage opened this issue May 31, 2018 · 9 comments
Open
3 of 8 tasks

Add more benchmark tests and seamless immutable #299

nlepage opened this issue May 31, 2018 · 9 comments
Assignees
Milestone

Comments

@nlepage
Copy link
Member

nlepage commented May 31, 2018

Some ideas:

  • Set property
  • Set deep property Add setDeepProp in benchmark #324
  • Score per operation
  • Add elements array
  • Update elements in array
  • Update elements in array slice (covered by todo list)
  • Add elements in deeply nested arrays
  • Different operations at different paths
@nlepage nlepage added this to the 2.0 milestone May 31, 2018
@nlepage nlepage self-assigned this May 31, 2018
@nlepage
Copy link
Member Author

nlepage commented May 31, 2018

New output format in order to display more results:

ES2015 destructuring immutable 3.8.2 immer 1.2.0 qim 0.0.52 immutad●t 2.0.0
Update small todos list (1000 items) 14841ops/s (0.07ms/op) 5961ops/s (0.17ms/op) 2668ops/s (0.37ms/op) 10515ops/s (0.10ms/op) 11837ops/s (0.08ms/op)
Update medium todos list (10000 items) 1567ops/s (0.64ms/op) 554ops/s (1.81ms/op) 270ops/s (3.70ms/op) 1095ops/s (0.91ms/op) 1361ops/s (0.73ms/op)
Update large todos list (100000 items) 119ops/s (8.40ms/op) 54ops/s (18.54ms/op) 24ops/s (40.89ms/op) 91ops/s (11.02ms/op) 106ops/s (9.44ms/op)

@nlepage nlepage changed the title Add more benchmark tests Add more benchmark tests and seamless immutable Jun 1, 2018
@nlepage
Copy link
Member Author

nlepage commented Jul 3, 2018

Also allow to run only one benchmark with yarn start setProp or yarn fast setProp.
This is not so easy with jest since each test file is isolated...

@nlepage nlepage mentioned this issue Jul 3, 2018
@nlepage
Copy link
Member Author

nlepage commented Nov 30, 2018

Also allow to run only one benchmark with yarn start setProp or yarn fast setProp.
This is not so easy with jest since each test file is isolated...

Tried doing this with jest, got some headaches... so no.

@nlepage
Copy link
Member Author

nlepage commented Nov 30, 2018

Latest output:

ES2015 destructuring immutable 3.8.2 seamless-immutable 7.1.3 immer 1.2.0 qim 0.0.52 immutad●t 2.0.0 qim 0.0.52 curried immutad●t 2.0.0 curried
Set a property 5868546ops/s (170ns/op) 518726ops/s (1.93µs/op) 67781ops/s (14.8µs/op) 165929ops/s (6.03µs/op) 883236ops/s (1.13µs/op) 980969ops/s (1.02µs/op) 829188ops/s (1.21µs/op) 907277ops/s (1.10µs/op)
Update small todos list (1000 items) 75077ops/s (13.3µs/op) 5769ops/s (173µs/op) 580ops/s (1.72ms/op) 2610ops/s (383µs/op) 17674ops/s (56.6µs/op) 24510ops/s (40.8µs/op) 17941ops/s (55.7µs/op) 26289ops/s (38.0µs/op)
Update medium todos list (10000 items) 7185ops/s (139µs/op) 555ops/s (1.80ms/op) 60ops/s (16.8ms/op) 259ops/s (3.86ms/op) 1886ops/s (530µs/op) 3354ops/s (298µs/op) 1884ops/s (531µs/op) 3354ops/s (298µs/op)
Update large todos list (100000 items) 286ops/s (3.50ms/op) 48ops/s (21.0ms/op) 6ops/s (181ms/op) 24ops/s (41.6ms/op) 135ops/s (7.38ms/op) 193ops/s (5.19ms/op) 130ops/s (7.69ms/op) 204ops/s (4.90ms/op)
Final score 100 10 1 5 28 41 27 42

Performance improvements were made on apply and curry, so this is even more important, in order to validate those improvements.

Also, could be nice to have the score on each line.

@nlepage
Copy link
Member Author

nlepage commented Jan 17, 2019

@nlepage
Copy link
Member Author

nlepage commented Jan 25, 2019

Soooooo...
Right now on master (b6b470e) I get this :

ES2015 destructuring immutable 3.8.2 seamless-immutable 7.1.4 immer 1.12.1 qim 0.0.52 immutad●t 2.0.0 qim 0.0.52 curried immutad●t 2.0.0 curried
Set a property 5555557ops/s (180ns/op) 171347ops/s (5.84µs/op) 59411ops/s (16.8µs/op) 108341ops/s (9.23µs/op) 880437ops/s (1.14µs/op) 808669ops/s (1.24µs/op) 827541ops/s (1.21µs/op) 748391ops/s (1.34µs/op)
Set a deeply nested property 961724ops/s (1.04µs/op) 22309ops/s (44.8µs/op) 4789ops/s (209µs/op) 10879ops/s (91.9µs/op) 78197ops/s (12.8µs/op) 53322ops/s (18.8µs/op) 69412ops/s (14.4µs/op) 57379ops/s (17.4µs/op)
Update small todos list (1000 items) 6611ops/s (151µs/op) 1465ops/s (682µs/op) 416ops/s (2.40ms/op) 1242ops/s (805µs/op) 4356ops/s (230µs/op) 19795ops/s (50.5µs/op) 3931ops/s (254µs/op) 18376ops/s (54.4µs/op)
Update medium todos list (10000 items) 599ops/s (1.67ms/op) 125ops/s (7.99ms/op) 44ops/s (23.0ms/op) 120ops/s (8.34ms/op) 439ops/s (2.28ms/op) 3064ops/s (326µs/op) 488ops/s (2.05ms/op) 3004ops/s (333µs/op)
Update large todos list (100000 items) 58ops/s (17.4ms/op) 14ops/s (69.7ms/op) 5ops/s (209ms/op) 13ops/s (79.2ms/op) 44ops/s (22.6ms/op) 196ops/s (5.10ms/op) 44ops/s (22.7ms/op) 195ops/s (5.13ms/op)
Final score 100 15 5 13 48 234 48 228

On Update todos, it might be a false positive... I'll investigate.

@nlepage
Copy link
Member Author

nlepage commented Jan 25, 2019

So going back before rewriting the parser in TS (551e60d) it goes back to former results...
This would mean that somehow, rewriting the parser in TS gave a huge performance gain on slices!
Makes no sense...
@hgwood

@nlepage
Copy link
Member Author

nlepage commented Jan 25, 2019

And it's faster than ES2015 🙃

@nlepage
Copy link
Member Author

nlepage commented Jan 25, 2019

See #332
ES2015 updateTodos was slow...

@nlepage nlepage modified the milestones: 2.0, Future May 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant