Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace boost::filesystem with std #6432

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

DennisOSRM
Copy link
Collaborator

@DennisOSRM DennisOSRM commented Oct 30, 2022

Resurrection of a previous PR from ~2 yrs ago that was blocked on a compiler upgrade.

Benchmark Results

Benchmark Base PR
alias aliased u32: 1084.1
plain u32: 1092.15
aliased double: 943.262
plain double: 951.262
aliased u32: 1093.49
plain u32: 1098.46
aliased double: 955.182
plain double: 945.487
e2e_match_ch Total: 2866.3229942321777ms
Min time: 2.378702163696289ms
Mean time: 21.880328200245632ms
Median time: 17.128944396972656ms
95th percentile: 69.81468200683594ms
99th percentile: 80.22732734680172ms
Max time: 91.30644798278809ms
Total: 2953.6821842193604ms
Min time: 2.3229122161865234ms
Mean time: 22.547192245949315ms
Median time: 16.93129539489746ms
95th percentile: 71.17104530334473ms
99th percentile: 93.03522109985337ms
Max time: 106.96721076965332ms
e2e_match_mld Total: 2036.7751121520996ms
Min time: 1.959085464477539ms
Mean time: 15.547901619481676ms
Median time: 8.754730224609375ms
95th percentile: 49.26502704620361ms
99th percentile: 59.59143638610834ms
Max time: 66.94531440734863ms
Total: 2023.53835105896ms
Min time: 1.8956661224365234ms
Mean time: 15.44685764167145ms
Median time: 9.106159210205078ms
95th percentile: 49.02195930480957ms
99th percentile: 58.420515060424755ms
Max time: 67.14892387390137ms
e2e_nearest_ch Total: 1360.6998920440674ms
Min time: 1.157522201538086ms
Mean time: 1.3606998920440674ms
Median time: 1.2727975845336914ms
95th percentile: 1.7728805541992188ms
99th percentile: 1.844923496246338ms
Max time: 2.0592212677001953ms
Total: 1340.0096893310547ms
Min time: 1.1315345764160156ms
Mean time: 1.3400096893310547ms
Median time: 1.252889633178711ms
95th percentile: 1.7454862594604492ms
99th percentile: 1.7991209030151367ms
Max time: 2.203702926635742ms
e2e_nearest_mld Total: 1376.8606185913086ms
Min time: 1.161336898803711ms
Mean time: 1.3768606185913086ms
Median time: 1.2848377227783203ms
95th percentile: 1.7946362495422363ms
99th percentile: 1.8730378150939941ms
Max time: 2.532482147216797ms
Total: 1342.3388004302979ms
Min time: 1.1341571807861328ms
Mean time: 1.3423388004302979ms
Median time: 1.257181167602539ms
95th percentile: 1.750779151916504ms
99th percentile: 1.8007922172546387ms
Max time: 1.8575191497802734ms
e2e_route_ch Total: 3286.7469787597656ms
Min time: 1.4109611511230469ms
Mean time: 3.2867469787597656ms
Median time: 3.338336944580078ms
95th percentile: 4.311764240264893ms
99th percentile: 4.7554707527160645ms
Max time: 7.899045944213867ms
Total: 3046.8108654022217ms
Min time: 1.3473033905029297ms
Mean time: 3.0468108654022217ms
Median time: 3.070235252380371ms
95th percentile: 3.989088535308838ms
99th percentile: 4.319231510162353ms
Max time: 5.080938339233398ms
e2e_route_mld Total: 3891.2603855133057ms
Min time: 1.409769058227539ms
Mean time: 3.8912603855133057ms
Median time: 3.9354562759399414ms
95th percentile: 5.363774299621582ms
99th percentile: 5.883448123931885ms
Max time: 8.146524429321289ms
Total: 3594.128131866455ms
Min time: 1.3620853424072266ms
Mean time: 3.594128131866455ms
Median time: 3.6159753799438477ms
95th percentile: 4.920792579650878ms
99th percentile: 5.352127552032471ms
Max time: 5.910396575927734ms
e2e_table_ch Total: 16748.405694961548ms
Min time: 2.274036407470703ms
Mean time: 16.748405694961548ms
Median time: 16.15273952484131ms
95th percentile: 30.414676666259762ms
99th percentile: 32.040865421295166ms
Max time: 34.389495849609375ms
Total: 16570.21999359131ms
Min time: 2.0542144775390625ms
Mean time: 16.57021999359131ms
Median time: 15.847444534301758ms
95th percentile: 30.512356758117676ms
99th percentile: 31.929316520690918ms
Max time: 34.13581848144531ms
e2e_table_mld Total: 64707.21697807312ms
Min time: 4.0760040283203125ms
Mean time: 64.70721697807312ms
Median time: 60.78791618347168ms
95th percentile: 124.93581771850586ms
99th percentile: 135.2039337158203ms
Max time: 144.10638809204102ms
Total: 63485.92114448547ms
Min time: 4.117488861083984ms
Mean time: 63.485921144485474ms
Median time: 60.352444648742676ms
95th percentile: 122.16832637786865ms
99th percentile: 130.7430386543274ms
Max time: 136.3832950592041ms
e2e_trip_ch Total: 11812.232732772827ms
Min time: 1.8534660339355469ms
Mean time: 11.812232732772827ms
Median time: 11.372208595275879ms
95th percentile: 20.015501976013184ms
99th percentile: 21.95692539215088ms
Max time: 23.360252380371094ms
Total: 10952.143669128418ms
Min time: 1.4870166778564453ms
Mean time: 10.952143669128418ms
Median time: 10.489463806152344ms
95th percentile: 18.980419635772705ms
99th percentile: 20.49271821975708ms
Max time: 21.42643928527832ms
e2e_trip_mld Total: 18348.47855567932ms
Min time: 1.6946792602539062ms
Mean time: 18.34847855567932ms
Median time: 17.85588264465332ms
95th percentile: 29.64588403701782ms
99th percentile: 31.76283597946167ms
Max time: 40.78340530395508ms
Total: 17659.510612487793ms
Min time: 1.4958381652832031ms
Mean time: 17.659510612487793ms
Median time: 17.229795455932617ms
95th percentile: 28.639817237854004ms
99th percentile: 30.400996208190918ms
Max time: 32.085418701171875ms
json-render String: 6.6256ms
Stringstream: 9.40825ms
Vector: 6.98584ms
String: 6.61516ms
Stringstream: 9.48784ms
Vector: 6.92702ms
match_ch Default radius:
4.41701ms/req at 82 coordinate
0.053866ms/coordinate
Radius 5m:
4.39726ms/req at 82 coordinate
0.0536251ms/coordinate
Radius 10m:
14.9905ms/req at 82 coordinate
0.182811ms/coordinate
Radius 15m:
36.6432ms/req at 82 coordinate
0.446868ms/coordinate
Radius 30m:
312.284ms/req at 82 coordinate
3.80834ms/coordinate
Default radius:
4.45263ms/req at 82 coordinate
0.0543003ms/coordinate
Radius 5m:
4.44804ms/req at 82 coordinate
0.0542444ms/coordinate
Radius 10m:
15.2578ms/req at 82 coordinate
0.18607ms/coordinate
Radius 15m:
37.3354ms/req at 82 coordinate
0.45531ms/coordinate
Radius 30m:
315.825ms/req at 82 coordinate
3.85153ms/coordinate
match_mld Default radius:
2.77595ms/req at 82 coordinate
0.0338531ms/coordinate
Radius 5m:
3.15799ms/req at 82 coordinate
0.038512ms/coordinate
Radius 10m:
10.0989ms/req at 82 coordinate
0.123157ms/coordinate
Radius 15m:
25.6677ms/req at 82 coordinate
0.313021ms/coordinate
Radius 30m:
301.457ms/req at 82 coordinate
3.6763ms/coordinate
Default radius:
2.78979ms/req at 82 coordinate
0.0340218ms/coordinate
Radius 5m:
2.77894ms/req at 82 coordinate
0.0338896ms/coordinate
Radius 10m:
10.3304ms/req at 82 coordinate
0.125981ms/coordinate
Radius 15m:
26.4537ms/req at 82 coordinate
0.322607ms/coordinate
Radius 30m:
307.43ms/req at 82 coordinate
3.74914ms/coordinate
osrm_contract Time: 97.28s Peak RAM: 185.60MB Time: 92.63s Peak RAM: 185.76MB
osrm_customize Time: 1.31s Peak RAM: 115.13MB Time: 1.30s Peak RAM: 115.15MB
osrm_extract Time: 12.53s Peak RAM: 413.84MB Time: 12.14s Peak RAM: 425.94MB
osrm_partition Time: 2.35s Peak RAM: 148.69MB Time: 2.11s Peak RAM: 148.50MB
packedvector random write:
std::vector 9744.76 ms
util::packed_vector 73708.4 ms
slowdown: 7.5639
random read:
std::vector 8382.99 ms
util::packed_vector 30172.8 ms
slowdown: 3.59929
random write:
std::vector 9784.24 ms
util::packed_vector 73961.1 ms
slowdown: 7.5592
random read:
std::vector 8450.05 ms
util::packed_vector 30341.6 ms
slowdown: 3.5907
route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
511.71ms
0.51171ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
351.75ms
0.35175ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
636.061ms
0.636061ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
150.861ms
0.150861ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
97.2494ms
0.0972494ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
132.307ms
0.132307ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
150.495ms
0.150495ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
96.7609ms
0.0967609ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
131.825ms
0.131825ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
506.616ms
0.506616ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
348.185ms
0.348185ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
621.29ms
0.62129ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
152.685ms
0.152685ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
99.4786ms
0.0994786ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
134.985ms
0.134985ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
151.681ms
0.151681ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
98.4373ms
0.0984373ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
134.081ms
0.134081ms/req
route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
641.501ms
0.641501ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
438.796ms
0.438796ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
815.427ms
0.815427ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
270.021ms
0.270021ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
163.533ms
0.163533ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
294.747ms
0.294747ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
271.422ms
0.271422ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
163.265ms
0.163265ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
294.765ms
0.294765ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
637.188ms
0.637188ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
436.324ms
0.436324ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
809.08ms
0.80908ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
269.899ms
0.269899ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
163.864ms
0.163864ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
291.164ms
0.291164ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
265.69ms
0.26569ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
163.422ms
0.163422ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
287.426ms
0.287426ms/req
rtree 1 result:
208.485ms -> 0.0208485 ms/query
10 results:
242.047ms -> 0.0242047 ms/query
1 result:
209.193ms -> 0.0209193 ms/query
10 results:
243.84ms -> 0.024384 ms/query

@DennisOSRM DennisOSRM marked this pull request as draft October 30, 2022 12:59
@DennisOSRM DennisOSRM closed this Nov 1, 2022
@DennisOSRM DennisOSRM reopened this Dec 10, 2022
@@ -42,7 +42,7 @@ module.exports = function () {
this.OSRM_PORT = process.env.OSRM_PORT && parseInt(process.env.OSRM_PORT) || 5000;
this.OSRM_IP = process.env.OSRM_IP || '127.0.0.1';
this.OSRM_CONNECTION_RETRIES = process.env.OSRM_CONNECTION_RETRIES && parseInt(process.env.OSRM_CONNECTION_RETRIES) || 10;
this.OSRM_CONNECTION_EXP_BACKOFF_COEF = process.env.OSRM_CONNECTION_EXP_BACKOFF_COEF && parseFloat(process.env.OSRM_CONNECTION_EXP_BACKOFF_COEF) || 1.0;
this.OSRM_CONNECTION_EXP_BACKOFF_COEF = process.env.OSRM_CONNECTION_EXP_BACKOFF_COEF && parseFloat(process.env.OSRM_CONNECTION_EXP_BACKOFF_COEF) || 1.1;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is to always use exponential back off. This fixed test runs on my dev machine

@DennisOSRM DennisOSRM marked this pull request as ready for review May 19, 2024 12:12
Copy link
Member

@SiarheiFedartsou SiarheiFedartsou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something is wrong with CI jobs using gcc-13 - it seems it cannot find compiler...

@SiarheiFedartsou
Copy link
Member

#6886

@DennisOSRM DennisOSRM marked this pull request as draft May 24, 2024 09:51
@SiarheiFedartsou
Copy link
Member

Hey @DennisOSRM
It seems MSVC was updated in base image, so may be we won't have issues with Windows in this PR anymore, may be let's try to rebase it after merge of #6930 ?
Feel free to ask me to do this rebase :)

@DennisOSRM
Copy link
Collaborator Author

Feel free to go ahead 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants