Add /refresh to update sources and some configurations

[mesudBe-Orbit]

Try to fix #288

• Inject Arc<RwLock> to server, T: state.tiles, state.sprites, state.fonts, catalog, config
• Add a /refresh to endpoint(read and merge config and flush the cache)
• Add test to tesh.sh
• Update doc

[nyurik]

looks like a good start, thanks! I will have to think of how to test this best - perhaps add a new shell script test-watch.sh that is similar to existing test.sh, except that it will:

• start martin with --auto-bounds skip (so that it starts faster)
• wait for it to start
• get a catalog into catalog1.json
• use psql to create a new table and a function, and possibly make some other modifications like deleting something else
• use curl to POST a refresh command to some Martin's endpoint that triggers the refresh - I guess this request should not return until refresh is done? Make sure to have some global "refresh lock" to avoid multiple simultaneous refresh requests?
• get another catalog saving it to catalog2.txt

The existing bless/diff setup will check that the catalog results are as expected

[sharkAndshark]

Some ways to push this forward?

[sharkAndshark]

I'm trying to keep going on this.

[nyurik]

There is a well known dashmap crate that essentially functions as a concurrent hashmap (i.e. replaces RwLock<HashMap<K, V>>). We may want to use that as a simpler alternative to store sources / fonts / sprites. Or it might be a bad idea - only time will tell.

[sharkAndshark]

And should we keep the watch config option like if the watch is enabled the /refresh would be added and if it's disabled would be not.

[nyurik]

lets implement /refresh separately - there are a lot of functionality required to make this work - so i think first we should get all the plumbing in place to make it possible, and then we will introduce:

• watching file system for changes (e.g. pmtiles / mbtiles / sprints / fonts)
• timer based refresh of postgres
• calling POST on /refresh, possibly restricting it to localhost only

[sharkAndshark]

Seems it's almost there? Will fix the tests and update doc later.

[sharkAndshark]

Weired. That shouldn't fail. CI / Build and test docker images

Testing points3857_srid_0_0_0.pbf from http://localhost:3111/points3857/0/0/0
***** Test server response for PMTiles source *****
Testing pmt.json from http://localhost:3111/stamen_toner__raster_CC-BY-ODbL_z3
curl: (22) The requested URL returned error: 404

The source stamen_toner__raster_CC-BY-ODbL_z3 has been configured actually.

[2024-06-05T02:55:58Z WARN  martin::utils::id_resolver] Source `stamen_toner__raster_CC-BY+ODbL_z3` (/tests/fixtures/pmtiles/stamen_toner__raster_CC-BY+ODbL_z3.pmtiles) was renamed to `stamen_toner__raster_CC-BY-ODbL_z3`. Source IDs must be unique, cannot be reserved, and must contain alpha-numeric characters or `._-`
[2024-06-05T02:55:58Z INFO  martin::file_config] Configured source stamen_toner__raster_CC-BY-ODbL_z3 from /tests/fixtures/pmtiles/stamen_toner__raster_CC-BY+ODbL_z3.pmtiles

[sharkAndshark]

There must be a lot to clean up.