implement tunable registry defaults for registry and update mirrors #4521
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the problem this change solves?
In large environments with many cloud instances are running
ollama serve
, accidentally pushing code to runollama pull llama3
can result in 100's of cloud instances are trying to download fromollama.ai
.The correct change for production should have been
ollama pull https://registry.prod.someside.tld/library/llama3
. The registry mirror atregistry.prod.someside.tld
is necessary to reduce bandwidth costs for high volume data, like an AI model or container image.Mistakes like this can go unnoticed by novices building scalable infrastructure for their developers, until they get the resulting bill.
Also registry owners often have to implement rate limiting to keep bandwidth costs down. Hitting a rate limit in a production environment often results in an outage. Further making convenient mirroring options desirable.
What are the changes being made?
defaults
to hold tunable values.github.com/ollama/ollama/types/defaults
Are there any tasks remaining?
I need some guidance on how testing should work for these changes.