You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to set up a single configuration package that all of my other packages will "inherit" from (let's call it confpkg).
The idea is to be able to change the code in confpkg to add "standard" configuration values, which can then be overridden in whatever package is being developed (call it yourpkg), or some dependency that also uses confpkg (call it mid_pkg).
Desired Solution
It is desired to have this work with absolute minimal user interaction, and to have a very nice api exposed in the code of yourpkg.
Now this package ('yourpkg') has these available:
- CONF['data_dir'] == "/home/user/.local/share/yourpkg/data" (Or w/e platformdirs outputs)
- CONF['cache_dir'] == "/home/user/.cache/yourpkg"
- etc - many different directories from package dirs
- CONF['pkg_name'] == __package__ (The package name will be 'yourpkg' here, but will change for different packages)
- CONF['pkg_dir'] == {some logic typically like: os.path.dirname(os.path.realpath(__file__))} (This should work to give the 'yourpkg' top-level package directory, automatically)
These values should of course be overridden by the 'yourpkg' directory's config.toml file.
This can allow easily setting things like a 'default' database filepath, etc.
(The "Another Example / Perspective" section covers how confpkg should handle other logic around this.)
Another example (from yourpkg still) using mid_pkg, to demonstrate this further:
frommid_pkgimportBigDatabase# NOTE: Also uses this same `confpkg` line within the `mid_pkg` codefromconfpkgimportpkg_setupCONF, LOG=pkg_setup()
db=BigDatabase()
# DB is created using it's packages code (where CONF from `confpkg` is called),# which can contain a default `data_dir` computed by `platformdirs` - specific to the `mid_pkg`db.query("select * from Docs")
However, specifically, it is desired to not have to have the same config_maker.py script in package A, B, C, etc - but rather just to have oneconfig_maker.py or config.py in the confpkg package, which will load the directories and add the package name and dir - for the package that is calling the CONF, LOG = pkgsetup().
What is a good way to make this happen?
Solution Attempts
I have solved some of this with the inspect package, or having to load the string of the file from a package, and then execute it from within the calling package.
Pkg_C needs a data_dir which will hold, say 3TB of space (to construct a database for Pkg_A).
This can be achieved with the system desired in this post - specifically a separate confpkg that can construct or append to a config.toml, (or just create entries in CONF without touching config.toml) for Pkg_A, Pkg_B, and Pkg_C, that has defaults from platform_dirs.
However, importantly here - the confpkg should check if that default can allow for this space, and ask the user for input to alter the config.tomlif and only if needed.
It's clear that copy/pasting this code (probably a few hundred lines) to every single package created by the user is not a viable solution. Especially if an institution wants to make an addition for all of the packages that confpkg uses (e.g. a standard 'welcome message' for a LOG or something.
So what is a good way to deal with this (extremely common) type of scenario?
The text was updated successfully, but these errors were encountered:
Hello, sorry for the delay. I have looked at this but didn't know how to respond immediately.
This looks like a useful use case, but we don't provide a specific out-of-the-box solution for the problem as you describe. As a library, Dynaconf provides some features for handling config, such as getting config from multiple sources, merging then, lazy evaluating, defining validators, etc. I believe you could leverage those features to implement this kind of "extensible framework" that you want. And if there is any specific gap/feature that you feel is missing to support this, it might be easier to request and implement that (as opposed to this more complex proposal).
Your use case looks somehow similar to the project I work on.
It uses a plugin system for django, and it use dynaconf to layer the settings like (sorted by loading order):
Problem
I am trying to set up a single configuration package that all of my other packages will "inherit" from (let's call it
confpkg
).The idea is to be able to change the code in
confpkg
to add "standard" configuration values, which can then be overridden in whatever package is being developed (call ityourpkg
), or some dependency that also usesconfpkg
(call itmid_pkg
).Desired Solution
It is desired to have this work with absolute minimal user interaction, and to have a very nice api exposed in the code of
yourpkg
.For example, in
yourpkg
:Those two lines are all that need to be included.
Now this package ('yourpkg') has these available:
-
CONF['data_dir'] == "/home/user/.local/share/yourpkg/data"
(Or w/eplatformdirs
outputs)-
CONF['cache_dir'] == "/home/user/.cache/yourpkg"
- etc - many different directories from package dirs
-
CONF['pkg_name'] == __package__
(The package name will be 'yourpkg' here, but will change for different packages)-
CONF['pkg_dir'] ==
{some logic typically like:os.path.dirname(os.path.realpath(__file__))
} (This should work to give the 'yourpkg' top-level package directory, automatically)These values should of course be overridden by the 'yourpkg' directory's config.toml file.
This can allow easily setting things like a 'default' database filepath, etc.
(The "Another Example / Perspective" section covers how
confpkg
should handle other logic around this.)Another example (from
yourpkg
still) usingmid_pkg
, to demonstrate this further:However, specifically, it is desired to not have to have the same
config_maker.py
script in package A, B, C, etc - but rather just to have oneconfig_maker.py
orconfig.py
in theconfpkg
package, which will load the directories and add the package name and dir - for the package that is calling the CONF, LOG = pkgsetup().What is a good way to make this happen?
Solution Attempts
I have solved some of this with the
inspect
package, or having to load the string of the file from a package, and then execute it from within the calling package.The solution used functions similar to this: https://gitlab.com/chase_brown/myappname/-/blob/12113f788b84a4d642e4f7f275fe4200b15f0685/myappname/util.py#L15-41
(*Yes, the package is basically useless and you shouldn't use it, but it illustrates the point)
However, I just noticed
Dynaconf
exists, and it seems to be very close to what I need (if not a perfect fit - I am still learning this package).So I figured I should not re-invent the wheel, and rather use the standard tools that are extremely popular in the language.
Another Example / Perspective
A good example to illustrate the problem is the following:
data_dir
which will hold, say 3TB of space (to construct a database for Pkg_A).This can be achieved with the system desired in this post - specifically a separate
confpkg
that can construct or append to aconfig.toml
, (or just create entries in CONF without touchingconfig.toml
) for Pkg_A, Pkg_B, and Pkg_C, that has defaults fromplatform_dirs
.However, importantly here - the
confpkg
should check if that default can allow for this space, and ask the user for input to alter theconfig.toml
if and only if needed.It's clear that copy/pasting this code (probably a few hundred lines) to every single package created by the user is not a viable solution. Especially if an institution wants to make an addition for all of the packages that
confpkg
uses (e.g. a standard 'welcome message' for a LOG or something.So what is a good way to deal with this (extremely common) type of scenario?
The text was updated successfully, but these errors were encountered: