New Drizzle Kit config API

[AlexBlokh]

Drizzle ORM is designed to be dialect specific and used with a particular range of database drivers per dialect. While it lets Drizzle move fast and rapidly add support for whatever new dialects/drivers/databases emerged on the market, it makes it harder for newcomers to onboard.

As of now we're going through a major release of Drizzle Kit and adding support for mostly all drivers available, we decided to sit and talk through how we can make it better.

as of now developers need to specify both dialect and driver in the config file. Dialect being either of pg | mysql | sqlite and this list will be joined by mariadb | cockroachdb | mssql | turso(libsql) and on the other hand we have drivers like node-pg | postgresjs | neon-serverless | aws-data-api-pg | xataio and that's only for postgres dialect.
Drizzle config is meant to be typed and driver param presence infers dbCredentials object so that you can enjoy typed params experience. While we already have a variety of different drivers, their naming brings even more confusion and you end up with something like this:

import { Config } from "drizzle-kit";

export default {
  out: "./drizzle",
  dialect: "pg",
  driver: "pg", // or 'node-pg'
} satisfies Config;

in that case driver: "pg" is a node-postgres driver which is named pg on the npm
we also have postgresjs driver which is named postgres on the npm, which will lead to:

export default {
  out: "./drizzle",
  dialect: "pg",
  driver: "postgres", // or 'postgresjs'
} satisfies Config;

and it brings confusion, cause you as a developer install npm i pg but in our config it will be called node-pg which is how the library is called, or driver will be called pg and thus another confusion of why do you have to state dialect: pg and driver: pg. Goes the same with postgresjs driver which is going to be either dialect: pg and driver: postgres or driver: postgresjs.

We're aiming move away from the model of developer passing a driver instance to Drizzle constructor to the knex-like one, to reduce the initial learning curve/complexity for developers to jump into Drizzle:

import { drizzle } from "drizzle-orm"

const db = drizzle("postgres", { 
   ... // connection params
});

Taking in count all the above we've came up with potential solution for the config file to unify driver and dialect fields into one, which will consist of dialect:driver and change pg dialect to postgresql:

import { Config } from "drizzle-kit";

export default {
  out: "./drizzle",
  database: "postgresql:neon",
} satisfies Config;

That would let us keepdbCredentials typed and will end up in a list of database params be

// for postgres databases
// either of
postgresql:node-pg 
postgresql:pg

// either of
postgresql:postgres
postgresql:postgresjs

postgresql:neon
postgresql:aws-data-api
postgresql:xata

// for mysql databases
mysql:mysql2
mysql:planetscale
mysql:tidb

// for sqlite databases
sqlite:turso
sqlite:libsql
sqlite:bun
sqlite:better-sqlite3
sqlite:expo
sqlite:react-native

Big questions:

• this will change the behaviour for people who are not using push, pull and studio commands, since those only require dialect.
• When user provides postgres:neon for example, he might be able to connect to neon database not through @neondatabase/serverless driver, but through pg or postgres drivers. Should we check runtime deps strictly for @neondatabase/serverless and fail if it's not present or also try both pg and postgres

For the first one we might let them just provide database: "dialect" without :driver part?
For the second one we lean heavily towards trying all possible drivers and connect through the first one available as long as all of them compatible

[rphlmr]

in that case driver: "pg" is a node-postgres driver which is named pg on the npm

We should keep the name that is in the dependencies because node-postgres is something else in npm.

For the first one we might let them just provide database: "dialect" without :driver part?

So now it will be mandatory?
Is there a reason for that?
What will happen with only database: "dialect"?
Will we have to set dbCredentials too?
If yes, why should we then have to select a driver?

However, it can be hard to realize later that I need to add the driver to unlock specific config for the driver/vendor I use, so, if it is mandatory, better to be explicit.

For the second one we lean heavily towards trying all possible drivers and connect through the first one available as long as all of them compatible

There is a case I know that would instantly fail: PGLite (With dataDir constructor param, the database in the local file system, like SQLite).
It is a 'postgresql' dialect but its configuration will differ from other 'postgresql'.
So it will require its database: "postgresql:pglite" to have a proper configuration typing and its "official" driver to work.

Maintaining a list of cross-dialect compatibilities may be risky and time-consuming?

I would be strict: faster to implement, better configuration (what I set is what I use).
It should not really happen that often, right?

[AlexBlokh]

We went further and potentially managed to find a sweet spot. Our #1 goal is to cover 100% use cases while keeping most commonly used ones the simplest for the developers. pglight, aws-data-api users are edgier and potentially more experienced users as opposed to ones doing their first steps towards databases in nodejs/js world. We decided to not do database param and stick with dialect + optional driver

so lets say for postgres we won't require driver param for either you're using postgres server, postgres in docker container or neon. All you will have to do is specify dialect: postgres and we will connect to database with pg, postgres or @neondatabase/serverless whichever is currently installed in your project and dbCredentials stay fully typed and when you introduce driver: aws-data-api - we automatically infer credentials type for you

[rphlmr]

Looks simpler with this split 👍

[AlexBlokh]

perfect, thank you!