feat: Better templating for @vercel/postgres

[tcc-sejohnson]

TODO:

• Security audit
• API approval
• README

What problem does this fix?

Right now, SQL templating is very limited. This is cool:

// this works great!
const { rows } = await sql`SELECT * FROM users WHERE user_id = ${userId}`;

... but it falls apart quickly:

// nope!
const { rows } = await sql`SELECT * FROM users WHERE user_id IN (${userIds})`;

It completely falls apart for any sort of dynamic query building, such as is often required for bulk UPDATE / INSERT statements. This adds the following API surface area:

Utility	Signature	Purpose
debug	(strings: TemplateStringsArray, ...values: unknown[]) => [string, unknown[]]	Returns a tuple of your query and its parameters -- useful if you need to log the final compiled query to see what's going wrong.
fragment	(strings: TemplateStringsArray, ...values: unknown[]) => [string, unknown[]]	Has the same API as query, but returns a TqlFragment node which can be recursively nested within itself and included in a top-level query.
identifiers	(ids: string | string[]) => TqlIdentifiers	Accepts a list of strings, escapes them, and inserts them into the query as identifiers (table or column names). Identifiers are safe and easy to escape, unlike query values! Will also accept a single identifier, for convenience.
list	(parameters: unknown[]) => TqlList	Accepts a list of anything and inserts it into the query as a parameterized list. For example, [1, 2, 3] would become ($1, $2, $3) with the original values stored in the parameters array.
values	(entries: ValuesObject) => TqlValues, where ValuesObject is { [columnName: string]: unknown } or an array of that object type.	Accepts an array of records (or, for convenience, a single record) and builds a VALUES clause out of it. See the example below for a full explanation.
set	(entry: SetObject) => TqlSet, where SetObject is { [columnName: string]: unknown }.	Accepts a record representing the SET clause, and returns a parameterized SET clause. See example below for a full explanation.
unsafe	(str: string) => TqlTemplateString	Accepts a string and returns a representation of the string that will be inserted VERBATIM, UNESCAPED into the compiled query. Please, for all that is good, it's in the name -- this is unsafe. Do not use it unless you absolutely know your input is safe.

The API for sql remains unchanged.

An example, using the above (obviously somewhat contrived to show how various API works):

import { sql, fragment, identifiers, values } from '@vercel/postgres';

const newUser = { first_name: 'vercelliott', family_name: 'smith' };

// these could come from a column picker
const columns = identifiers(['first_name', 'family_name']);

const insertStatement = fragment`
  INSERT INTO users ${values(newUser)}
  RETURNING family_name
`;

const { rows: familyMembers } = await sql`
WITH new_user AS (
  ${insertStatement}
)

SELECT ${columns} 
FROM users
WHERE family_name IN (SELECT family_name FROM new_user);
`

Full docs for the TQL spec

[changeset-bot]

🦋 Changeset detected

Latest commit: 710f1c0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@vercel/postgres	Minor
@vercel/postgres-kysely	Minor
vercel-storage-integration-test-suite	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
vercel-storage-next-integration-test-suite	✅ Ready (Inspect)	Visit Preview	Nov 21, 2023 11:53pm

[tcc-sejohnson]

the type of values is now unknown[] instead of Primitive[], but this is backwards-compatible because all Primitive values are assignable to unknown.

[luismeyer]

maybe this is just me, but i expected debug to behave differently. Here is how:

1. I run into a bad query and get some neon db error i don't understand
2. I go into the code and switch sql with debug
3. get the query logged or thrown

instead the return type of debug is different to sql, so TS is unhappy and also my runtime error changed because, for example i can't read .rows on the result of debug. Does it make sense to wrap this with a little function to enhance DX event more?

[tcc-sejohnson]

Hmm, that might be a good call... log the query and the parameters (with a check on something like process.env.DEVELOPMENT or something) and still send the query to the database...