The future of jsforce

[cristiand391]

Hey folks! We have some exciting news to share.

After talking with @stomita , we've agreed the Salesforce CLI team will help maintain jsforce!
jsforce is a core component of the Salesforce CLI, Salesforce Extensions for VSCode, DevOps Center and a bunch of other Salesforce tools and we hope we can help Shinichi keep the project alive while delivering new features.

We know there hasn't been much activity triaging issues and PRs except for a few specific bug fixes/features, you can expect us to be more active in the repo while we work towards the v3 GA.

Dropping support for old node and browsers

jsforce v2 has been in beta for almost 3 years supporting node >= v8 and some old browsers (polyfills).
To avoid breaking projects already using v2 we plan to skip and go straight to v3 for the GA, requiring node >= 18 (current LTS) and modern browsers.

Browser usage

While we don't use jsforce in browsers (CLI and others run via node), we are committed to keep jsforce an isomorphic library.
We've already improved our CI test setup to run browser tests on latest Chrome, updated webpack/babel deps and fixed some flaky tests to ensure a safe path while we finish the last bits of work.

We will also remove the edge versions (/dist) these were already outdated when using a CDN. If you wish to continue using them, you can continue using the CDN versions here and the jsforce npm package includes these so they are available in modern CDNs that serve from npm.

Remaining work

• Migration guide v1 → v3
• Triage issues and PRs targeting v2 branch
• Documentation equivalent to the current jsforce.github.io (http://jsforce.github.io/). PR ci: use typedoc for docs #1362
• Drop support for old node and browsers https://github.com/jsforce/jsforce/pull/1369/files#r1363986316
• node-only release, see: node-only release #1374
• Annotations to the code for API references.
• Run CLI tests on PRs to catch regressions PR ci: add Salesforce CLI NUTs to the test matrix #1359
• Run node test on Windows
• Automate TS typegen (followup issue)

[gmaliar]

@cristiand391 how can we be helpful? The remaining work is unclear in how we can participate.

[techieshark]

we hope we can help Shinichi keep the project alive while delivering new features.

This is fantastic news! Very cool that Salesforce is investing in open source. As a Salesforce user/customer, very glad to see this happening as this library is very important to us as well.

Would love to know if Salesforce will be sponsoring Shinichi or any other open source developers?

Also curious how Salesforce will prioritise work - will it be any/all of the open issues, or issues with most activity, or PRs the community puts forward, or more tightly focused on features that more closely align with the CLI team's needs? I imagine knowing this might help I or others know how best to collaborate.

[cristiand391]

@gmaliar

The CLI team is currently working on the listed work above. I wouldn't feel comfortable asking people to contribute while we still have a ton of PRs to triage. That said, if anyone is interested in working on something, just ping me in the issue/pr and we can discuss any fix/feature request before submitting a PR 👍🏼

[cristiand391]

@techieshark

Would love to know if Salesforce will be sponsoring Shinichi or any other open source developers?

Salesforce has a program called FOSS Fund where employees can nominate and vote on projects to receive $10K, this is done every quarter. I know jsforce won it a few years ago (before I joined Salesforce) but seems it ended up rejected, see:
https://opencollective.com/salesforce/expenses/40748

@stomita do you remember what happened with that grant?

Anyway, the one for this quarter is closed but I would expect jsforce could be nominated for the next one (will be checking with the program manager since it's jsforce isn't owned by Salesforce but now we help maintain it).

I'm not aware of other Salesforce program to fund individual developers.

Also curious how Salesforce will prioritise work - will it be any/all of the open issues, or issues with most activity, or PRs the community puts forward, or more tightly focused on features that more closely align with the CLI team's needs? I imagine knowing this might help I or others know how best to collaborate.

We'll probably be fixing a few bugs as we triage issues and PRs targeting v2.
Features/bug fixes that align with our team's need would be our priority but as part of maintainers we'll also review community contributions and/or work on stuff not related to the CLI as much as we can.
In the past other internal teams offered to work on new features and we helped them reviewing/testing (Heroku added support for the Bulk V2 api in jsforce v2).

[FeiPengDev]

Sorry to disturb you by asking questions here. I would like to ask if there is any progress on the following issues?
I am using webpack5 now and I still get errors as below
#1222
#1270

[cristiand391]

Hey @FeiPengDev, sorry for the late reply. If you still have issues with webpack 5 see this comment: #1222 (comment)

[cristiand391]

quick update:
node-only plan: #1374

feedback is welcome 👍🏼

I'll post here once the first node-only release is up on npm (right now we published an empty pkg to reserver the name)

[cristiand391]

Alright, a few hours ago we published the first v3 releases under the next dist tag on npm:

jsforce:
https://www.npmjs.com/package/jsforce/v/3.0.0-next.1

jsforce-node:
https://www.npmjs.com/package/@jsforce/jsforce-node/v/3.0.0-next.1

I'll be on PTO next week but will start putting a few of the planned breaking changes for v3 here when I'm back.

[pavitra-infocusp]

Where can I learn about v3 API changes? Link to docs?

[john-bflow-mcgowan]

@cristiand391: first of all, thanks for working on this awesome package! It's made approaching a new integration a lot more digestible.

Like @pavitra-infocusp I'm also curious to know about documentation for V2 and V3. Does the currently hosted documentation at https://jsforce.github.io/ reflect the changes already in the V2 beta version? I'm unable to find any reference to a jsforce version number other than what is mentioned in specific package method documentation (as seen in the Operation for Multiple Records section for example) -- but i'm assuming that the live docs refer to the latest stable release 1.11.1, and not V2.

I see the TODO for Documentation (linked below) is marked as complete. Is the command referenced a way to generate docs locally?

[X] Documentation equivalent to the current [jsforce.github.io](http://jsforce.github.io/) (http://jsforce.github.io/). PR https://github.com/jsforce/jsforce/pull/1362

[cristiand391]

Hey @john-bflow-mcgowan , sorry for the late reply 🙏🏼

Does the currently hosted documentation at https://jsforce.github.io/ reflect the changes already in the V2 beta version?

no, we had those under https://jsforce.github.io/jsforce/doc/ for v2 but had to revert back to v1 docs because of the current docs setup.

I see the TODO for Documentation (linked below) is marked as complete. Is the command referenced a way to generate docs locally?

yeah, those generate API docs using typedoc, the end result is smt like this:
https://forcedotcom.github.io/sfdx-core/

but that's just the API reference, the document section will also be updated for v3 changes
https://jsforce.github.io/document/

[pavitra-infocusp]

@cristiand391 Is there any expected release date for v3?

[tehnrd]

I would like to request that the base package be truly isomorphic, as it is not today. With many frameworks that do server-side rendering (SSR), like Sveltekit and Next.js, this library does not work. These run javascript on the server but do so with a browser runtime (not node) for rendering, so if you include node modules like Buffer, it will fail to build and run.

For example, if you import jsforce like this:
import jsforce from 'jsforce';

And then when you attempt to build for a serverless environment like Cloudflare Pages, you will receive the following error:

✘ [ERROR] Could not resolve "buffer"

    node_modules/core-util-is/lib/util.js:103:27:
      103 │ exports.isBuffer = require('buffer').Buffer.isBuffer;
          ╵                            ~~~~~~~~

  The package "buffer" wasn't found on the file system but is built into node. Are you trying to
  bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

As this library is rebuilt to use modules, it would be good to import node-specific libraries with the following import as many hosting platforms will automatically polyfill these:

// Do this:
import { Buffer } from 'node:buffer';

// Do not do this:
import { Buffer } from 'buffer';

[cristiand391]

@tehnrd which version of jsforce are you using?

that import comes from core-util-is which I don't see being shipped in prod. versions.

These run javascript on the server but do so with a browser runtime (not node) for rendering, so if you include node modules like Buffer, it will fail to build and run.

Yeah, by isomorphic we mean node + browsers for now.
Not sure if we would "officially" support new runtimes now, but maybe if it's not too much effort we can help with stuff like you mentioned about imports, feel free to open new issues if something doesn't work with the node-compat mode 👍🏼 .

IIRC, using node:* imports triggered some errors in the build/bundle process with webpack but I can check again.

[tehnrd]

I tried 1.11.1 and 3.0.0-next.3 but I need to do some deeper debugging.

@cristiand391 ya, I will spin up a new issue. It think (haven't investigated too much) the issue is that there is code directly referencing modules like Buffer (and others) object globally but this isn't recommended in the official node docs and it should be explicitly imported with the node: prefix. This way it can easily be polyfilled when doing builds across runtimes.

While the Buffer class is available within the global scope, it is still recommended to explicitly reference it via an import or require statement.

const { Buffer } = require('node:buffer');

// or

import { Buffer } from 'node:buffer';

https://nodejs.org/api/buffer.html#buffer

[satanshiro]

the newest releases are not part of the -next- versioning format yet the prerelease tag is still existent.
depandabot has been recommending to upgrade for this reason.
is the pre release over?

[GBvanDam]

I believe Connection.query(soql) is the only way to retrieve results in the format containing both totalSize, nextRecordsUrl and records. As far as I know this function only accepts a soql query in string format. I have been looking for a way to retrieve this response structure whilst using the conditions in JSON format (just like in conn.sobject("Contact").find({ ...conditions }).

If I did not miss any different method to accomplish this whilst using the conditions in object style, I think there are several possibilities:

1. open up the Query class such that we can make calls like new Query(conn, { ...conditions }, locator).toSOQL() (currently this class is protected)
2. support conditions in object style in the Connection.query() call
3. or anything that I'm currently overlooking

Could you enlighten me, or could this become a new feature?

[stomita]

@GBvanDam

open up the Query class such that we can make calls like new Query(conn, { ...conditions }, locator).toSOQL() (currently this class is protected)

Actually the find() method returns Query object so you can do the following:

const soql = await conn.sobject("Contact").find({ ...conditions }).toSOQL();
const queryResult = await conn.query(soql);

[GBvanDam]

Thank you @stomita I clearly oversaw that option!