Drop login options / gatekeeper/ Heroku?

[BryanQuigley]

1. Heroku is dropping their free plan
2. Many CMSes now exist for editing static sites https://jamstack.org/headless-cms/
3. Would simplify the goals of the JKAN project away from editing/etc.

This would involve deleting the admin page, the add-dataset, add-organization, dropping the ability to login.

If this changed would affect your org -please let us know!

[JackGilmore]

A potential stopgap solution could be to use Render which is a similar service to Heroku. It offers 750 free usage hours a month and doesn't require a credit card to sign up.

I managed to set it up pretty easily for my test instance of JKAN. You basically just plug in the link to the Gatekeeper repo and tell Render what commands to use to build and run the app. This is what I'm considering using instead of Heroku for our JKAN instance on opendata.scot

[BryanQuigley]

Render won't work for my use case (and how long will free last, see Heroku..) - although it does seem pretty neat. Gatekeeper maintenance doesn't seem active which might be a bigger issue.

My plan would be to switch to:

• add an edit on Github button. (This would just be editing Markdown on GH)
• Drop logins and all the edit/add html files and all the related code (which is a good amount).

Does a Markdown workflow block any of your users?

[JackGilmore]

Out of interest, what is your use case?

To be honest, we don't really used the front-end editing that much unless we're onboarding new contributors to the project who want to get started with something simple like updating organization details. All our dataset pages are generated by a python script as part of our dataset pipeline so we don't really use the editors for those. In short, removing the editor front-end and editing raw markdown files shouldn't block our users.

We already have an "Open in GitHub" button on our org and dataset pages like the one you described and they work well for jumping from the page to the source behind it. You can see examples here:

• Dataset: https://opendata.scot/datasets/aberdeen+city+council-aberdeen+city+council+contracts+register/
• Organization: https://opendata.scot/organizations/aberdeen_city_council/

[BryanQuigley]

We just don't want to have any external services as dependencies for this project. Our use cases is migrating https://www.opendataphilly.org/ form CKAN to JKAN.

+1 on no blockers for dropping front-end editing.

Awesome re:Open in GitHub, will bring the edit in Github bits into JKAN first.

[BryanQuigley]

Tried to go and remove it but the intermediate steps were easier: #196 #195

In my testing so far dropping the login/inline editing capabilities could reduce the size of Javascript by more then 50%

I meant to ping earlier, but wanted to know if this change would affect any of you (and if so, how): @pezholio @keeganmcbride @JJediny @tursics @amercader @lxyu0405 @tobinbradley @wilsaj @davidread @dracos

While I'm pinging all previous contributors, I wanted to ask if you want to be more involved in JKAN upstream - I can ask you for reviews or we can try to pick up some older PRs.

[JackGilmore]

I'm happy to help pick up some of the JKAN upstream stuff. Please feel free to tag me in PRs or request review @BryanQuigley @timwis

[timwis]

Oh my goodness...I setup some email filters to hide a subset of GitHub notifications, and apparently made them too eager, because I completely missed the ones from this repo this month! 🙈 Awfully sorry about that. Funny enough, I was knee-deep in this two weeks ago, too! I realised that authentication on demo.jkan.io would break once Heroku kicked off the free demo gatekeeper instance, so I wanted to migrate it to something else as a stop gap, and then write up some thoughts about the this idea as well (decoupling editing).

I first tried migrating it to vercel, which I've used plenty of times for Node.js apps, and failed miserably. They seem to expect the function to be in the api directory, and I quickly exceeded my 'timebox' trying to figure out how to customise it. So I moved it to render, which I'd also heard about, and it was pretty easy. The next step would have been to configure the "deploy to render" button, but I ran out of time (and had similar concerns to what @BryanQuigley raised above). Only once I finished did I realise that literally no one uses the authentication feature on the demo site anyway, because I'm the only one who has write access to github.com/timwis/jkan-demo 🤦🏻‍♂️ Brilliant.

Will write up my thoughts on the wider question of auth/editing shortly and post here.

[JackGilmore]

As a sidenote whilst we're on the topic of the getting started process with JKAN. I had wondered whether it's worth changing the setup process from forking to setting up the upstream repository as a template repository instead? Given that we'll all have different datasets and orgs we're hosting, I didn't really see a scenario in which forking makes sense for contributing upstream from our own JKAN instances?

Whilst I've not looked in detail, if I understand template repos correctly, I think it's still possible to pull downstream changes from the original template into your instance if you wanted to pull in new changes and bug fixes? This was a short article I found on the topic https://medium.com/geekculture/how-to-use-git-to-downstream-changes-from-a-template-9f0de9347cc2

[timwis]

Hey @JackGilmore that's a really interesting idea. @BryanQuigley also mentioned on our call a few weeks ago the idea of potentially pulling in JKAN as a gem (lots of new possibilities are open now that we could potentially switch to using GitHub Actions to build JKAN instead of the built-in jekyll builder). Would you mind posting a new discussion topic about that?

[timwis]

I've given this some thought since our chat, @BryanQuigley. Given that:

• Anecdotally, most organisations using JKAN are editing the markdown/yaml directly, rather than via the UI
• As you point out, a large percentage of JKAN's code base is for the editing UI

I'd agree that it makes sense to decouple the editing UI from the core of JKAN. I'd suggest we continue to offer an editing UI, but have it be opt-in. The nice thing about having the site's database sit in a public github repo is we can just have a completely separate application point at it, and the user-facing presentation application is none-the-wiser. (EDIT: although Netlify CMS may provide a nice middle ground here)

This would also let JKAN remain a purely static site, hosted for free, and contain the server-side stuff to the optional editing UI journey.

As I've been thinking about this, though, I've come to a number of crossroads in my mind, and realised that it may be helpful to make the projects guiding principles more explicit (and surface some areas where they're not entirely defined). I'll post those separately for discussion.

EDIT: Just had another idea.. might be a bit out there, but we could run a central jkan editing instance, which any org can use to update their jkan site. Of course they could also run the editing instance themselves, but for those who want to skip the faff of hosting, this could be quite simple.

[BryanQuigley]

Another limitation is that the UI editing doesn't work for those who don't have Write access to the repo - no drive by contributions by this nicer way.

It certainly could work making the UI editing opt-in or even a separate repo. The idea of hosting it centrally is neat - although might be a good amount of work.

I'm not sure I will have time to invest in the UI editing components (we are happy with markdown flow), but my preference would be a workflow something like this:

1. cd to local_edit subdirectory (this could be a separate git repo that is optional to pull in)
2. Run docker compose there, which loads up the site with local editing enabled
3. Changes or new files are written out to the local fs directly
4. It's on the user to open a PR with the correct new files/changes

Happy to reduce the dependency on docker compose to just Docker if that would help (would let other container system more easily do it..).

[timwis]

Actually, I was playing around with Netlify CMS for this on Friday and I think it may provide a really compelling solution here with minimal build effort required. I'll try to pull together a little demo later this week.

[timwis]

I've just posted a demo in #203 - let me know what you think!