Jupyter Flowchart Notebook

[CharlesAverill]

Hi there,

I'm new to the Jupyter community so I hope I've gone through this submission process properly. I'd like to add a new UI to Jupyter that allows users to format notebooks in a branching flowchart style. I think the addition of a more graphical UI could benefit users that prefer visuals to plain text while learning and creating.

I've received a bit of positive feedback from friends and colleagues who say they would also enjoy a flowchart-based design.

EDIT: I'd also like to mention that I've already started working on this idea, but in a different format. Satyrn performs most of the basic functions that Jupyter notebooks can, but is built for multi-threaded code. It's completely Python-based, but the experience from that project will make it a lot easier to add this functionality to Jupyter.

Thank you!

Charles

[blink1073]

Hi @CharlesAverill, thanks for the proposal! I personally like the idea, having played with Scratch with my elementary-school-aged children.

[meeseeksmachine]

This pull request has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/flowchart-based-notebook-ui-idea/4541/2

[choldgraf]

It feels to me like this might benefit from some rounds of prototyping etc before making it into a JEP, perhaps with some general discussion in the community forum (linked above, but here again: https://discourse.jupyter.org)

Also ping @alexmorley who I believe was interested in this idea as well!

[CharlesAverill]

@choldgraf I definitely agree on prototyping. I'm working on something right now that should be able to eventually lay the groundwork for implementation in Jupyter, but it's still pretty far away from that point. I just wanted to get the word out early to see if I could find any teammates for the project.

[alexmorley]

Hey 👋 I can dig out the designs I did for this a while back. I have a post on the forum, the work their didn't really go ahead as planned but would love to revive it. https://discourse.jupyter.org/t/closed-experimental-jupyter-ux-web-development-contract-opportunity/1314

[CharlesAverill]

@alexmorley That would be great! I looked through your git and it looks like we've got a lot of the same ideas. If you're up to reviving the project then I'm sure we can get a lot done.

[bollwyvl]

Just linked over from the above (QuantStack/jupyterlab-drawio#61), so I'll toss some thoughts out there. Given the above, I'm biased but:

The JupyterLab examples would be fine places to start, but at that point, just having a labextension would be the stronger play. nteract is also a beautiful set of components, but (to my knowledge) has only two paths: a) get into core, or b) build your own thing out of them.

Again, biased, but mxgraph/drawio might be a good base layer. Here's their ports example of tying blocks that contain arbitrary HTML

• it's pretty opinionated, but very robust
• codemirror (and no doubt other things) don't like being embedded in a foreignObject inside an svg, but maybe it's just a bookkeeping chore to keep a rect and a div in the same place

making something work really well with python is good... but making something that worked with every kernel...

• instead of interacting at the code/token level, the flow could be a picture of kernel messages which make things happen on the client and/or server
• interacting at the kernel message layer would even allow a single flow to connect multiple running kernels
• guard conditions (however they're visually encoded) could be things like,
  • "when A runs to completion without error, run B"
  • "when B's text/csv output changes, run C (preceded with this generated code)"
  • "when C widget's trait value changes, change this other widget's value"

Picking a well-characterized formalism, rather than inventing something new (however it's visually encoded) is probably worth considering. These usually have some kind of conformance suite or at least reference implementation that can be used for testing. Some examples might be:

• petri nets
• hierarchical finite state machines
• UML activity diagrams

Having written a few wall-chart sized "programs" in SysML constraints (think strongly-typed, visual excel formulae), a few things become extremely important at any scale beyond toy/tutorial:

• complexity hiding/collapsing
  • which might as well handle importing/transclusion out of the box
• search: by content, guard/edge type, upstream/downstream
• auto layout: the strongest open source option at this time is probably elkjs
• annotation
• version control

It's worth pointing out that a number of existing tools already provide visual flows of various sorts that work with various parts of the Jupyter stack, but they usually treat a whole notebook as a node (and are therefore kernel-agnostic), and aren't as concerned with the authoring of the individual code blocks:

• https://github.com/cylc
• https://github.com/dagster-io/dagster
• https://github.com/PrefectHQ/prefect

Good luck!

[Zsailer]

Hi @CharlesAverill 👋—Zach from the @jupyter/software-steering-council here.

We're working through old JEPs and closing proposals that are no longer active or may not be relevant anymore. Under Jupyter's new governance model, we have an active Software Steering Council who reviews JEPs weekly. We are catching up on the backlog now. Since there has been no active discussion on this JEP in awhile, I'd propose we close it here (we'll leave it open for two more weeks in case you'd like to revive the conversation). If you would like to re-open the discussion after we close it, you are welcome to do that too.