Encoding more information in the virtual scroll bar

[mlucool]

Problem

The virtual scroll bar is nice, but now that its a custom scrollbar, it seems like we can use it to encode more information.

Proposed Solution

For example, maybe code and markdown cells should look slightly different. We could also imagine encoding something about which are the big code cells with some sort of very slim sparkline. I'm not sure of any exact idea - but it seems navigation could be greatly enhanced by hinting at what is in the cell. Other ideas including showing which cells have an error etc.

Above cell 3 is markdown (gray) and cell 2 has more than 2x the code than cell 4.

Additional context

[krassowski]

Related to:

• "minimap" of notebook structure for navigation #3738
• https://github.com/bollwyvl/ipython-notebook-navigator

We do not want to render a mini version of the cell in the scrollbar as this would take away the performance benefits of virtualization.

• We already have the size information about cells and outputs, we could resize the scrollbar cells accordingly.
• I think showing the execution count number by the code cells could be very useful, because right now there is a disconnect between cell index and the execution number which users see in the notebook. I think the cell index is not needed if we have a rich representation of the cells, but it could still be shown on hover.
• For code we could store information about number of lines in the editor, and:
  • use 1-3 variants like:
    • for an editor with a lot of text/code,
    • for an editor with medium amount of text, and
    • for an editor with little text.
  • or we could even generate the number of lines (and lengths) dynamically to to match the text content; this will still be way faster to render than syntax highlighted code
• Users will be interested to find cells with specific outputs, cells which are executing, and cells which failed to execute; we could show:
  • a spinner in output section of cells which are executing
  • an alert for cells which have an exception traceback in outputs
  • I think that a generic text output should have no icon; maybe text lines like in editor possibly in a different colour
• For outputs which include plots we could show:
  • or one image/plot icon regardless of number of plots/outputs
  • or show "multiple pictures" icon if there are multiple plots
  • show one icon per one output

We could also have a different icon for ipywidgets too.

There are different tradeoff depending on which information we decide to display. For example encoding both the execution count/cell type and the cell editor content seems to leave no space for output unless we break with the "each cell occupies equal height in the scrollbar" convention (or make the cell box very wide). Showing just the editor content and output seems like a better use of space but then there is no execution counter to show:

Execution counter/cell type + editor content	Editor content (left) + output (right)

In the Editor content + output option we could show the execution counter on hover. The executing cells could have a spinner (and in a future a progress bar, one API for this is available).

[krassowski]

Implementation-wise we would need to modify the virtual renderer for the notebook:

jupyterlab/packages/notebook/src/widget.ts

Lines 1317 to 1335 in d86e3cd

	createScrollbar(): HTMLOListElement {
	return document.createElement('ol');
	},
	createScrollbarItem(
	notebook: Notebook,
	index: number,
	model: ICellModel
	): HTMLLIElement {
	const li = document.createElement('li');
	li.appendChild(document.createTextNode(`${index + 1}`));
	if (notebook.activeCellIndex === index) {
	li.classList.add('jp-mod-active');
	}
	if (notebook.selectedCells.some(cell => model === cell.model)) {
	li.classList.add('jp-mod-selected');
	}
	return li;
	}

it looks like it already can access cells so rendering it should be easy (set aside cross-browser CSS issues we encountered during the initial implementation).

We may need to ensure that rendering is selectively updated when cell editor content, execution status or outputs change. Currently the entire scrollbar is redrawn on every update request, but as we start making the nodes more complex and need to synchronise state more frequently we need more granular rendering updates.

[mlucool]

• We already have the size information about cells and outputs, we could resize the scrollbar cells accordingly.

I like that they are all the same size now. It's a good mental model for me

• I think showing the execution count number by the code cells could be very useful, because right now there is a disconnect between cell index and the execution number which users see in the notebook. I think the cell index is not needed if we have a rich representation of the cells, but it could still be shown on hover.

I'm on the fence here, but I see the argument to go to execution number.

• For code we could store information about number of lines in the editor, and:

  • use 1-3 variants like:

How would we define these amount of code? I also find the icons proposed a bit large

• or we could even generate the number of lines (and lengths) dynamically to to match the text content; this will still be way faster to render than syntax highlighted code

This is similar to the line I proposed above - but mine was more of a hint.

• Users will be interested to find cells with specific outputs, cells which are executing, and cells which failed to execute; we could show:

I like the idea of showing a spinner and an alert for failed cells. I agree - success should not show anything in your face (I could see executed vs. not having a very subtle indication)

• For outputs which include plots we could show:

I personally think this gets too cluttered.

There are different tradeoff depending on which information we decide to display. For example encoding both the execution count/cell type and the cell editor content seems to leave no space for output unless we break with the "each cell occupies equal height in the scrollbar" convention (or make the cell box very wide). Showing just the editor content and output seems like a better use of space but then there is no execution counter to show:

I think we should really keep these as minimal hints and shy away from anything too in your face. How about:

1. Cell colors for type and error (maybe this is a shade of red)
2. Icons for important things that change the cells status: running, queued.
3. Numbering system is configurable (execution count or cell number)
4. Two constants for code size. If is <10 LOC it shows a small marking, < 50 LOC a medium one, and >50 a larger marking. I'd still prefer something very slim and subtle.

[krassowski]

I like that they are all the same size now. It's a good mental model for me

I agree that this is advantageous, especially because very long cells can be skipped easily. If we wanted to go for the squiggle lines representing the actual number of code lines in the editor, we could cap it at showing at most say 20 lines + a gradient or symbol to indicate that there is more lines.

I also find the icons proposed a bit large

My screenshots of the MDI icons were possibly a bit large, and we may simplify them slightly, but there is only so much we can do here.

1. Cell colors for type and error (maybe this is a shade of red)

Using color would conflict with using color for cell selection status. Currently we use light blue for selection and dark blue for active cell:

A frequent feature of minimap is to overlay both selection, and the boundary of the viewport, as done by https://github.com/replit/codemirror-minimap (blue - selection, grey - viewport):

If we use colors for cell types (and execution status), then this becomes tricky, especially when we get to the details of making it meet AA accessibility standards.

We could:

• (a) increase the border width and then use the border color to indicate selection/active cell, and make background indicate cell type/status (but this is still conflating multiple types of information with the same visual indicator), or
• (b) strive for a more faithful representation of the notebook structure where each cell would have two boxes, one for input and one for output (if present), and the background around these boxes would change color to indicate selection/active status; then the content of the boxes can take any color (but also, markdown cells would have no output, so they would already be easier to spot).

I feel like (b) could be more intuitive to users, although the size of the scrollbar would increase.

[mlucool]

The highlighting that is there seems like a waste of all that space as users rarely multi-select cells (at least in my exp). I was thinking its outline as it is in the notebook. That is:

Exclude the bolds/lack of thinness of the lines, but now you can see what is on screen at any time easily (which to me is the a key part of the minimap).

Now we still have some room inside the cell to mark running/queued (e.g. reusing the *) and use a simple single color to call out markdown (e.g. a very light gray).

[krassowski]

now you can see what is on screen at any time easily (which to me is the a key part of the minimap)

For reference this was previously included in a mockup of virtual scrollbar by @GabrielaVives here: #13343 (comment) but by using background of the cell:

I think we could have some kind of outline box with high transparency, like in the codemirror-minimap:

this still allows to see the colors underneath.

[krassowski]

#16392 makes a first steps by implementing the viewport indicator and refactoring code so that we can add more information to the scrollbar without performance penalty:

[krassowski]

The next step is adding more information. From quick experimentation I found out that it is feasible to just add the source code of the cell (without syntax highlighting/rendering markdown/math - anything) to the scrollbar and it works very smooth even with very large notebooks - unfortunately the recording does not do it justice (it skips many frames each time I try to use the browser scrollbar - might be a Wayland compositor issue on my side):

It is currently trimmed to a few lines but we can adjust this further.

I will now implement running status indicator and open a draft PR to allow for testing.