Stuck in executing notebook

[bsridatta]

🐛 Bug

Launching the execute_notebook() gets stuck after 100% completion. Upon interrupting, looks like its waiting for something forever.
I can also see that the output notebook is also saved with the results, but the method doesnt complete.
Any help is much appreciated, Thank you!

Environment:

Name: papermill
Version: 2.4.0

Name: nbclient
Version: 0.7.2

Name: nbformat
Version: 5.7.3

Executing: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 6/6 [03:46<00:00, 37.71s/cell]
Traceback (most recent call last):
  File "src/train.py", line 49, in generate_report
    pm.execute_notebook(
  File "/home/xxx/lib/python3.8/site-packages/papermill/execute.py", line 113, in execute_notebook
    nb = papermill_engines.execute_notebook_with_engine(
  File "/home/xxx/lib/python3.8/site-packages/papermill/engines.py", line 49, in execute_notebook_with_engine
    return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
  File "/home/xxx/lib/python3.8/site-packages/papermill/engines.py", line 367, in execute_notebook
    cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
  File "/home/xxx/lib/python3.8/site-packages/papermill/engines.py", line 436, in execute_managed_notebook
    return PapermillNotebookClient(nb_man, **final_kwargs).execute()
  File "/home/xxx/lib/python3.8/site-packages/papermill/clientwrap.py", line 46, in execute
    info_msg = self.wait_for_reply(self.kc.kernel_info())
  File "/home/xxx/lib/python3.8/site-packages/jupyter_core/utils/__init__.py", line 168, in wrapped
    return loop.run_until_complete(inner)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1823, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.8/selectors.py", line 468, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

[bsridatta]

Downgrading jupyter-client to 7.3.5 worked. But seems to be dependency conflicts with other packages. Give other issues like not being able to open the output notebook (cannot read properties of undefined reading 'onDidChangeContent') any better solution here? Thank you

[kdubovikov]

This issue reproduces for me as well in Github Actions

[jjstader]

I had this same problem executing papermill from the cli, for me downgrading jupyter-client to 7.4.9 fixed the problem

[woutdenolf]

I encountered the same problem and #726 fixed it for me.

I saw a failure with that fix as well so I closed it.