Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run cells with specific language #892

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

shay1bz
Copy link

@shay1bz shay1bz commented Mar 4, 2024

Description

Addresses the same functionality as #598 but with some differences, this PR is for allowing users to execute cells with specific languages against the same Spark Session. Given that #598 is nearly 5 years old and seems abandoned, this implementation provides a minimal, backward-compatible solution.

Due to a lack of familiarity with the sparkmagic code base, no tests have been added. I kindly request guidance on how to implement unit tests.

The changes have been manually tested on a notebook, ensuring compatibility with shared DataFrames and UDFs between Scala and Python.

Checklist

  • Wrote a description of my changes above
  • Formatted my code with black
  • Added a bullet point for my changes to the top of the CHANGELOG.md file
  • Added or modified unit tests to reflect my changes
  • Manually tested with a notebook
  • If adding a feature, there is an example notebook and/or documentation in the README.md file

@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. kind:enhancement A new Sparkmagic feature labels Mar 4, 2024
@shay1bz
Copy link
Author

shay1bz commented Mar 12, 2024

@devstein would you mind taking a look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:enhancement A new Sparkmagic feature size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant