Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): Add Workspace API #7661

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft

feat(sdk): Add Workspace API #7661

wants to merge 32 commits into from

Conversation

andrewtruong
Copy link
Contributor

@andrewtruong andrewtruong commented May 17, 2024

Description

This PR adds limited programmatic control over the workspace in python, including:

  • Panels (configs, layout, etc.)
  • Sections (names, ordering, etc.)
  • The runs side bar (visibility, colours, etc.)

Example API usage:

ws.View(
    entity="megatruong",
    project="workspace-api3",
    name="my custom view !!",
    sections=[
        ws.Section(
            name="Hello world I am a test 🔥",
            panels=[
                wr.LinePlot(y=["val_loss"], title="My amazing line plot of val loss"),
                wr.BarPlot(metrics=["val_accuracy"], title="A fantastic bar plot"),
                wr.ScatterPlot(x="precision", y="recall"),
            ],
        ),
    ],
).save()

Produces this workspace:
image

See the colab for more samples:
https://colab.research.google.com/drive/1JSdvDecgKurQlxQG0TAO3GeiYc-fbr_Q#scrollTo=ZAfkP1dtYPgO

  • I updated CHANGELOG.md, or it's not applicable

Testing

How was this PR tested?

Copy link

codecov bot commented May 17, 2024

Codecov Report

Attention: Patch coverage is 0% with 577 lines in your changes are missing coverage. Please review.

Project coverage is 71.24%. Comparing base (0d2f959) to head (5dcf54b).
Report is 3 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #7661      +/-   ##
==========================================
- Coverage   74.36%   71.24%   -3.13%     
==========================================
  Files         500      504       +4     
  Lines       55789    54636    -1153     
==========================================
- Hits        41489    38926    -2563     
- Misses      13888    15300    +1412     
+ Partials      412      410       -2     
Flag Coverage Δ
func 40.91% <0.00%> (-0.41%) ⬇️
system 59.76% <0.00%> (-3.99%) ⬇️
unit 53.14% <0.00%> (-2.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
wandb/apis/workspaces/__init__.py 0.00% <0.00%> (ø)
wandb/apis/reports/v2/internal.py 0.00% <0.00%> (-93.69%) ⬇️
wandb/apis/workspaces/internal.py 0.00% <0.00%> (ø)
wandb/apis/reports/v2/interface.py 0.00% <0.00%> (-86.81%) ⬇️
wandb/apis/workspaces/expr.py 0.00% <0.00%> (ø)
wandb/apis/workspaces/interface.py 0.00% <0.00%> (ø)

... and 124 files with indirect coverage changes

@andrewtruong andrewtruong requested a review from ssisk May 22, 2024 18:04
wandb/apis/workspaces/expr.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/expr.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/expr.py Outdated Show resolved Hide resolved
Comment on lines +12 to +20
# these internal objects should be factored out into a separate module as a
# shared dependency between Workspaces and Reports API
from wandb.apis.reports.v2.internal import * # noqa: F403
from wandb.apis.reports.v2.internal import (
PanelBankConfig,
PanelBankSectionConfig,
Ref,
Runset,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please - I think this should happen first before we get the workspaces API in.
I recommend pulling shared logic first because it'll make tracing code and debugging more clear esp for those who are looking at it for the first time

wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Show resolved Hide resolved
wandb/apis/workspaces/interface.py Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/internal.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Outdated Show resolved Hide resolved
wandb/apis/workspaces/interface.py Outdated Show resolved Hide resolved
@jo-fang
Copy link
Contributor

jo-fang commented May 29, 2024

commenting where our default values lie in FE because we need to match it (after discussing with Andrew)

// features/workspaceSettings/store.ts
// from https://wandb.atlassian.net/browse/WB-17958
export const DefaultWorkspaceSettings = {
  pointVisualizationMethod: POINT_VISUALIZATION_OPTIONS.SamplingByDefault,

  // sections
  showEmptySections: false,
  autoOrganizePrefix: OrganizationPrefix.LastPrefix,
  sortAlphabetically: false,

  // panels
  suppressLegends: false,

  // tooltips
  tooltipNumberOfRuns: TooltipNumberOfRunsOptions.Default,
  highlightedCompanionRunOnly: false,
  colorRunNames: true,

  // runs
  showMinMaxOnHover: false,
};
// features/workspaceSettings/defaults.ts
/**
 * Get the max runs that are defaulted in the panels
 * If a user can use the custom default, return that value
 */
export const DEFAULT_NUM_RUNS_OLD = 10; // legacy setting
/**
 * This was bumped to 30 because want to start increasing the default number
 * of runs in a workspace panel. However, this slows down the RSDQ given that
 * backend parallelism is limited and not likely to be delivered imminently. So
 * we ware bumping this value back to 10 until we can process things more
 * efficiently on the backend.
 *
 * https://weightsandbiases.slack.com/archives/C010Y174QGH/p1714422864169979
 */
export const DEFAULT_NUM_RUNS_NEW = 10;
export const DEFAULT_MAX_NUM_RUNS = 300; // users cannot set max runs higher than this value
export const DEFAULT_MAX_NUM_BUCKETED_RUNS = 50;
export const DEFAULT_MAX_GROUP_RUNS = 100;

// panelsettings.ts
export const EMPTY_XAXIS_SETTINGS: XAxisSettings = {
  xAxis: PlotHelpers.XAxisValues.Step,
  xAxisMin: undefined,
  xAxisMax: undefined,
};

export const EMPTY_SMOOTHING_SETTINGS: SmoothingSettings = {
  smoothingWeight: 0,
  smoothingType: 'exponential',
};

export const EMPTY_SETTINGS: Settings = {
  ...EMPTY_XAXIS_SETTINGS,
  ...EMPTY_SMOOTHING_SETTINGS,
  ignoreOutliers: false,
  xAxisActive: false,
  smoothingActive: false,
  useRunsTableGroupingInPanels: true,
};

// panelbank.ts
export const EMPTY_PANEL_BANK_SETTINGS: PanelBankSettings = {
  autoOrganizePrefix: OrganizationPrefix.LastPrefix,
  showEmptySections: false,
  sortAlphabetically: false,
};

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants