Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(snuba-search): snuba group dataset search handle additional columns p2 #70812

Merged

Conversation

scefali
Copy link
Member

@scefali scefali commented May 13, 2024

This PR handles additional filters for the snuba-heavy search:

  • issue:
  • project:
  • date:
  • first-release
  • has:x
  • Generic tags like server:

I'm attempting to use as many pieces from existing search code as possible which is why I'm leveraging the discover query builder as well as format_search_filter. It's kind of a Frankenstein monster of composed parts, however.

@scefali scefali requested review from a team as code owners May 13, 2024 21:07
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label May 13, 2024
Copy link

codecov bot commented May 13, 2024

Codecov Report

Attention: Patch coverage is 96.29630% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 80.04%. Comparing base (8c9fbbd) to head (b45f23d).
Report is 139 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #70812       +/-   ##
===========================================
+ Coverage   50.46%   80.04%   +29.58%     
===========================================
  Files        6469     6509       +40     
  Lines      289328   290807     +1479     
  Branches    49888    50126      +238     
===========================================
+ Hits       146003   232772    +86769     
+ Misses     142892    57601    -85291     
- Partials      433      434        +1     
Files Coverage Δ
...entry/issues/endpoints/organization_group_index.py 92.89% <ø> (+63.93%) ⬆️
src/sentry/search/events/builder/discover.py 96.34% <100.00%> (+74.25%) ⬆️
src/sentry/search/snuba/executors.py 96.33% <96.00%> (+73.89%) ⬆️

... and 2129 files with indirect coverage changes

@@ -1322,6 +1322,9 @@ def default_filter_converter(
1,
)
else:
# pull out the aliased expression if it exists
if isinstance(lhs, AliasedExpression):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we have to do this? will this affect the other discover search uses?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge it would just error out. I am pretty certain it doesn't impact discover or else we'd see the error about AliasedExpression

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does discover not need this previously?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge It only fails on the project_id input that's used as a filter. It seems like we only use this function default_filter_converter on small subset of filter types currently even though it's written to accommodate a lot more. I don't really know the history here and the code is convoluted. My guess is this function used to be used for more things but at some point only got used for a subset of possible filters.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it. mind just adding more info to that comment then?

Comment on lines 1232 to 1236
raw_column = map_field_name_from_format_search_filter(lhs[1][0])
rhs = [query_builder.resolve_column(raw_column)]
if len(lhs[1]) > 1:
rhs.append(lhs[1][1])
lhs = Function(lhs[0], rhs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we have to do this? looks weird

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge we have to deal with nested arrays

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, mind adding an example in the comment above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge added

raw_column = map_field_name_from_format_search_filter(lhs)
lhs = query_builder.resolve_column(raw_column)
else:
# need to convert dates to timestamps
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to do this timestamp business?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge because we need to convert from a UNIX timestamp to datetime

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? do we do this in our other query executors?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshFerge I realize now that we already handle the date filter with calculate_start_end so I can remove that special logic, we just skip that filter

self.get_basic_event_snuba_condition(
search_filter, joined_entity, organization.id, project_ids, environments
)
conditon = self.get_basic_event_snuba_condition(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: conditon -> condition

@scefali scefali merged commit 45a6bfc into master May 14, 2024
49 checks passed
@scefali scefali deleted the feat/snuba-group-dataset-search-handle-additional-columns-p2 branch May 14, 2024 20:49
Copy link

sentry-io bot commented May 15, 2024

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ IndexError: list index out of range /api/0/organizations/{organization_id_or_slug}/... View Issue
  • ‼️ TypeError: expected string or bytes-like object, got 'list' /api/0/organizations/{organization_id_or_slug}/... View Issue

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants