Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve test quality of org.elasticsearch.index.codec package #108706

Open
11 tasks
martijnvg opened this issue May 16, 2024 · 1 comment
Open
11 tasks

Improve test quality of org.elasticsearch.index.codec package #108706

martijnvg opened this issue May 16, 2024 · 1 comment
Assignees
@martijnvg
Labels
:StorageEngine/Logs You know, for Logs :StorageEngine/TSDB You know, for Metrics Team:StorageEngine >test Issues or PRs that are addressing/adding tests

Comments

@martijnvg
Copy link
Member

martijnvg commented May 16, 2024 •
edited

This is achieved by improving test coverage and adding test duels for specific codecs.
The following sub packages need increased test coverage:

  • org.elasticsearch.index.codec.bloomfilter package. Currently 90% of lines and 71% of branches tested.
  • org.elasticsearch.index.codec.postings package. Currently 94% of lines and 79% of branches tested.
  • org.elasticsearch.index.codec.tsdb package. Currently 97% of lines and 85% of branches tested.
  • org.elasticsearch.index.codec.zstd package. Currently 93% of lines and 50% (?) of branches tested.

The following classes need increased test coverage:

  • CodecService. Currently 81% of lines and 62% of branches tested.
  • ForUtil. Currently 93% of lines and 82% of branches tested.
  • LegacyPerFieldMapperCodec. Currently 85% of lines and 50% of branches tested.
  • PerFieldFormatSupplier. Currently 75% of lines and 61% of branches tested.
  • PerFieldMapperCodec. Currently 57% of lines and 50% of branches tested.

Lastly, we should add test duel for a few custom codecs:

  • Tsdb doc values codec. Write a test that generates random content and encode this using the default Lucene doc values codec and then by our tsdb doc values codec. Assert that both codecs produce the same output. Added tsdb doc values codec test duel. #108853
  • Zstd stored fields. Write a test that generated random content and encode it using Lucene's default stored fields codec and then the zstd codec. Then finally assert that both Lucene indices read the same content.
@martijnvg martijnvg self-assigned this May 16, 2024
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label May 16, 2024
@martijnvg martijnvg added >test Issues or PRs that are addressing/adding tests :StorageEngine/TSDB You know, for Metrics :StorageEngine/Logs You know, for Logs and removed needs:triage Requires assignment of a team area label labels May 16, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

martijnvg added a commit to martijnvg/elasticsearch that referenced this issue May 21, 2024
@martijnvg
Added tsdb doc values codec test duel.
78b338c 
The test duel indexes random content in two different Lucene indices. The first index uses the default doc values codec codec and the second index the tsdb doc values codec. The test duel then asserts that both Lucene indices produce the same content.

Relates to elastic#108706
@martijnvg martijnvg mentioned this issue May 21, 2024
Merged
martijnvg added a commit that referenced this issue May 22, 2024
@martijnvg
Added tsdb doc values codec test duel. (#108853)
908fdaf 
The test duel indexes random content in two different Lucene indices. The first index uses the default doc values codec codec and the second index the tsdb doc values codec. The test duel then asserts that both Lucene indices produce the same content.

Relates to #108706
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
:StorageEngine/Logs You know, for Logs :StorageEngine/TSDB You know, for Metrics Team:StorageEngine >test Issues or PRs that are addressing/adding tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@martijnvg @elasticsearchmachine