Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Diem Data Client] Implement intelligent data summary polling. #10113

Open
JoshLind opened this issue Jan 4, 2022 · 1 comment
Open

[Diem Data Client] Implement intelligent data summary polling. #10113

JoshLind opened this issue Jan 4, 2022 · 1 comment

Comments

@JoshLind
Copy link
Contributor

JoshLind commented Jan 4, 2022

Today, the Diem Data Client is very primitive: in order to identify the availability of data around it, it samples the storage summaries of peers in a random fashion. As a result, nodes often don't have a sufficient view of the state around them until enough samples have been taken (which may take a non-negligible amount of time).

To address this, we should make the Diem Data Client a little more intelligent:

  1. First, when a new peer connects to a node, the Diem Data Client should immediately poll that peer (i.e., to identify the peer's data summary as soon as possible).
  2. Second, when regularly polling peers (i.e., to stay up-to-date with changes in data availability), the Diem Data Client should avoid using random sampling and instead provide some form of fairness or guarantees (e.g., a round-robin scheme, or selecting the next peer by identifying the one with the oldest storage summary).

There may also be other optimizations/improvements we can make, but I'll leave that up to the reader 😄.

@JoshLind
Copy link
Contributor Author

JoshLind commented Jan 4, 2022

'cc @khiemngo -- something you may be interested in tackling when you get the time! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant