Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON spec (ChainTypes) for defining chain specific types #3

Merged
merged 15 commits into from
May 21, 2024

Conversation

jsdw
Copy link
Collaborator

@jsdw jsdw commented May 14, 2024

The main thing we added here is a ChainTypes type.

  • This can be deserialized from a given JSON format.
  • One can then call ChainTypes::for_spec_version to get something that's ready to resolve types for a specific spec version.
  • One can also merge Chaintypes instances via .extend to eg merge a JSON file with basic substrate types with a JSON file with eg polkadot specific types on top.

Under this, a TypeRegistrySet is added, which is a borrowed (or owned) set of TypeRegistrys that resolves types from them in a specific order, falling back to previous registries if types aren't found, and resolving pallet types before global ones.

@jsdw jsdw marked this pull request as ready for review May 14, 2024 16:38
@jsdw jsdw requested a review from a team as a code owner May 14, 2024 16:38
///
/// # Example
///
/// ```rust
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example is a good starting place to understand the format (or see the tests) :)


/// Extend this chain type registry with the one provided. In case of any matches, the provided types
/// will overwrite the existing ones.
pub fn extend(&mut self, other: ChainTypeRegistry) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh cool, so this kind of mirrors the Overrides in PJS. Very cool.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I added this is mainly that we might want t odefine a load of general substrate types in a JSON file somewhere, and then define eg polkadot specific types in another file, and then load both and merge them to get the ChainTypeRegistry for a chain (polkadot in this case).

Copy link
Member

@niklasad1 niklasad1 May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't read the implementation yet but how are we dealing with duplicate JSON keys in the ChainTypeRegistry?

I have done weird stuff with JSON before, duplicate keys not recommended :)

Copy link
Collaborator Author

@jsdw jsdw May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a high level, the HashMap serde impl will handle it (ie it probably will overwrite earlier keys with later ones). The enum logic will just assume that an enum has two variants with the same name, and structs would have two fields of the same name.

If you mean dupe keys as in, name mentioned in "global" types and also in some specific spec range or whatever, then any type in a specific "pallet" wins, and any type in the last seen spec range wins!

Overall I think it's ok if such behaviour is "undefined" and it's a user error to provide dupe keys in the same object :D

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think it's ok if such behaviour is "undefined" and it's a user error to provide dupe keys in the same object :D

Correct

Cargo.toml Outdated Show resolved Hide resolved
Copy link
Member

@TarikGul TarikGul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks really solid to me!

Copy link

@lexnv lexnv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
TypeNotFound(String),
LookupNameInvalid(String, lookup_name::ParseError),
#[display(fmt = "Type not found")]
TypeNotFound,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be useful to know which type/name that wasn't found but there are others that already displays that such as LookupNameInvalid, so what's the reason why it is omitted here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question; I'll have to remind myself why I removed the string, because yeah it would be useful!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so actually I removed TypeNotFound entirely, since I should be calling visitor.visit_not_found() instead, I think the idea there was that the person who hands the visitor also hands the type ID, so they will always know what it is anyway, but I'm not super convinced and will have to play around and see whether we should be removing .visit_not_found() or adding the TypeId as an arg to it or whatever!

/// can not be parsed into a valid [`TypeName`].
pub fn parse_unwrap(input: &str) -> TypeName {
Self::parse(input).unwrap()
impl core::convert::TryFrom<&str> for LookupName {
Copy link
Member

@niklasad1 niklasad1 May 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think when we been down this route before but it doesn't work to impl this from TryFrom<AsRef<str>>, no?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, impl <T: AsRef<str>> TryFrom<T> is no good because there's already a generic impl on T in the std library (and it doesn't consider trait bounds!

impl core::fmt::Display for TypeName {
impl PartialEq for LookupName {
fn eq(&self, other: &Self) -> bool {
// Bit of a hack, but it does the trick:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine by me but can't you implement PartialEq for TypeNameDef and compare them instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hid this impl behind a cfg(test) for now, since we only care about this comparison stuff in testing, and it's a bit weird :)

Ok(DeserializableShape::AliasOf(name))
}

// Either '{ _enum: ... }' for enum descriptions, or '{ a: ..., b: ... }' for struct descriptions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dq: is this a notation you came up with to distinguish enums and structs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@niklasad1 niklasad1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good stuff :)

@jsdw jsdw merged commit 74ec121 into main May 21, 2024
7 checks passed
@jsdw jsdw deleted the jsdw-type-registry-set branch May 21, 2024 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants