Change default value for origin_time

[ThetaSinner]

Summary

I'm proposing this change because it's caught me out a couple of times and I think it's a confusing default.

The pros for changing it:

• Timestamp::now() moves so that the gossip range for each install is different. This is valid from Kitsune's point of view but probably not what the author intended.
• This is the most common value I've seen for this field. Only a few happs make use of this and they use it to discard older data on a network where a new version of an app has been released that is compatible with the previous one but they don't want to see the old data. So in a sense, this is already the default.

The cons for changing it:

• It's possible to set the epoch time, but not possible to set Timestamp::now() if somebody does want this behavior.
• It's potentially quite a surprising change if any happ is relying on it.

I'm inclined to say that if happ devs want to control visibility of existing data when new users join a network then that should be app behavior and not Kitsune skipping gossiping. If users of a happ want to skip syncing a lot of historical data because they don't care about it, they're free to set a recent origin_time when they install the happ.

So I think with this change all the behavior we want is maintained, but with a less surprising default value for this field.

TODO:

• CHANGELOG(s) updated with appropriate info
• Just before pressing the merge button, ensure new entries to CHANGELOG(s) are still under the UNRELEASED heading

[maackle]

There are other important implications to this. One is that origin_time is a useful lower bound on the range of valid timestamps, and if we have an origin time going all the way back to the 70's (or even back to last year), that's just more wiggle room for bad actors to try something.

More importantly, the origin_time is fundamental to the way we do gossip and is the basis of a lot of other future features we want to add. In particular the further back from the current moment the origin_time is, the more "regions" we need to use to represent that history with. The number of regions we send at each gossip round is logarithmically proportional to the total length of history since origin_time.

I loaded this into my brain again and I might not have it perfectly but I'll try to show the math and get to the result: we need to send an average of something like

$$ 32 \cdot 12 \cdot log_2 \frac{ \Delta t }{ q } $$

bytes at each gossip round just to figure out how up to date we are. $\Delta t$ is the time since origin_time. $q$ is the quantum time, 5 minutes. 32 is the number of bytes per region hash. 12 is the mean number of region hashes per time slice.

So one way to say this is, every time $\Delta t$ doubles in terms of quantum time, we have to use an extra 384 bytes at every gossip round. If $\Delta t$ is 50 years back to the UNIX epoch, then that's 23 time slices, so 8.5 KB of region data per round. If it's 1 year, that takes 17 slices, or 6.5 KB per round. So that's 2KB of wasted bandwidth every gossip round just for picking a long-ago epoch.

A lot of this could be tuned differently, and we could even put in a threshold so we stop logarithmically growing the space we need to represent the distant past up to a point which would make the origin time less important. But this is currently how things stand.

Also, if we do this, Timestamp::ZERO is not actually the UNIX epoch, it's much further back. We could use HOLOCHAIN_EPOCH though, which is Jan 1 2022. That would start us at around the 6.5KB per round mark which seems reasonable.

A question is, do the benefits outweigh the cost? Maybe so, since the cost is quite low right now. But there is a cost.

[ThetaSinner]

Thanks for the details, I hadn't realised it was quite that much extra data if this default gets used.

I would like to see this field set by each app and the default not used ideally because I don't think there's an ideal default value.

What I'm after is just a value that doesn't move on different installs because that means new nodes joining a network expecting to see existing data will quietly not see it. So if the default value is a fixed point in time and usually overridden that'd be good.

Alternatively, this could be a required field and we could make hc app pack set a value of Timestamp::now() if no value is set. That would be very similar to what we have now but it wouldn't move on different installs. It would potentially be equally confusing though when updating an app's coordinator zomes. Existing users would keep the original origin time and new users installing the updated app would get a different value... So that's not a great option either. The value needs to be decided with some thought by the developer. So a required field is an option but automatically setting it probably isn't.

[maackle]

Oh I didn't realize that was your impetus. If different people install the same app, I'm pretty sure they're not all going to get different origin times. The default is just on the Builder, which I think only gets used in a handful of places, and only in tests.

The place where now() is used for real DNAs is in hc init. But as soon as the DNA is constructed, anyone who's installing it will get the same value that it was built with. The fact that multiple people are able to join the same network at all I think attests to this.

I can't vouch for the origin time not changing when updating coordinator zomes, but any time you have a DNA manifest at hand, and use it to do anything, those values will not be changed automatically. Pretty sure the only time defaults are invoked is when you're creating a manifest from scratch, which you only typically do at the very beginning of a development project, once.

[ThetaSinner]

Ahh okay, I ran into this in a Kitsune test where I thought I ended up with different timestamps but maybe I did something there that isn't possible on code paths from Holochain, or maybe something else was wrong.

I'll do some more testing with this and come back with some evidence or maybe this can just be closed :)

[github-actions]

This item has been open for 30 days with no activity.

[github-actions]

This item has been open for 30 days with no activity.

[maackle]

@abe-njama I think it's OK to let stalebot do its thing and close stale PRs like this one. We all get notified when it goes stale after a month, and if nobody does anything about that, it's probably safe to let it fade away

[github-actions]

This item has been open for 30 days with no activity.