Metrics - query by date range

[ccollie]

Looking at the current implementation of metrics, it seems that data is stored in a list as opposed to a sorted-set, so it's not clear if there is a way of accessing data by date range rather than index. IOW, if I suspect that there is a spike every day at noon, there no non-trivial way to get data specifically for that period (other than separately storing the timestamp of the first data point and doing some offset math)

In collectMetrics, the timestamp is passed in so we associate it with the data in a sorted-set. Was the decision to use a list based on considerations of memory usage ?

In any case I think it is not useful as is IMO, because it seems to be a timeseries without timestamps.
If memory is a concern, we can (as suggested) store the initial timestamp in the meta key, and provided data is stored consistently at 1-minute intervals, perform range searches using simple math.

Thoughts, @manast ?

[manast]

Take a look at "collectMetrics.lua", we are storing all the data necessary in a meta key in order to be able to perform ranges, aggregations, and what not. A list is both more efficient and less memory consuming, and also enough for our purposes.

[ccollie]

I took a look. Let me be more specific - how can I for example get completed counts for 3 days ago between 12:00 and 1pm ? i have a vague idea of how, but it doesn't seem immediately obvious since there is no API to surface the functionality. That's understandable given that it's new - and I guess I'm the only one currently asking.

I may just go ahead and create a PR if I have time in the next few days.

[manast]

Since you know the last metric point timestamp and that every point represents 1 minute, you should be able to compute the range.

[ccollie]

Thanks, just as I thought...