Avoid returning data for incomplete/partial days for new dashboards

[PatrickNercessian]

As per @bajtos

We are already 'using Grafana's feature called "time shift" to hide the most recent data in the charts. "-1d" for daily values, "-30d" for monthly values' for other dashboards

But 'Maybe we can implement this inside the spark-stats service? We already have some logic to handle date ranges; it understands the concept of “today”. But maybe not “this month”.'

spark-stats/lib/request-helpers.js

Lines 22 to 38 in 5e1b373

	// Provide default values for "from" and "to" when not specified
	if (!to) {
	to = today()
	shouldRedirect = true
	}
	if (!from) {
	from = to
	shouldRedirect = true
	}
	if (shouldRedirect) {
	res.setHeader('cache-control', `public, max-age=${600 /* 10min */}`)
	res.setHeader('location', `${pathname}?${new URLSearchParams({ from, to })}`)
	res.writeHead(302) // Found
	res.end()
	return { from, to }
	}

Maybe we can modify the SQL queries to something like this: MAX(filter.to, date_trunc('month', now()) - "1 month")

If we never return stats for the current period (day/week/month), then we can also simplify the response cache control header:

spark-stats/lib/request-helpers.js

Lines 75 to 87 in 5e1b373

	const setCacheControlForStatsResponse = (res, filter) => {
	// We cannot simply compare filter.to vs today() because there may be a delay in finalizing
	// stats for the previous day. Let's allow up to one hour for the finalization.
	const boundary = getDayAsISOString(new Date(Date.now() - 3600_000))
	if (filter.to >= boundary) {
	// response includes partial data for today, cache it for 10 minutes only
	res.setHeader('cache-control', 'public, max-age=600')
	} else {
	// historical data should never change, cache it for one year
	res.setHeader('cache-control', `public, max-age=${365 * 24 * 3600}, immutable`)
	}
	}

'