I have a nominal 20 second interval for most of my raw data and I've found that many/most of the eagle.io charts and data exports that I try to create are unsuccessful, i.e., the spinner spins indefinitely and the browser never shows the result. I assume this is happening because the volume of data and/or the number of aggregation calculations are overloading eagle.io.

For example, when I try to chart the monthly averages of 10 temperature parameters in a chart to show temperature gradients in a building over the past year, the system can't produce the chart. It seems that every operation that eagle.io does is on-demand. The flexibility that this offers is fantastic and greatly appreciated! However, one consequence of this complete flexibility is that seems to be very easy to overwhelm the system. Presumably, the raw data are indexed, but it is apparent that the system does not perform any automatic aggregations. I assume that the example chart with the 10 monthly average temperatures would be lightning fast for the system to generate if automatic aggregated monthly values were available.

Even more significant is that if the example chart was created as a custom chart and was viewed 100 times on a dashboard, all the on-demand aggregations would have to be performed 100 times instead of 1 time had hierarchical aggregations been performed.

I made a little Hierarchical Aggregation Calculator in Excel that shows how many aggregation values would be necessary, given a particular raw data interval. Considering the substantial benefits of hierarchical aggregations in terms of performance gains, the cost in terms of the additional # of values is shockingly low. Obviously, an additional burden would be placed on the system upon upload and update operations as the calculations are performed. However, the required system resources to perform hierarchical aggregations during write operations should pale in comparison to the system resources saved by the hierarchical aggregations during read operations.

EDIT: When I read this article by MongoDB, I realized I was using a potentially misleading name in my original post when I called this "pre-aggregation."