The purpose of the statistics server is to collect statistical information from all servers in the system. The statistics server stores the data, in memory, saving the data at a specified interval of time.

The statistics system operates with two types of statistics types: timed information and timeless information. Timed information is a value recorded over time (ex. Response time, workload etc.). Timeless information, is a value but where time does not have any meaning (ex. Internal execution time, counting etc.).

Timed statistical elements are stored internally, in arrays where each array element (called timeslot), represents one or more statistical data elements. Each timeslot is updated when new data is “added”.

Time Based Statistics and How It Is Stored

Time based statistics use an expression called time-slot. A time-slot is the a statistical unit which can contain a series of measurements on which average, minimum and maximum values and be calculated. A measurement period can therefore contain many time-slots.

If, for example, in one hour there are 3600 time slots defined, then a time slot corresponds to one second. That means that all measurements within one second are stored in the same time-slot. 

There are 4 levels of detail within the time based statistics:

1. 900  time-slots per 30 minutes. This corresponds to a two second level of detail and maximum of 30 minutes history. This level of detail is used if statistics of less than 30 minutes in ”interval” in the  GUI) are being examined.
2. 720 time-slots per day. This corresponds to a 2 minute level of detail and a maximum of 1 days history. This level of detail is used if statistics of less that one day are being examined (6 hours, 12 hours, 24 hours; in ”interval” in the GUI).
3. 720 time-slots per month period. This corresponds to a 1 hour level of detail and a maximum of 1 month history. This level of detail is used if statistics of less than 1 month are being examined (30 days, 15 days; in ”interval” in the GUI).
4. 365 time-slots per 1 year period. This corresponds to a 1 day level of detail and a maximum of 1 year of history. This level of detail is used if statistics of less than 1 years are being examined (1 year, 6 months).

The above values are the default time slots and intervals – they can be changed by modifying the appropriate configuration properties for the statistics server. These properties are:

resolution.secondinterval, resolution.secondcount, resolution.minuteinterval, resolution.minutecount,

resolution.hoursinterval, resolution.hourscount,  resolution.daysinterval, resolution.dayscount

E.g. changing resolution.secondinterval  to 1 and resolution.secondcount  to 3600 will change the resolution  to 1 second measurements with a history of 1 hour, but at the cost of increasing the memory usage in the statistics server significantly.

A way of minimizing the memory usage could be to set the resolution.secondcount  to 0, and increasing resolution.minuteinterval  to 5 while setting resolution.minutecount  to 288. This will lower the resolution of the available statistics, but also the memory usage.

If you need the statistics to be kept longer that one year, simply change resolution.dayscount  to the number of days you want the statistics kept, e.g. 1095 for 3 years.

A statistical event, will be stored in each of the 4 levels of detail in the time-slot which corresponds to the level of detail. This means that if two events occur with a 5 second interval, the first event will be stored in one time slot and the second event will be stored in another time slot at level of detail 1 above, while the two events together will be stored in the same time slot in item 2, 3 and 4 level of detail above.

When an event is stored in a time slot an update is done of the actual time slot, so that a time slot and its summaries are updated and stored real time.

A time slot contains the following attributes:

• The sum of the values from the event. (sum)
• The sum of the square roots of the values from the event (sqrsum)
• The maximum value of the events (max)
• The minimum value of the events. (min)
• Number of events (count)
• The time when the first event is stored in the time slot (start)
• The time when the last event is stored in the time slot (stop)

From a time slot the following can be calculated:

• The average of the values: mean = sum/count
• The variance: variance = abs((sqrsum – count * mean * mean)/count-1)
• Deviation: deviation = sqrt(variance)
• Standard error: stderr = deviation/ sqrt(antal)
• Estimate of the lowest 95% fractile: lower95 = mean – 1.96 * stderr
• Estimate of the upper 95% fractile: upper95 = mean  + 1.96 * stderr

The calculation of fractiles assumes that measurements follow a normal distribution. If the measurements in certain cases do not follow a normal distribution, then the estimate will not reflect the lower and upper fractiles. In this case these calculations must be ignored.