X.Spectator 2.0
A few years have passed since the initial release of the X.Spectator library. During this time, several new versions of .NET have been released, and Microsoft has introduced some built-in tools for monitoring system health. This evolution has been instrumental in keeping the X.Spectator library relevant.
As you may recall, the concept of the X.Spectator library was born during my university days. It all started with a small project written in .NET 4.0, from which I occasionally borrowed small code snippets for real projects. Those were the days of learning and experimentation, and they have shaped the X.Spectator library into what it is today. After dedicating a few evenings and a couple of weekends, I finally polished the code and shared it on GitHub.
With the rapid advancements in .NET and Microsoft’s introduction of native tools, updating the library to ensure its relevance and compatibility with the latest technologies became imperative. Let’s enumerate some most important changes:
– Enum SystemState was replaced on the HealthStatus class from the .NET
– In the Probe class, the Data property has evolved from a string to a ReadOnlyDictionary<string, object>.
– The enum State was also replaced with HealthStatus.
These changes are important for a seamless transition and integration with the platform’s native tools. Leveraging the built-in tools from .NET ensures better system performance, reliability, and maintainability, aligning perfectly with Microsoft’s best practices and standards. This alignment provides a solid foundation for the future of the X.Spectator library.
Let’s take a closer look at what and how it is implemented in this project to determine the need to change the system’s operating mode using an approach involving the introduction of virtual Probes and Observers.
Entities and Definitions
The basic entities used are:
- Probe: Responsible for checking the state of one of the system’s indicators.
- Observer: Polls one or more probes. Changes its state depending on the current readings of the probes.
- State Evaluator: Calculates the current state based on the metrics log.
- State Journal: A set of indicators for each probe, including the polling time.
Each abstraction has a basic implementation, and mechanisms for simple and convenient extensions are provided. Let’s look at them in detail.
Probe
The primary interface is IProbe. Classes implementing IProbe provide the value of the system’s parameters or a specific module/service they are monitoring upon request.
[PublicAPI]
public interface IProbe
{
/// <summary>
/// Probe name
/// </summary>
string Name { get; }
/// <summary>
/// Execute probe
/// </summary>
/// <returns></returns>
Task<ProbeResult> Check();
}As a result of the Check method, an instance of IProbe returns a ProbeResult structure:
[PublicAPI]
public struct ProbeResult
{
/// <summary>
/// Probe name.
/// </summary>
public string ProbeName { get; set; }
/// <summary>
/// Probe execution time.
/// </summary>
public DateTime Time { get; set; }
/// <summary>
/// Probe result status.
/// </summary>
public HealthStatus Status { get; set; }
/// <summary>
/// Provides diagnostic data.
/// </summary>
public IReadOnlyDictionary<string, object> Data { get; set; }
/// <summary>
/// Provides exception information.
/// </summary>
public Exception? Exception { get; set; }
/// <summary>
/// Returns the fully qualified type name of this instance.
/// </summary>
/// <returns>
/// The fully qualified type name.
/// </returns>
public override string ToString() => $"{Time}: {Status}";
}The Success field indicates whether the parameter check was successful from the probe’s point of view, and the Data and Exception fields can store additional information for debugging or logging if necessary.
Observer
The basic interface is ISpectator<TState>. Observes the system, generates events when the system’s state changes to notify all subscribed modules. Uses an instance of a class implementing the `IStateEvaluator` interface to calculate the current state.
[PublicAPI]
public interface ISpectator<TState> where TState : struct, IConvertible
{
/// <summary>
/// Event that is triggered when the state of the spectator changes.
/// </summary>
event EventHandler<StateEventArgs<TState>> StateChanged;
/// <summary>
/// Event that is triggered when the health of the spectator is checked.
/// </summary>
event EventHandler<HealthCheckEventArgs> HealthChecked;
/// <summary>
/// State of the spectator.
/// </summary>
TState State { get; }
/// <summary>
/// Uptime of the spectator.
/// </summary>
TimeSpan Uptime { get; }
/// <summary>
/// Name of the spectator.
/// </summary>
string Name { get; }
/// <summary>
/// Adds a probe to the spectator.
/// </summary>
/// <param name="probe"></param>
void AddProbe(IProbe probe);
/// <summary>
/// Checks the health of the spectator.
/// </summary>
void CheckHealth();
}During each probe poll, the observer triggers the StateChanged event and passes a StateEventArgs object containing information about the current system state to the event subscribers.
State Evaluator
The basic interface is IStateEvaluator<TState>. Calculates the current system state based on the state journal of the probe.
[PublicAPI]
public interface IStateEvaluator<TState>
{
/// <summary>
/// Evaluates the state of the spectator.
/// </summary>
/// <param name="currentState">
/// Current state of the spectator.
/// </param>
/// <param name="stateChangedLastTime">
/// The time when the state of the spectator was changed last time.
/// </param>
/// <param name="journal">
/// Journal of the spectator.
/// </param>
/// <returns></returns>
TState Evaluate(TState currentState, DateTime stateChangedLastTime, IReadOnlyCollection<JournalRecord> journal);
}State Journal
A collection of JournalRecord structure instances. An instance of JournalRecord stores information about all polled probes at the time the observer initiated the poll.
/// <summary>
/// Represents a journal record.
/// </summary>
[PublicAPI]
public class JournalRecord
{
/// <summary>
/// Default constructor.
/// </summary>
/// <param name="time"></param>
/// <param name="values"></param>
public JournalRecord(DateTime time, IEnumerable<ProbeResult> values)
{
Time = time;
Values = values.ToImmutableList();
}
/// <summary>
/// Time of the journal record.
/// </summary>
public DateTime Time { get; set; }
/// <summary>
/// Values of the journal record.
/// </summary>
public IReadOnlyCollection<ProbeResult> Values { get; set; }
/// <summary>
/// Returns a string representation of the journal record.
/// </summary>
/// <returns></returns>
public override string ToString() => $"{Time}: [{string.Join(",", Values)}]";
}How it Works
The system state calculation process can be described as follows: each probe integrated into the system can determine one of the observed system/module/service parameters. For example, it can record the number of active external API requests, the amount of occupied RAM, the number of records in the cache, etc.
Each probe, a versatile component, can be assigned to one or more observers, providing flexibility in system monitoring. Each observer, with its adaptability, can work with one or more probes. An observer must implement the ISpectator interface and generate events in case of state changes or probe polls.
During the subsequent system state check, the observer polls all its probes, forming an array for recording in the state journal. If the system state has changed, the observer generates the corresponding event. Event subscribers can change the system’s operating parameters based on this information. Different types of evaluators can be used to determine the system’s state.
Synchronous and Asynchronous Modes of Operation
Let’s consider two main scenarios in which an observer initiates sensor polling.
Synchronous Mode
Probes polling, followed by system state recalculation, is triggered by a direct call from one of the system’s modules to the observer. In this case, the observer and sensors operate in the same thread. System state calculation is performed during the execution of some operations inside the system. The project already includes a basic implementation of such an observer — SpectatorBase.
Asynchronous Mode
In this case, sensor polling occurs asynchronously from the system processes and can be performed in a separate thread. The project also includes a basic implementation of such an observer — AutomatedSpectator.
Conclusion
Key updates include aligning with native .NET health monitoring tools, enhancing data structures for better diagnostics, and streamlining state management.
This article delves into X.Spectator’s core principles and mechanisms, showcasing its effectiveness in enhancing system stability, particularly in high-load and distributed environments. We explore how X.Spectator employs probes, observers, state evaluators, and journals to monitor and assess system health.
