HLS Analyzer Monitoring Overview

HLSAnalyzer.com is an online HLS analyzer and monitoring service.  Streams are individually added to the monitoring system (Master or Media playlists), after which they can be configured for monitoring and alerting. Alerts can be delivered via email, or HTTP callback POSTs. HLSAnalyzer.com generates both stream delivery outages alerts as well as ad-insertion (SCTE-35) cue stoppage alerts. The monitoring engine downloads every segment and playlist, and logs the primary characteristics of the stream delivery process for subsequent evaluation and processing.  Summary reports for SCTE-35 cues as measured by the system, as well as an overview of all alert events can be downloaded for up to 90 days.

There are two primary ways of interacting with our system: a browser-based UI interface that allows access to most features of the analyzer, as well as an HTTP REST-based API that facilitates integration with automated pipelines. For examples on integrating our APIs with real-world applications, refer to our sample code available on Github.

Document Index

Browser Interface

Monitor All Streams

View API Status

Perform Stream Processing

Configure Alert Parameters

Batch Processing

Logs

Alert Functionality

Cloud Monitoring and Management

API Error Response Codes

Analyzer Error and Warning Codes

Browser Interface

The HLSAnalyzer.com browser-based UI can be used to add and remove streams, check the current API status, add files from a .csv file, and configure alert parameters. The interface utilizes the same API calls that are documented in the next section, ensuring a consistent experience while interacting with the HLS Analyzer monitoring server. The browser can also be used to monitor the state of all links associated with an API key in real-time.  

For the best experience, the use of Google Chrome, Firefox, or Microsoft Edge is recommended.

Monitor All Streams

/monitor-hls-playlists/monitor-api/api_key/

Description: All streams associated with the particular API key will be available for inspection, via an interface similar to the following:

Features include:

• Consolidated summary of all streams being monitored:

• Total number of non-live variants
• Master playlist status and media playlist status for each stream

• Playback URL for individual stream monitoring
• LinkID for the master and variant streams
• Total monitoring time for each stream
• The estimated playback and outage values:

• For playlists with multiple variants, these numbers are averaged across all variants
• For playlists with a single variant, these numbers are the actual values for that stream

• Alarm state for each stream, as well as enabling or disabling stream outage and SCTE-35 outage alerts individually for each stream. SCTE-35 alerts are disabled by default.

View API Status

All of the important parameters of the API can be monitored, including the current alert configuration state.

Perform Stream Processing

HLS streams can be added and removed from this interface. LinkIDs may be supplied for each stream, to make their identification easier. When not supplied, LinkIDs will automatically be assigned by the server when adding streams. Stream outage alerts as well as SCTE-35 outage alerts (disabled by default) may be individually enabled or disabled for each stream.

Configure Alert Parameters

Global alert parameters can be configured via this panel. The outage and clear settings apply globally to all streams, with delivery stream outages being specified in units of seconds and SCTE-35 outages in units of minutes. Alerts may be globally enabled or disabled from this panel as well. One or both alert types (email and HTTP Post callback) can be filled or left blank.

Batch Processing

Batch processing functions perform functions that involve a large set of streams:

• Export the currently active stream list
• Reset all stream statistics without removing them, or remove all streams (requires enablement via check-box)
• Add streams based on a .csv file with each line containing the HLS link followed by an optional LinkID, an optional SCTE-35 alert enablement (0/1), and an optional stream-delivery alert enablement (0/1) as shown here:
  
  

Logs

There are two types of logs: summary logs and detailed logs. Summary logs can be selected for up to 7 days, while detailed logs up to 24 hours. The following logging functions can be performed by the system:

• Alert Event Summary: the summary of stream outage and SCTE-35 outage alerts for the specified duration
• SCTE-35 Hourly Cue Summary: the sum of SCTE-35 cue durations (between out to in) as measured by the analyzer for every hour in the specified time range
• SCTE-35 Daily Cue Summary: the sum of all SCTE-35 cues durations every 24 hours in the specified time range
• All Alert Events: export every individual alert event
• All SCTE-35 Cues: export all SCTE-35 cue measurements

Logs will be available for up to 30 days in the past.

Alert Functionality

The monitoring service supports two types of outage alerts: stream delivery outages, and SCTE-35 cue outages. SCTE-35 alerts are disabled by default and must be enabled via the browser-based UI or the REST-APIs. Global alerts must be enabled before either stream outage alerts or SCTE-35 cue outage alerts are monitored.

When playlists or segments are not delivered on time, then alerts may be received according to configuration parameters. Media playlists may be in any of the following states:

When either media playlist or segments arrive at a slower rate than the expected time duration, the playlist buffer will be depleted and eventually reach zero, which would indicate a rebuffering event at the player. There are 3 sensitivity modes for alerting, starting with most sensitive to the least:

• The most sensitive mode (3).  The total time spent while the player is rebuffering is considered as the amount of outage and is used to trigger alerts. In this mode, segments are still arriving at the player, but cannot be downloaded in time and a player would experience rebuffering. The outage duration is compared to the duration of the rebuffering state (e.g. the hypothetical playback buffer is zero), and that can trigger an alert. This mode will track both complete stream outages, as well as intermittent delivery issues, but may generate many alerts since player outage durations are a common fact of online streaming.

• The normal sensitivity mode (2). In this mode, segments are also arriving slower than in real time. However, the analyzer will monitor the buffering state and an alert will trigger when the analyzer is continuously in a rebuffering state and the sum of Outage Durations exceeds the API-specified threshold.
  
  

• The least sensitive mode (1). In this mode, the analyzer will trigger an alert only when no segments can be downloaded. This mode can only track the total outages of the player, and will trigger when a continuous outage period has exceeded the threshold configured via APIs. This mode is suitable for customers who are only interested to know when playback has fully stopped due to a complete stream outage.

• Master playlist alerts. Master playlists are downloaded once every 20 seconds. If the playlist cannot be downloaded for a specific period of time, an alert will be raised. The same API-specified threshold used for stream outages will be used for master playlists.

A stream must be in outage condition for a minimum amount of time before an alert is generated. After the alert condition has been resolved, the clear condition must continue for a minimum amount of time before the clear condition is recognized by the system. This is shown in the next diagram:

The sensitivity setting only applies to media playlist outages. Alerts for SCTE-35 outages and master playlists utilize a binary condition: whether the asset is present or not. In case of SCTE-35, alerts are raised when cues are not observed for the API-specified duration. We recommend customers evaluate the alert sensitivity settings before determining a level suitable for their system. Alerts may be programmed to report the inconsistent delivery times, as well as when those conditions have been cleared. Alert email messages will generally have the following format:

Subject: OUTAGE: HLSAnalyzer Outage Alert for 5fa1ab24559c

This is an alert update from HLSAnalyzer.com for:

 https://example.com/example.m3u8

Alerts:

[5fa1ab24559c] Outage Time (UTC): 1610541723.00, Alert Condition Duration: 125.00 (sec), Total Accumulated Stream Outage: 128.3 (sec), Status: Stalled , Variant:  https://example.com/example.m3u8

[5fa1ab24559c] This stream experienced inconsistent delivery for 125.00 seconds.

Subject: OK: HLSAnalyzer Clear Alert for 5fa1ab24559c

This is an alert update from HLSAnalyzer.com for:

 https://example.com/example.m3u8

Alerts:

[5fa1ab24559c] Outage Time (UTC): 1610542123.00, Alert Condition Duration: 305.00 (sec), Total Accumulated Stream Outage: 386.0 (sec), Status: Live , Variant: https://example.com/example.m3u8

[5fa1ab24559c] This stream's alert condition has been cleared after a consistent delivery period of 305.00 seconds.

Cloud Monitoring and Management

The HLSAnalyzer.com service is a fully managed and monitored solution that provides high service availability for our customers. The health of our system is monitored by:

• Checking specific CPU and bandwidth threshold alerts every 15 minutes
• Conducting hourly tests of the liveness of the monitoring server
• Testing the end-to-end alert delivery capabilities of every monitoring instance every 6 hours
• Evaluating the API key expiration date every 24 hours
• Backing up of all configuration and stream settings every 6 hours

Our list of internal monitoring tools is always expanding to ensure that our service remains up and available and continuously monitors our clients’ HLS streams.

API Error Response Codes

Analyzer Error and Warning Codes

Copyright © 2022 Task One, LLC. All Rights Reserved.