Today we want to share with you a great article by our teammate Serhii Romanov, SDET Manager on Pluto TV. Since he’s been specializing on testing video streaming services for several years and has experience working with various streaming protocols, he decided to share some insights about the most popular one—HTTP Live Streaming (HLS). 

Overview of HLS 

HLS is an HTTP-based adaptive bitrate streaming protocol, developed by Apple Inc. and introduced at WWDC 2009 alongside iOS 3.0. Apple regularly updates this protocol, so here you can see what was added in 2024. 

Although HLS was originally developed for Apple devices, it is now supported by many other platforms, from Android to set-top boxes and Smart TVs of various brands. As of 2024, the annual video industry survey consistently shows that it is the most popular streaming format. 

LS uses a simple architecture that does not require special hardware. The process begins with encoding video using H.264 or H.265 codecs. These codecs provide efficient compression, allowing the video size to be reduced without significant quality loss. The encoded video is then segmented into short segments, typically from 5 to 10 seconds, although this length can be changed for optimization according to the type of content or network conditions. 

Each segment is self-contained and can be downloaded and played separately, providing flexibility in case of connection loss or other issues. For example, if a segment is not fully downloaded, the client can try to retrieve only that segment, minimizing delays. 

Role of Master Playlist 

HLS uses two types of playlists: Master Playlist and Media Playlist. The Master Playlist serves as an index file that references multiple Media Playlists and provides information about different stream variants, such as video quality, bitrate, and codecs. This allows players to dynamically switch between streams depending on network conditions. The following image shows the relationship between Master Playlist and Media Playlist. 

Here is an example of a Master Playlist: 

#EXTM3U  
#EXT-X-VERSION:3  
#EXT-X-STREAM-INF:BANDWIDTH=1280000,RESOLUTION=640x360,CODECS="avc1.42e01e,mp4a.40.2"  
http://example.com/360p.m3u8  
#EXT-X-STREAM-INF:BANDWIDTH=2560000,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2"  
http://example.com/720p.m3u8  
#EXT-X-STREAM-INF:BANDWIDTH=5120000,RESOLUTION=1920x1080,CODECS="avc1.4d401f,mp4a.40.2"  
http://example.com/1080p.m3u8 

Let's break down the directives of the Master Playlist: 

#EXTM3U: Indicates that the Master Playlist is based on the extended M3U format. 

#EXT-X-VERSION:3: The version of the HLS protocol supported by this playlist. The higher the version, the more advanced features of HLS can be used. 

#EXT-X-STREAM-INF: Contains stream attributes: 

• BANDWIDTH: The peak transfer rate (bitrate) for the segments of a specific stream variant in bits per second. 

• RESOLUTION: The optimal resolution in pixels for displaying the entire video within the selected stream variant. 

• CODECS: The codecs used for video and audio. 

After this directive, the URL of the Media Playlist corresponding to the specific stream variant is specified. 

The Master Playlist allows for adaptive streaming, where the client automatically selects the best stream variant based on network conditions and device capabilities. 

Role of Media Playlist 

The Media Playlist describes a sequential list of media segments that compose a single stream variant. The main function of the Media Playlist is to provide the client with information about each video segment, including its duration, URL, and possible metadata. 

Here is an example of a Media Playlist: 

#EXTM3U 
#EXT-X-VERSION:3 
#EXT-X-MEDIA-SEQUENCE:0 
#EXT-X-TARGETDURATION:10 
#EXT-X-PLAYLIST-TYPE:VOD 
#EXTINF:10.000, 
http://example.com/segment1.ts 
#EXTINF:10.000, 
http://example.com/segment2.ts 
… 
#EXT-X-ENDLIST

The Media Playlist includes directives that are a set of special instructions. They provide information about the overall format of the playlist, the structure of the stream, the characteristics of the segments, and other parameters necessary for the correct processing and playback of video content. Let's break down some of them: 

#EXTM3U and #EXT-X-VERSION:3: We discussed these earlier. 

#EXT-X-MEDIA-SEQUENCE:0: Indicates the sequence number of the media file of the first media segment that appears in the Media Playlist. 

#EXT-X-TARGETDURATION:10: A decimal integer that indicates the maximum duration of a media segment in seconds. 

#EXT-X-PLAYLIST-TYPE:VOD: Indicates the playlist mode and regulates the possibility of making changes to it: 

• VOD (Video on Demand): The playlist must be static and cannot be changed after creation. 

• EVENT: The playlist can be changed, and new segments can be added to the end of the list. 

#EXTINF:10.000,: The duration of the media segment, followed by the URL of the segment. 

#EXT-X-ENDLIST: Indicates that no new segments will be added to the Media Playlist. It can appear anywhere in the playlist but is most often used in playlists for VOD (Video on Demand), where the content is static and does not change. 

There are many such directives, and all of them can be found in the official specification from Apple. 

Components of an HLS-based System 

A system based on the HLS protocol typically includes three key components: 

• Packager: Software that splits video into segments and compiles the Master Playlist and Media Playlist, detailing the order and characteristics of each media segment. 

• Server: Acts as a storage for segmented media files and corresponding playlists and provides them upon request. It uses standard web server technology to distribute HLS content. 

• Player or Client: A device or application that downloads and plays the video. It starts playback by obtaining playlists, then sequentially downloads and plays media segments. 

The image shows the interaction of all parts of the system. 

Streaming can be divided into several types, the main ones being live streaming and video on demand (VOD). 

Live Streaming 

This is a method of distributing multimedia content that is delivered to viewers in real-time. Live streaming is ideal for events, webinars, breaking news, or sports events. Examples of live streaming platforms include Twitch, Vimeo Livestream, and YouTube Live. 

In the case of live streaming, the client requests an m3u8 playlist containing metadata and URLs of media segments. The media player reads the m3u8 playlist and begins downloading and playing media segments in the order specified in the playlist. In live streaming, the .m3u8 file is updated dynamically as new media segments are generated and published. The media player periodically requires an updated .m3u8 file to ensure it has the latest segments. This process allows for real-time broadcasting of events. 

Video on Demand (VOD) 

This is a method of distributing media that allows users to choose content to watch at their convenience. In the case of VOD, the client receives a static m3u8 playlist in which all available media segments are already listed and can play the content in full at any time since all video material is available immediately after publication. VOD content is usually offered as a collection or organized as a library, so users can play selected content with one click. VOD content can be monetized using most of the same methods as live streaming. 

Testing HLS 

Testing HLS involves checking the entire streaming system, both the server and client parts. The server part includes encoding and segmenting video, creating media playlists, and ensuring their availability through HTTP servers or CDN for geographic content distribution. The client part involves playing streams on different devices and browsers, checking adaptive bitrate, compatibility, and broadcast delay. This ensures that HLS streams can be correctly received and displayed on a wide range of devices with different internet speeds. 

Server-side Testing 

Server-side testing begins with validating m3u8 playlists. Validating playlists in HLS is a crucial step in ensuring proper video playback. As mentioned earlier, playlists (.m3u8) in HLS are divided into Master Playlist and Media Playlist, each playing its role in delivering streaming content. Checking these files helps avoid playback errors and system crashes during streaming. 

To verify the correctness of the Master Playlist, you need to perform several steps, starting with analyzing its structure and content. Here are the main aspects to focus on: 

• Format and Syntax: The file should start with the #EXTM3U directive. 

• Correct Links to Media Playlists: Ensure all links to media playlists are valid and point to existing resources. 

• Stream Metadata: Verify that the #EXT-X-STREAM-INF directives contain the correct attributes, such as BANDWIDTH, RESOLUTION, and CODECS. 

• Availability of All Stream Variants: The master playlist should list all necessary variants for different resolution and bitrate options. 

To ensure the proper functioning of the Media Playlist, pay attention to its main elements that affect video playback: 

• Syntax Directives: The playlist should start with the #EXTM3U directive, followed by segment directives such as #EXTINF for segment duration and links to segments. 

• Correct Links to Segments: Ensure all segments (links to files with the .ts extension) are available and can be downloaded. 

• Segment Sequence: The #EXT-X-MEDIA-SEQUENCE and #EXT-X-ENDLIST directives should correctly manage the order of segments and the end of the playlist. 

• Correct Segment Duration: The #EXTINF directives should contain the correct time values so that the client correctly calculates the playback time intervals. 

Additionally, pay attention to: 

• Adaptive Bitrates: Ensure playlists include streams with different bitrates and permissions to support adaptive streaming. 

• DRM Support: If DRM (Digital Rights Management) is used, ensure playlists contain the necessary directives for encryption and content protection. 

• Live Streams: For live streams, it is important to check the correctness of media playlist updates to include new media segments in real-time. 

• Directives for Identifying Content Type, specifically advertisements. 

For server-side testing, writing automated tests is also beneficial. There are many projects or libraries that allow parsing m3u8 playlists and performing all necessary checks. 

For example, for writing tests in Java, I prefer the m3u8-parser project. It allows parsing m3u8 playlists into Java objects according to RFC 8216 HTTP Live Streaming. The author of this project states that "m3u8-parser does not try to validate playlists. You are responsible for creating valid playlists," so writing checks should be the responsibility of the testing team. 

Here is an example of how to parse a Master Playlist using m3u8-parser and perform checks on it: 

public class HlsStreamTest { 
 private static final String STREAM_PATH = "https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s-fmp4/"; 
 private static final String MASTER_PLAYLIST = "f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8"; 
 private final MasterPlaylistParser masterPlaylistParser = new MasterPlaylistParser(); 
 private final MediaPlaylistParser mediaPlaylistParser = new MediaPlaylistParser(); 
 @Test 
 public void testSimpleMasterPlaylistParsing() throws IOException { 
   // Fetch and parse the Master Playlist 
   String playlistContent = fetchPlaylistContent(STREAM_PATH + MASTER_PLAYLIST); 
   MasterPlaylist masterPlaylist = masterPlaylistParser.readPlaylist(playlistContent); 
   // Validate the MasterPlaylist object is not null 
   assertNotNull(masterPlaylist, "MasterPlaylist should not be null"); 
   // Validate version 
   assertTrue(masterPlaylist.version().isPresent(), "Version should be present and is critical to the playlist's metadata."); 
   assertEquals(6, masterPlaylist.version().get().intValue(), "Expected version is 6"); 
   // Additional checks that could be implemented: 
   // 1. Validate the number of variants in the playlist: 
   // 2. Validate each variant's attributes such as URI, bandwidth, resolution, and codecs: 
   // 3. Validate alternative renditions, such as audio tracks: 
   // 4. Check for custom tags or comments, if needed. 
   // Other validations 
 } 
}

And here is an example of how to parse a Media Playlist using m3u8-parser and perform the necessary checks: 

@Test 
public void testSimpleMediaPlaylistParsing() throws IOException { 
 // Fetch and parse the Master Playlist 
 String playlistContent = fetchPlaylistContent(STREAM_PATH + MASTER_PLAYLIST); 
 MasterPlaylist masterPlaylist = masterPlaylistParser.readPlaylist(playlistContent); 
 //Fetch and parse the Media Playlist 
 String mediaPlaylistContent = fetchPlaylistContent(STREAM_PATH + masterPlaylist.variants().get(0).uri()); 
 MediaPlaylist mediaPlaylist = mediaPlaylistParser.readPlaylist(mediaPlaylistContent); 
 assertEquals(mediaPlaylist.targetDuration(), 4, "Target duration should be 4"); 
 // 1. Validate that the media sequence starts at the expected value 
 // 2. Validate that the playlist type is VOD or EVENT 
 // 3. Validate independent segments flag 
 // 4. Validate that all media segments have a positive duration 
 // 5. Validate that all segment URIs are not null 
 // 6. Validate the total number of segments matches an expected count 
 // 7. Validate that there are no discontinuities unless expected 
 // 8. Validate the duration of the final segment 
 // 9. Validate that partial segments (if any) are processed correctly 
 // Other validations 
}

Similarly, tests can be written using Python. For Python, there are also many libraries for working with playlists, and one of the most popular examples is the m3u8 library. It allows downloading, parsing, and checking playlists, providing a simple interface for working with various attributes. 

Here is an example of what a test written in Python using the m3u8 parser might look like: 

class TestMasterPlaylist(unittest.TestCase): 
   MASTER_PLAYLIST_URL = "https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s-fmp4/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8" 
   @classmethod 
   def setUpClass(cls): 
       """ 
       Fetch and parse the playlist content once for all tests. 
       """ 
       response = requests.get(cls.MASTER_PLAYLIST_URL, timeout=5) 
       response.raise_for_status() 
       cls.playlist_content = response.text 
   def test_general_metadata(self): 
       """ 
       Validate general metadata of the playlist. 
       """ 
       playlist = m3u8.loads(self.playlist_content) 
       self.assertEqual(playlist.version, 6, "Playlist version should be 6.") 
       self.assertTrue(hasattr(playlist, 'media'), "Playlist should contain media attributes.") 
   def test_audio_tracks(self): 
       """ 
       Validate audio tracks in the playlist. 
       """ 
       playlist = m3u8.loads(self.playlist_content) 
       self.assertEqual(len(playlist.media), 1, "Playlist should have one audio track.") 
       audio = playlist.media[0] 
       self.assertEqual(audio.type, "AUDIO", "Media type should be AUDIO.") 
# Other validations

Server-side testing also includes checking the operation of the packager and the correctness of the media files themselves, especially when content transcoding is also part of the testers' responsibilities. 

 That’s it for today, but part 2 is coming. There, I’ll cover media files testing, using FFprobe and integration with ChatGPT for analysis, and playback and performance testing. Until next time!