HTTP Live Streaming直播(iOS直播)技术分析与实现
http://en.wikipedia.org/wiki/HTTP_Live_Streaming
HTTP Live Streaming
| Filename extension | .m3u8 |
|---|---|
| Internet media type | application/vnd.apple.mpegurl[1] |
| Developed by | Apple Inc. |
| Initial release | April 2014 |
| Type of format | Playlist |
| Extended from | extended M3U |
HTTP Live Streaming (also known as HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. as part of their QuickTime, Safari, OS X, and iOS software. It works by breaking the overall stream into a sequence of small HTTP-based file downloads, each download loading one short chunk of an overall potentially unbounded transport stream. As the stream is played, the client may select from a number of different alternate streams containing the same material encoded at a variety of data rates, allowing the streaming session to adapt to the available data rate. At the start of the streaming session, it downloads an extended M3U playlist containing the metadata for the various sub-streams which are available.[2]
Since its requests use only standard HTTP transactions, HTTP Live Streaming is capable of traversing any firewall or proxy server that lets through standard HTTP traffic, unlike UDP-based protocols such as RTP. This also allows content to be delivered over widely available CDNs.
HLS also specifies a standard encryption mechanism[3] using AES and a method of secure key distribution using HTTPS with either a device specific realm login or HTTP cookie which together provide a simple DRM system. Later versions of the protocol also provide for trick mode fast-forward and rewind and integration of subtitles. upLynk has also added the AES scrambling and base-64 encoding of the DRM content key with a 128-bit device specific key for registered commercial SWF applications together with a sequential initialization Vector for each chunk to their implementation of the standard.[4]
Apple has documented HTTP Live Streaming as an Internet Draft (Individual Submission), the first stage in the process of submitting it to the IETF as an Informational Request for Comments. However, while Apple has submitted occasional minor updates to the draft, no additional steps appear to have been taken towards IETF standardization.[5]
Contents
Architecture
Http Live Streaming uses a conventional web server to distribute audiovisual content and requires specific software to fit into the proper format transmission in real time. The service architecture comprises:
- Server
- Codify and encapsulates the input video flow in a proper format for the delivery. Then it\’s prepared for distribution by segmenting it into different files. In the process of intake, the video is coded and segmented to generate video fragments and index file.
- Encoder: codify video files in H.264 format and audio in MP3, HE-AAC or AC-3. This is encapsulated by MPEG-2 Transport Stream to carry it
- Segmenter: divides the MPEG-2 TS file into fragments of equal length, kept as .ts files. It also creates an index file that contains references of the fragmented files, saved as .m3u8
- Distributor
- Formed by standard web Server, accepts requests from clients and delivery all the resources needed for streaming.
- Client
- Demand and download all the files and resources, assembling them so that they can be presented to the user as a continuous flow video. The client software downloads first the index file through a URL and then the several media files available. The playback software assembles the sequence to allow the continue display to the user.
Features
HTTP Live Streaming provides mechanisms to provide a scalable and adaptable to network, allowing playback quality in wireless networks with high bandwidth and low quality playback on 3G networks, where the bandwidth is reduced. HTTP Live Streaming also provides protection against errors, generating alternative different flows video to use them if there are any errors in segment.
Scalability
To make the system scalable and adaptable to the bandwidth of the network, the video flow is coded in different qualities. Thus, depending on the bandwidth and transfer network speed, the video will play at different qualities.
To implement this, the system must encode the video in different qualities and generate an index file that contains the locations of the different quality levels.
The client software internally manages the different qualities, making requests to the highest possible quality within the bandwidth of the network. Thus always play the video the highest possible quality, viewing lower quality on 3G networks and highest quality in Wi-Fi broadband.
Protection against errors
Protection against errors works like scalability. In this case is generated a different flows with the same quality video and locations are listed in the index file.
The management of all files is done from the client, so that if it fails the first flow, use the next and successively.
HTTP Live Streaming also allows you to combine the protection of errors and scalability. In this case generates an index file that contains for each bandwidth the alternative flows of the videos.
Server implementations
- Adobe Media Server supports HLS for iOS devices (HLS) and Protected HTTP Live Streaming (PHLS).
- Akamai supports HLS for live and on-demand streams.
- Amazon CloudFront supports HLS for on-demand streams.
- Cisco Systems: Supports full end to end delivery for Live/TSTV/VOD/HLS and Cloud DVR services.
- EdgeCast Networks supports cross-device streaming using HLS.
- Helix Universal Server from RealNetworks supports iPhone OS 3.0 and later for live and on-demand HTTP Live or On-Demand streaming of H.264 and AAC content to iPhone, iPad and iPod.
- IIS Media Services from Microsoft supports live and on-demand Smooth Streaming and HTTP Live Streaming.
- Level 3 supports HLS live and on-demand streams.
- Limelight Networks supports HLS for some accounts.[6]
- Nginx with the RTMP Module supports HLS in live mode.
- TVersity supports HLS in conjunction with on-the-fly transcoding for playback of any video content on iOS devices.
- Unreal Media Server supports low latency HLS as of version 9.5.
- VLC Media Player supports HLS for serving live and on-demand streams as of version 2.0
- Wowza Streaming Engine from Wowza Media Systems supports HLS and encrypted HLS for live (with DVR) and on-demand streaming.
Usage
- Adobe Systems demonstrated an update to its Adobe Flash Media Server product supporting HTTP Live Streaming at the NAB Show in April 2011
- Apple Inc. used this on September 1, 2010 to stream their iPod Keynote event live over the internet, and on October 20, 2010 to stream their \’Back to the Mac\’ Keynote event live over the internet.
- Google added HTTP Live Streaming support in Android Honeycomb and later.[7]
- Helix Universal Server from RealNetworks supports iPhone OS 3.0 and later for live and on-demand HTTP Live or On-Demand streaming of H.264 and AAC content to iPhone, iPad and iPod initial release April 2010, latest release November 2012
- flashls is an open-source Adobe Flash/Adobe AIR plugin that provides HTTP Live Streaming support for Chromeless Flash/Air Player, Flowplayer, Video.js, mediaelement.js and Open Source Media Framework 2.0
- HP added HTTP Live Streaming support in webOS 3.0.5.[8]
- Microsoft added support for HTTP Live Streaming in IIS Media Services 4.0.[9]
- Yospace added HTTP Live Streaming support in Yospace HLS Player and SDK for flash version 1.0
- Sling Media added HTTP Live Streaming support to its Slingboxes and its SlingPlayer apps[10]
Supported players and servers
-
This list is incomplete; you can help by expanding it.
Clients
| Product | Platform | Live Streaming | DRM | As of Version | Vendor |
|---|---|---|---|---|---|
| Android (native media support) | Android | Yes | Yes | 3.0+[11] Support for HLS streams in the stock browser or Chrome is generally stable from Android 4.1 onwards (with minor issues). |
|
| iOS (native media support) | iOS | Yes | Yes | 3.0+ Has full HLS support. HLS streams can be accessed via Safari without the need of any 3rd-party app. |
Apple |
| QuickTime Player | OS X | Yes | Yes | 10.0+ | Apple |
| iTunes (music player) | Windows, OS X | Yes | Yes | 10.1+[12] Has full HLS support. To play a HLS stream, go to File > Open Stream and replace “http://” with “itls://” in the stream URL. |
Apple |
| VLC media player | Windows, OS X, Linux, Android, iOS, Windows Phone | Yes | No | 2.0[13] Has partial HLS support. Some HLS streams may not playback properly, having a slight pause/stutter or missing audio between each file segment.[14] |
VideoLAN |
| Kodi (software) (home entertainment application) | Windows, OS X, Linux, Android, iOS | Yes | Partial | 12.0 Alpha 5 and later DRM support requires a monthly/nightly build |
XBMC Foundation |
| MythTV (home entertainment application) | Windows, OS X, Linux, FreeBSD | Yes | Yes | 0.26 | MythTV |
| 3ivx (HLS Player and SDK) | Windows 8, Windows Phone 8[15] & Xbox One[16] | Yes | Yes | 2.0 | 3ivx |
| XiiaLive | Android, iOS | Yes | Yes | 3.0+ Plays internet radio streams (audio only). |
Visual Blasters LLC |
| Tunein radio | Android, iOS | Yes | Yes | 3.3+ Plays internet radio streams (audio only). |
TuneIn |
| DicePlayer | Android 2.2+ | Yes | Yes | Diceplayer 1.0+ | INISOFT |
| Amino x4x STB | Amino set-top boxes | Yes | Yes | 2.5.2 Aminet | Aminocom.com |
| HP Touchpad | WebOS | Yes | Yes | 3.0.5 | HP |
| nangu.TV | Motorola STB | Yes | Yes | 2.0 | nangu.TV |
| Roku Digital Video Player | Roku set-top boxes | Yes | Yes | Roku OS / SDK 2.6 | Roku |
| THEOplayer (HLS Player and SDK) | HTML5 | Yes | Partial | OpenTelly | |
| Flowplayer (HLS Player and SDK) | Adobe Flash, iOS, Android | Yes | Yes | The Flash HLS plugin is available from GitHub. | Flowplayer Ltd |
| JW Player (HLS Player and SDK) | Adobe Flash, iOS, Android | Yes | Yes | A Premium or Enterprise license is required for HLS support. | JW Player |
| Radiant Media Player (HLS Player) | Adobe Flash, HTML5 | Yes | Yes | 1.5.0[17] | Radiant Media Player |
| Yospace (HLS Player and SDK) | Adobe Flash | Yes | Yes | 2.1 | Yospace |
| Onlinelib (HLS Player and SDK) | Adobe Flash | Yes | Yes | 2.0 | Onlinelib.de |
| VODOBOX (HLS Player and SDK) | Adobe Flash, iOS, Android | Yes | Yes | VODOBOX | |
| NexPlayer SDK (OTT multiscreen player) | iOS, Android, Windows Phone | Yes | Yes | NexStreaming | |
| ffplay/avplay | Yes | Partial | FFmpeg/Libav | ||
| GPAC (cross-platform media player) | Yes | No | 0.5.0 | Telecom ParisTech inc. |
Servers
| Product | Technology | As Of Version | Vendor | Notes |
|---|---|---|---|---|
| VLC | 1.2 | |||
| Unicorn Once | SaaS | Unicorn Media | ||
| IIS Media Services | 4.0[9] | Microsoft | ||
| Antik Media Streamer | Ingest Module (UDP/HTTP Transport Stream, Backup Stream with auto-switching, stream status monitoring and logging), Stream replication UDP/HTTP, HLS streaming, Video archive with snapshots, Server-side Timeshift, Timezone Shifting with multi-timezones, Stream Encryption using AES and key-rotation (with Antik Key Server) |
3.0 | Antik technology | |
| Adobe Media Server | Live and VOD streaming as origin and edge server | 5.0 | Adobe | |
| Evostream Media Server | Cross-platform including embedded systems such as encoders, IP cameras, DVRs, and more. Supports: Adobe Flash RTMP, RTMPS, LiveFLV, full transcoder for creating lower bitrate streams, HTTP Live Streaming (HLS) for streaming to iPhones, iPads and Androids, HTTP Dynamic Streaming (HDS) for Adobe Air, Microsoft Smooth Streaming (MSS) for Microsoft devices, RTSP with RTP or MPEG-TS, MPEG-TS (unicast/multicast), compatible Live Encoding, strong security for your content ( Verimatrix DRM, HLS AES encryption, Stream Aliasing, Watermarking), built-in clustering mechanism and more. |
1.6.5 | EvoStream | |
| MythTV | 0.25 | MythTV | ||
| MACNETIX VOD-Server | 3.0 | MACNETIX | ||
| Anevia ViaMotion Servers | Transcapsulation: from one input, several outputs (HLS, MS Smooth Streaming, ADS Flash, MPEG DASH) |
Anevia | ||
| Packet Ship OverView:Origin Server | Capture from IPTV multicast and chunking to HLS for multi-bandwidth live streams, with AES encryption | 2.1 | Packet Ship | |
| nangu.TV Streamers | on-the-fly adaptation: content is stored once enabling several outputs (HLS, MS Smooth Streaming, ADS Flash, MPEG DASH) |
nangu.TV | ||
| TVersity Media Server | 1.9 | TVersity | Pro Edition only | |
| Helix Universal Server | Live + VOD HLS with Verimatrix DRM integration, ABR, Multi-Resolution, AES encryption | 15.0+ | RealNetworks | High performance HLS (12,000+ concurrent devices) |
| Wowza Streaming Engine | Live and VOD streaming as origin and edge server with DVR, DRM Integration and Transcoding for adaptive delivery. Outputs to MPEG-DASH, HLS, HDS, Smooth Streaming, RTMP, and RTSP | 2.0+ | Wowza Media Systems | |
| Unified Streaming Platform | Muxes media content from one unified source to multiple outputs (Smooth Streaming, HDS, HLS and MPEG DASH) | Unified Streaming | ||
| Flixwagon Platform Video Server | Flixwagon | |||
| StreamCoder Live Encoder | Realtime video encoder (inputs : DVB/IP stream or video signal). Supports multi-bitrates and multi-languages | Ektacom | ||
| Apache HTTP Server | Apache Software Foundation | |||
| Unreal Media Server | 9.5 | Unreal Streaming Technologies | Latency of live streams can be as low as 2.5 seconds over the Internet | |
| lighttpd | 1.4.x | |||
| Nimble Streamer | RTMP / UDP MPEG-TS to ABR HLS. MP4 / MP3 to VOD HLS | 1.0.0-x | WMSPanel | |
| Nginx-rtmp-module | Free module for nginx server with support of HLS live streaming. Compliant with iOS and Android. | 0.9.x | Roman Arutyunyan | |
| Nginx Plus | VOD HLS as origin | NGINX, Inc. | ||
| Flussonic Media Server | Multi-platform support for HTTP, RTSP, RTMP, Time Shifting, DVR Functions with Unlimited Rewind Capabilities HLS streaming specific to iOS platform support. |
3.0+ | Flussonic, LLC. | Supporting a magnitude of features with full HTTP support. |
| VBrick Distributed Media Engine (“DME”) | 2.0 | VBrick Systems, Inc. | Live and stored HLS. Live can be transmuxed from several input mux including RTP, RTMP, and MPEG-TS using H.264 encoding |
See also
References
- Pantos, R.P. (April 2014). “HTTP Live Streaming draft-pantos-http-live-streaming-13”. Network Working Group: 1.
- Jordan, Larry (10 June 2013). “The Basics of HTTP Live Streaming”. Larry\’s Blog. Larry Jordan & Associates. Retrieved 18 June 2013.
- Pantos, R. (30 September 2011). “HTTP Live Streaming”. Internet Engineering Task Force. Retrieved 18 June 2013.
- “Application keys”. Digital rights management (DRM). upLynk Support. Retrieved 18 June 2013.
- van Kesteren, Anne (2010-09-01). “Internet Drafts are not Open Standards”. annevankesteren.nl. Self-published. Retrieved 2015-03-22.
- “Limelight Orchestrate Video Support”. Limelight Networks. Retrieved 14 November 2013.
- “Android 3.0 Platform Highlights”.
- “webOS 3.0.5 Updates”.
- “First Look: Microsoft IIS Media Services 4”. StreamingMedia.com. Retrieved 30 July 2011.
- Blogcritics, November 26, 2010. iPad App Review: SlingPlayer (Retrieved April 14, 2014)
- Supported Media Formats | Android Developers
- Hints of \’iTunes Live Stream\’ Service Found in iTunes 10.1 – Mac Rumors
- https://trac.videolan.org/vlc/ticket/2943 VLC Ticket #2943 (Support for HTTP Live Streaming as a client)
- https://forum.videolan.org/viewtopic.php?t=113275
- http://www.3ivx.com/technology/windows/metro/http_live_streaming.html
- http://www.3ivx.com/xbox/
http://www.cnblogs.com/haibindev/archive/2013/01/30/2880764.html
HTTP Live Streaming直播(iOS直播)技术分析与实现
不经意间发现，大半年没写博客了，自觉汗颜。实则2012后半年，家中的事一样接着一样发生，实在是没有时间。快过年了，总算忙里偷闲，把最近的一些技术成果，总结成了文章，与大家分享。
前些日子，也是项目需要，花了一些时间研究了HTTP Live Streaming（HLS）技术，并实现了一个HLS编码器HLSLiveEncoder，当然，C++写的。其功能是采集摄像头与麦克风，实时进行H.264视频编码和AAC音频编码，并按照HLS的协议规范，生成分段的标准TS文件以及m3u8索引文件。通过我的HLSLiveEncoder和第三方Http服务器（例如：Nginx），成功实现了HTTP Live Streaming直播，并在iphone上测试通过。我就把这当中的一些收获写在这里。
HLS技术要点分析
HTTP Live Streaming（HLS）是苹果公司(Apple Inc.)实现的基于HTTP的流媒体传输协议，可实现流媒体的直播和点播，主要应用在iOS系统，为iOS设备（如iPhone、iPad）提供音视频直播和点播方案。HLS点播，基本上就是常见的分段HTTP点播，不同在于，它的分段非常小。要实现HLS点播，重点在于对媒体文件分段，目前有不少开源工具可以使用，这里我就不再讨论，只谈HLS直播技术。
相对于常见的流媒体直播协议，例如RTMP协议、RTSP协议、MMS协议等，HLS直播最大的不同在于，直播客户端获取到的，并不是一个完整的数据流。HLS协议在服务器端将直播数据流存储为连续的、很短时长的媒体文件（MPEG-TS格 式），而客户端则不断的下载并播放这些小文件，因为服务器端总是会将最新的直播数据生成新的小文件，这样客户端只要不停的按顺序播放从服务器获取到的文 件，就实现了直播。由此可见，基本上可以认为，HLS是以点播的技术方式来实现直播。由于数据通过HTTP协议传输，所以完全不用考虑防火墙或者代理的问 题，而且分段文件的时长很短，客户端可以很快的选择和切换码率，以适应不同带宽条件下的播放。不过HLS的这种技术特点，决定了它的延迟一般总是会高于普 通的流媒体直播协议。
根据以上的了解要实现HTTP Live Streaming直播，需要研究并实现以下技术关键点
- 采集视频源和音频源的数据
- 对原始数据进行H264编码和AAC编码
- 视频和音频数据封装为MPEG-TS包
- HLS分段生成策略及m3u8索引文件
- HTTP传输协议
其中第1点和第2点，我之前的文章中已经提到过了，而最后一点，我们可以借助现有的HTTP服务器，所以，实现第3点和第4点是关键所在。
程序框架与实现
通过以上分析，实现HLS LiveEncoder直播编码器，其逻辑和流程基本上很清楚了：分别开启音频与视频编码线程，通过DirectShow（或其他）技术来实现音视频采集，随后分别调用libx264和libfaac进行视频和音频编码。两个编码线程实时编码音视频数据后，根据自定义的分片策略，存储在某个MPEG-TS格式分段文件中，当完成一个分段文件的存储后，更新m3u8索引文件。如下图所示：
上图中HLSLiveEncoder当收到视频和音频数据后，需要首先判断，当前分片是否应该结束，并创建新分片，以延续TS分片的不断生成。需要注意的是，新的分片，应当从关键帧开始，防止播放器解码失败。核心代码如下所示：
TsMuxer的接口也是比较简单的。
HLS分段生成策略和m3u8
1. 分段策略
- HLS的分段策略，基本上推荐是10秒一个分片，当然，具体时间还要根据分好后的分片的实际时长做标注
- 通常来说，为了缓存等方面的原因，在索引文件中会保留最新的三个分片地址，以类似“滑动窗口”的形式，进行更新。
2. m3u8文件简介
m3u8，是HTTP Live Streaming直播的索引文件。m3u8基本上可以认为就是.m3u格式文件，区别在于，m3u8文件使用UTF-8字符编码。
#EXTM3U m3u文件头，必须放在第一行
#EXT-X-MEDIA-SEQUENCE 第一个TS分片的序列号
#EXT-X-TARGETDURATION 每个分片TS的最大的时长
#EXT-X-ALLOW-CACHE 是否允许cache
#EXT-X-ENDLIST m3u8文件结束符
#EXTINF extra info，分片TS的信息，如时长，带宽等
一个简单的m3u8索引文件
运行效果
在Nginx工作目录下启动HLSLiveEncoder，并用VLC播放器连接播放
通过iPhone播放的效果