Did I choose the right WebRTC stack?

While searching for information on HTTP/3 and WebTransport, I came across the project SIPSorcery WebRTC Echoes which is a set of interoperability tests for the WebRTC stacks.

From that website, I discovered that several other WebRTC stacks exist. I was already aware of the Pion WebRTC stack as well as its equivalent in Rust, but I discovered other libraries.

For me libWebRTC was the de facto standard implementation for WebRTC compliant clients. In my head, having a WebRTC solution that works on several platforms (desktop, mobile, …) was synonymous of sharing the same multi-platforms WebRTC library. Not by mixing different ones. This is where libWebRTC is strong by offering that since the beginning.

But years after years, it was still a painful task for native WebRTC developers to compile and build libWebRTC as well as having a clear changelog of differences for all platforms: How to be sure your application compiled with version X of libWebRTC is still compliant with browser version Y? The only right answer was to test…

Additionally, new use cases have emerged such as capturing a video from IOT devices, analyze it on server side and then relay it to the person in charge… For that kind of use cases, libWebRTC is less adapted compared to these emerging libraries: Too big and too complex for embedded devices…

Regarding on the openness, the roadmap was always dictated by Google mainly for Google products: libWebRTC is an open-source library strongly managed by Google and not by a community for the benefit of all.

All these points (and those I missed) contributed to accompany the will of some to create alternative WebRTC stacks: Changing that status-quo and proposing stacks for different languages more adapted to developers’ needs but by keeping the compatibility.

This article is a guided tour of the different existing WebRTC client stacks targeting native development.

Existing WebRTC stacks

Imagine that you would like to do a new native project that requires real-time media. Which stacks (library) should you use ?

On my side, I had a look on the following WebRTC stacks which are ordered here by their popularity (stars) and contributors (number) on GitHub (except for libWebRTC). My goal was to discover them, understand the good and the bad parts of each and see in which cases to use them.

Note: As they are listed in WebRTC Echoes, they should be interoperable meaning that for example a client using libWebRTC should be able to initiate a video call to a client using Pion.

Updated (Nov’27th 2022) : Adding Node-WebRTC to the picture even if it seems to be discontinued. I saw projects that still use it.

LibWebrtc (C++), the original one

Although Chrome 18 was released in Jan’12 with the WebRTC stack accessible behind a flag, the libWebRTC stack was first released as an open-source in May’11.

“Today, Google made available WebRTC, an open source software package for real-time voice and video on the web that we will be integrating in Chrome. This is our initial contribution to achieve the mission of making audio and video available in all browsers, through a uniform standard set of APIs…” From Harald Alvestrand, May, 31 2011

And finally, 13 months after this initial release of libWebRTC as a native library, Google released the first official release of Chrome with the support of WebRTC in Chrome 21 (July’12).

libWebRTC was integrated into Chrome…

Basically, libWebRTC targets not only the browser but any native desktop integrations (Linux, Windows, macOS) as well as native Android and IOS integrations by having an ObjC and Java interface.

And more than that, other browsers such as Firefox and Safari chosen to integrate WebRTC based on this library and some home-made adaptations.

Note: The first company who demonstrated an equivalent of WebRCT (based on a Web API called ConnectionPeer) was Ericsson with its OpenWebRTC project in January’11 with a modified Webkit GTK+ based browser + libnice + GStreamer + some existing Stream/MediaStreamTransceiver APIs.

Building the WebRTC stack on an Apple M1 (Experiments)

Don’t forget, I’m a Web developer… But I’m tried to do a native application using libWebRTC…

For that and as I have a M1 Mac, I decided to build a native macOS application with WebRTC using Swift.

In order to build the WebRTC stack native library, I first needed to clone the Chromium depot_tools repository. This is a collection of tools for dealing with Chromium Development. This is a bit strange, but you need the Chromium toolchain to build WebRTC. It seems that you can’t build WebRTC from the WebRTC repository directly (not sure about that but didn’t know someone who did that).

Once cloned, I used the fetch tool to download the WebRTC source code, then I was able to use ninja to compile and build the library.

This manual page summaries the actions to do to install depot_tools.

Compiling libWebRTC for macOS (or iOS) needs a Mac (as usual). I followed several tutorials for compiling for Mac but never succeeded to integrate the binary to my application. Something was missing for sure but as I’m not Mac/IOS developer, it was difficult for me to overcome that problem. It is like the framework generated was not well recognized once imported to Xcode.

So I decided to use a non-official library from Github/stasel proposed as a Swift Package! What a pleasure to have finally this kind of integration. In less than a minute I got the WebRTC library integrated to my Swift application.

For those who want to go deeper, which means making changes in the libWebRTC source code and so compiling a specific version, you can find the script to build your own library. This script does more tasks than mine (this could explain my problem…).

Using libWebRTC

If you come from the Web development, you will be surprised as there is no equivalence of the W3C getUserMedia API. In native development, you need to rely on existing native methods to authorize, list and to access devices.

For a native macOS/iOS application, mainly, they are 2 steps :

• Authorization: This is done in the project’s settings (info.plist and entitlements)

• Devices capture: That part can be done by using the RTCCameraVideoCapturer and other Apple native APIs.

At that time of writing, I have succeeded to authorize and access the camera but my native application crashes without interesting information to help me. So I didn’t succeed to go further on the media part.

In the same time, I tried to use the WebRTC native methods to do the signaling part. That part is easier because it is similar to the Web. Here is just the beginning in Swift that create the offer.

// STUN Url
  let stunURL = "...";

  // Constraints
  let constraints: RTCMediaConstraints = RTCMediaConstraints(mandatoryConstraints: ["OfferToReceiveAudio" : "true", "OfferToReceiveVideo" : "true"], optionalConstraints: [:]);

  // ICE configuration
  let configuration: RTCConfiguration = RTCConfiguration();
  configuration.iceServers = [RTCIceServer(urlStrings: [stunURL])];

  // Peer connection
  pc = factory.peerConnection(with: configuration, constraints: constraints, delegate: self);

  pc?.offer(for: constraints, completionHandler: { (description, error) in

      guard let description = description else {
          return
      }

      self.pc!.setLocalDescription(description, completionHandler: { (error) in
      })
  })

Note: I will try to find more time to progress on that sample and dedicate an article once succeeded.

Note: I found this GitHub project from Crom-Misia that maintains all scripts for building WebRTC on all platforms.

Conclusion

Using libWebRTC to develop a native WebRTC application is still not something easy. I felt lost because I didn’t find many articles on internet on how doing a macOS integration and the macOS community seems to be few. More than that, with my M1 machine, I’m not sure if I need some adaptations… Documentation provided by Google on the native part is not really useful for that (or I missed the interesting parts). Swift samples are absent. Mobile developers are fortunate on this point.

In the opposite, using the WebRTC API seems relatively easy as it is something common to all platforms.

So, I would conclude by saying that for a native macOS/iOS developer, it should not be a problem and the integration should work nicely (I hope…) including the compatibility with the Web. But mastering the build toolchain seems inevitable.

Pion (Go)

The Pion WebRTC stack has been written in Go and so is “mainly” or exclusively dedicated to Go developers. As I never wrote one line in Go, thinks are not looking great…

For the little story, Pion is really a good example of a stack that has emerged due to the complexity behind libWebRTC. Sean DuBois the initiator of Pion has first developed a Turn server before working on the Pion WebRTC stack mainly because of 2 reasons: he rapidly understood that libWebRTC has limitations on some platforms and that investing time on something he can’t control is a lack of time.

So, he decided to create Pion.

Pion has a modular architecture which means that each piece of code can be taken separately. Each one has its own GitHub repository. For the compilation part, it takes only a few seconds depending on your machine. It seems to be really a lightweight stack compared to libWebRTC.

Regarding on the APIs, as for libWebRTC, they are the same as on the Web which means that there is no new API or behavior to learn.

Pion has been built without taking the camera as something mandatory. By default, Pion manipulates streams without the need of a camera. These streams could come from remote sources or from files. To access the camera, an additional and dedicated packages need to be loaded. This is the opposite as in the Web where the first use cases were for sharing the camera’s video.

By doing that, Pion proposes a library that can be used on more kind of devices as well as on client and on server side: Pion is more than a client WebRTC stack: Several projects exist such as Ion-sfu for having a complete WebRTC integration using Go (Client + SFU + Stun/Turn).

Note: I add in mind to develop a simple binary that checks the connectivity through a TURN server and that can be launched on server side. At that time of writing, I’m using an alternative with node-datachannel a Node.JS port of LibDataChannel (see further in the article).

Note: In the Big ideas or next steps, goal is to make Pion work with TinyGo. This will decrease the footprint and allow to work great on IoT machines such as on a Raspberry and Arduino.

aiortc (Python)

aiortc is a WebRTC stack written in Python. So mainly dedicated for those who have already Python skills.

In the same manner as Pion, aiortc has been designed to be simpler to libWebRTC and the library can be used on both client and server sides. Again, the library has been thought with other use cases in mind than libWebRTC.

Jeremy Lainé, the creator of aioRTC had a different goal at the beginning: He would like to create a scriptable WebRTC test tool that can run on server side. As he had a strong background on Python, he decided to use that language.

Then once achieved, he started to implement aioICE and finally the whole WebRTC stack. Which was a huge task for a single individual! He explained that this was possible thanks to Async/IO which was the enabler for implementing concurrency inside the law layer of aiortc.

Regarding the public API, aioRTC maps the JavaScript interface os as for Pion, it is easy to read and understand samples written with aiortc.

The main focus of aiortc is to enable use cases around media processing. Thanks to the huge existing Python AI/ML ecosystem, audio and video streams can be sent and processed on server side thanks to aiortc and then relayed to a browser for being displayed.

Additionally, the usage of Python makes easier to use on the Internet of Things (IoT), for example to get the video from a Raspberry and to stream it somewhere.

Note: So, Pion and aiortc are very similar in terms of use cases they provide. It depends mainly on the language you used. Perhaps, aiortc seems more adapted to IoT than Pion at that time and Pion more adapted to server side if multiple relays are needed (SFU).

Note: Except Jérémy Lainé and despite the 3k stars on GitHub, there is no real other contributor to the project… Hope this will change to help him…

webRTC.rs (Rust)

If I have well understood, webRTC.rs is the Rust port of Pion.

But, as this project is still in active development, I will not spend too much time on it. In fact, I didn’t find so much information on it. Often, Rust is more associated with LLVM and the possibility to execute complex tasks in the browser that need speed or demand high CPU usage such as media processing.

Rust is not as easy to learn as Python or Go so, I think that as Rust and C/C++ are easily interoperable and as Rust is more and more used to replace/update C/C++ codebase, having a RUST stack will be interesting for those who want to stick with that language.

SipSorcery (C#)

With respect, SipSorcery is the oldest library here. It has been developed by Aaron Glausson. He first started by a service to register and manage SIP devices and providers. Then in 2009, the first version of SipSorcery (previously mysipswitch ?) has been released in order for developers to have their own SIP server locally installed.

And then from that, SIPSorcery evolved to a full C# SIP and WebRTC stack that landed in 2016 with the addition of STUN, SDP, RTP and RTCP.

Currently, SIPSorcery allows to build C# and .Net WebRTC applications that work on Windows, Linux and Mac on both client and server side: SIPSorcery is a pure C# library without the use of any wrappers making it easy to implement into existing .NET applications.

What is strange is that for Windows, Microsoft proposed WinRTC which is based on libWebRTC. But I’m not sure Microsoft is still investing on that track…

SipSorcery seems to be the solution today for building WebRTC application with Visual Studio, C# and .Net.

LibDataChannel (C++)

LibDataChannel is a standalone implementation of WebRTC Data Channels, WebRTC Media Transport, and WebSockets in C++17. In fact, this is not completely a WebRTC stack as there is no media encoding/decoding. This has to be done manually and so outside LibDataChannel: LibDataChannel offers only the Media transport.

But, the library offers a lightweight library with minimal dependencies that is fully compatible with the browsers.

This library has been developed by Paul Louis Ageneau and I use the Node.JS binding called node-datachannel to make some connectivity tests on top of my Node.JS application.

The Node.JS library is really easy to use even if there are differences with the JS WebRTC API: But as it is a lightweight library, it took me only a couple of hours for implementing the full scenario with node-datachannel. Result was as expected.

Paul Louis Ageneau is also the creator of Violet, a lightweight STUN and TURN server I mentioned in my article on Coturn the fragile colossus and libJuice.

Depending on the use cases you have, such as opening a datachannel between a browser and your server, LibDataChannel or node-datachannel could be a candidate for handling the WebRTC native connection end-point.

Werift (NodeJs)

Werift is WebRTC Implementation for TypeScript (Node.js) that includes ICE/DTLS/SCTP/RTP/SRTP. It has been developed by Yoshiaki Shin

This library is still in an early development stage so some parts are not yet available such as ICE restart, TURN ICE TLS and TCP connections, getStats, simulcast (send), etc…

It can be an alternative to node-datachannel.

React-Native & Flutter

For developing a mobile WebRTC application, there is no alternative today than using libWebRTC:

• For a Native application based on Java/Kotlin or ObjC/Swift, the best is to have your own toolchain for compiling and producing your libWebRTC library for iOS and Android (no more official binaries provided by Google).

• For a Native Multi-platforms application, React-Native and Flutter propose a WebRTC library based on libWebRTC.

So whatever you choose, you will have libWebRTC that comes with a full compatibility and level-grade with the browsers.

The thing to keep in mind is the revision of libWebRTC used by your React-Native or Flutter version. Be careful to update regularly your library to stay closed to Chrome and so benefit from the latest improvements and compatibility.

I had the opportunity to test React-Native, it offers the same API as in the browser and so using it was not more complicated as in the browser… Additionally, a lot of samples are available with a tutorial that explains the different steps to initiate a call.

But as usual, pre-requisites are a signaling server and at least a STUN/TURN server.

I didn’t have the opportunity to develop an application with Flutter/Dart, but I hope that for Flutter developers, it should not be more complex as for React-Native developers.

A dedicated sample repository exist Flutter WebRTC demo that can be a good starting point.

RawRTC (C/C++)

RawRTC is an additional WebRTC stack mainly used for testing purpose. It is not in maintenance, but I’m not sure the stack will continue to evolve.

RawRTC has been developed by Lennart Grahl who worked also on Salty which is an end-to-end encrypted signalling protocol.

The library has been created in 2016, originally for testing data channel implementation, that’s why RawRTC focus support only data channel.

It can be used to develop communication channels between browsers and server side component.

Conclusion

For sure, in that article, I didn’t speak about interoperability between stacks (sdp, codecs, etc…) but as there are all listed in the WebRTC-Echoes tests suite, it should work :-)

But building a WebRTC solution with different stacks is still risky and requires more testing each time a component update its WebRTC part to be sure there is no regression and interoperability still works.

That said, new WebRTC stacks have emerged, but as seen, not really to compete with libWebRTC but rather to cover other use cases, other platforms or just to ease the stickiness with development languages.

This is true for desktop and server applications. But for mobile, libWebRTC remains the only stack available for native applications.

But whatever the stack used, adding WebRTC will always require the same skills:

• Some Shell scripting and CI/CD skills to build the library for your platforms with an autonomous process that does the whole chain from the repository (required for libWebRTC).

• Some Native OS skills to manage the device’s inputs and outputs and the media according to the platform specificity

• A lot of WebRTC skills to manage correctly the stack and do the WebRTC part