Introducing CoTerm, Your Collaborative Terminal for Pair Programming and Debugging | Datadog

Introducing CoTerm, your collaborative terminal for pair programming and debugging

Author Jason Thomas
Senior Staff Engineer
Author Till Pieper
Director, Product Management

Published: November 30, 2023

For too long, engineers have had to piece together an unwieldy combination of tools to collaboratively debug and resolve incidents while pair programming in real time. These activities normally require developers to work individually through a terminal, but the patchwork solutions that allow teams to work together in terminals all have significant drawbacks.

For example, teams looking for ways to share a terminal might initially turn to traditional video-conferencing tools, but this solution makes text blurry, adds significant latency, and even makes it easy to leak credentials if you accidentally reveal them on screen. Alternatively, engineers can opt to work together through terminals built into collaborative IDEs or by combining tmux with SSH, but these solutions don’t allow teammates to share secondary windows, such as emulators or browser windows, or to communicate through audio/video chat—which in turn requires teams to juggle multiple open apps just to work together.

To solve these problems and promote real-time engineering teamwork through a single tool, we’re excited to announce CoTerm, a shared terminal that lets you run commands, debug, and write code collaboratively with near-zero latency. CoTerm is a new beta feature in CoScreen, the collaboration tool by Datadog that was built for engineers, by engineers.

In this blog post, we will cover:

A new way to collaborate in terminals

With CoTerm, you can pair program and even mob program in your terminal with up to 10 team members. Terminal input and output are transmitted as PTY/text with near-zero latency. Teammates can share control to edit collaboratively in real time—and even copy and paste. CoTerm also comes with a secret-filtering feature that helps you keep sensitive strings like your passwords and private RSA keys private.

Key benefits of CoTerm beta:

  • Speed: CoTerm transmits terminal content as PTY/text data and not as video, delivering 80 percent lower latency than traditional screen sharing.
  • Enhanced collaboration: Up to 10 users can view the shared terminal and collaborate within it—while simultaneously taking advantage of built-in audio and video chat and multi-user screen sharing.
  • Annotations: With CoTerm, users can draw on the shared terminal to highlight items for other participants.
  • Security: Through the use of data encryption, along with RegEx pattern matching and entropy-based secret scanning and filtering, CoTerm enables you to share your terminal in a more secure way than is possible with other solutions.

And because CoTerm is built into CoScreen, engineers also benefit from all of CoScreen’s functionality as they work together within shared terminals, gaining the ability to:

  • Share the entire display, or individual windows, with built-in audio and video chat
  • Simultaneously share windows from multiple displays at the same time
  • Allow multiple users to share screens with each other at the same time
  • Retain control of any apps without letting anyone steal your window and mouse focus
CoTerm is a shared terminal in Datadog CoScreen that lets you run commands, debug, and write code collaboratively with near-zero latency.

How the collaborative terminal works

CoTerm consists of several components built around a PTY module, CoScreen, and the underlying open source video conferencing platform Jitsi. The PTY module is a high-performing and secure custom library, written in Rust, that connects to a pseudo-terminal on the underlying system. This pseudo-terminal is responsible for generating the input and output streams of the terminal session.

The data channel of CoScreen is used to securely transmit the encrypted PTY data alongside remote control data, annotations, and other in-band metadata about the ongoing session. If there are only two participants, the data channel produces, preferably, a peer-to-peer (P2P), DTLS-encrypted WebRTC data channel connection between the CoScreen peers. If there are more than two participants, and as a fallback method for two participants, it relies on a single secure WebSocket connection to the Jitsi VideoBridge. This connection can be established via outgoing SSL on port 443, which makes it compatible with most corporate firewall scenarios. The fallback WebSocket connection, also known as the BridgeChannel, is a standard connection to the Jitsi VideoBridge and is an HTTPS/SSL encrypted channel.

Terminals side by side with and without secret filtering

To ensure the protection of sensitive information such as credentials and passwords, CoTerm has an experimental mechanism to hide suspected secrets before they are transmitted. Among other components, this feature relies on Datadog Sensitive Data Scanner to detect sensitive information in strings that would otherwise be shared with remote users. The scanner is configured to detect various types of sensitive information, such as passwords, private RSA keys, GitHub API keys, tokens, credit card numbers, and Social Security numbers. When these strings are detected during a terminal sharing session, the scanner will replace it with wildcards before transmission.

Note that secret filtering is an experimental feature that can’t be disabled in this first version of CoTerm. If you want to share your unredacted terminal, you can still do so via the traditional CoScreen-way: by clicking on the “Share” tab above the terminal window.

How to get started with CoTerm

It’s easy to get started with terminal sharing in CoTerm. Simply take the following steps:

CoScreen menu showing a Share Terminal button

After this last step, the shared terminal will appear for you and all other participants in the CoScreen session. If you enable remote control in CoScreen, other users can type and click into your terminal.

What’s next?

CoTerm is now available as a beta feature. Currently, you can collaborate through the terminal built into CoScreen, with your local user profile and many of your customizations loaded. But since we are still building out this feature, we are actively soliciting your input about what other capabilities you’d like to see. Tell us what you’re missing the most.

If you want to try it out, you can get started with CoTerm in CoScreen for free in under two minutes. And if you’re brand new to Datadog, you can sign up for a to try CoScreen with Datadog today.