We wanted to share how the Co-browsing feature works within OpenReplay. Co-browsing allows its users to monitor their customers' real-time interactions with their websites or products and instantly engage with those who need assistance. OpenReplay, for its part, is a self-hosted session replay tool that helps developers troubleshoot web apps faster.

Here’s how we did it:

1. When a user starts a session, we open a WebSocket connection to initiate a new "room" on our backend. This room is capable of supporting one user session and multiple support agents. We use socket.io on the front end and uWS on the back end to ensure scalability and performance in case of tens of thousands of rooms with multiple WebSockets connections.

2. The user's session is then listed in the "Cobrowse" view but remains inactive in terms of data transmission through WebSocket. It awaits a support agent to initiate a live replay connection.

3. As soon as the support agent joins, we send a signal to our Tracker, which includes the OpenReplay Assist plugin. This plugin then hooks into our message processing method, copying all sent messages to the WebSocket channel and triggering our Tracker to restart to scan and send the entire DOM tree.

4. When both parties are connected, we create WebRTC peer channels for both, enabling the potential use of video calls.

Side Note: For sessions with canvas elements present in the DOM, we create a dedicated peer channel and send canvas data as a variable frame rate video stream, adjusting dynamically to changes in the DOM.

What sets our co-browsing feature from other tools’ co-browsing feature, like FullStory:

1. Video calling: to initiate video calls with users for direct support or onboarding. We send the support agent name as a call signal (call popup on the user's end) and wait for the user to respond (accept/decline)

2. Call recording: live sessions can be recorded in video format for review or training.

3. Group call: multiple support agents can join a single active user session and initiate a group call with the end user.

4. Remote control: to guide users directly to their target.

5. Annotations: with user consent for remote control, a support agent can draw directly on the end user's screen. (similar to the screen drawing feature in Slack huddle)

6. Time travel in session: to go back in time, observe past user actions, check his console logs, and then return to real-time or live mode. (like scrubbing in a YouTube video)

7. Live tabbed browsing: to observe users as they navigate through multiple browser tabs. 8. Muti-live sessions: to monitor up to 4 live sessions at the same time.

9. Watch console: to view active user console activities

For 1, 2, 3, 4, and 5: These features are activated via WebSocket upon request from the support agent and require end-user explicit confirmation, and can be stopped anytime.

- Remote control allows for scrolling, clicking, page visits, and inputs within the web app, adhering to what web API allows us to trigger from the JS side, as other features are restricted for user security.

- Annotations are made on a transparent canvas overlay, invisible until the agent begins drawing