The Brief

A better way to revisit doctor consultations.

Abridge is an app that helps patients better understand and manage their health. Built for both iOS and Android, Abridge securely records conversations between users and their medical providers, provides audio recordings and transcripts, and delivers intelligent insights on key medical terms and takeaways.

The team at Abridge approached us to augment their internal tech team, accelerate their product roadmap, and leverage our deep experience with native development.

The Vision

How Abridge empowers patients with insights.

Complex medical jargon is hard to follow, and gets in the way of doctor-patient conversations. Patients are split between listening to the doctor and taking vital notes in parallel.

In a high-pressure clinic environment, it’s easy to miss the minor details - information that can be crucial to a patient’s well-being and overall treatment plan.

Abridge bridges this gap by recording interactions both at the doctor’s and over the phone.

With audio, Abridge accurately highlights key medical terms, including symptoms, conditions, medication and procedures so that users can revisit their conversations with the doctor, understand their data better and stay on the same page.

Challenges

The problem with audio processing

The Abridge project was a unique technical challenge primarily because recording and encoding data in both iOS and Android was a complex process with poorly documented relevant APIs.

Our team also anticipated potential issues with handling multiple edge cases while streaming the audio to the server.

The Abridge app was built on React Native, which lacked straightforward solutions for audio recording and encoding. This created a need to bridge the framework to SDKs to support the feature.

The app also had to run advanced audio processing while streaming a recording to the server, which was an added challenge since it involved seamlessly executing multiple functions like compression and integration with web sockets.

Research & Findings

A window to our analysis and approach

Our experts exhaustively researched the latest design patterns and best practices on React Native to ensure that the app was up-to-date with the latest technologies.

The initial research was focused on diving into old developer documentation and sample code, especially while writing the new native module for audio functions.

We also studied the libflac library (written in C) in detail to implement it in Java.

Solutions

How we engineered advanced audio processing capabilities of the application

App Release & Launch

As consultants, we handled the complete app launch for iOS including setting up the Apple Developer Account, internal testing using TestFlight, creating the App Store listing and submitting the final app version for review.

We were also involved in crafting parts of the app user interface (UI) using React Native, including an ongoing v2 redesign of the complete product.

Built Robust Recording Capabilities

To improve the user experience, we engineered the native recording libraries to effectively handle interruptions like phone calls while recording, both on iOS and Android.

Real-time Audio Streaming

In the initial version of the app, the audio was uploaded to the server at the end of the recording. Abridge decided to add value to their users by providing key insights and transcripts in real-time, by enabling dynamic streaming and processing.

Our team built native modules from scratch for both iOS and Android to leverage the advanced audio processing capabilities on each platform (AVAudioEngine on iOS, and FlacEncoder/MediaCodecs on Android).