Clara Labs is establishing a class of coworker that combines the best of human and machine talents. Like a machine, Clara always remembers preferences. Like a good assistant, Clara appreciates the nuance underlying each of your messages and helps build rich relationships as a team member. To achieve this automated-yet-socially-adept experience, person and machine work and learn in tandem to complete tasks in a positive feedback loop that we think of as cooperative intelligence.


Our platform enables Clara remote assistants (CRAs) to be super-human: they can handle complex requests from hundreds of customers with little a prioribackground on each customer or conversation. In contrast, a conventional remote assistant may handle at most three to five clients. Scale is further amplified with machine learning (ML) which is used to partially (and sometimes fully) automate the work.

A real-time human-in-the-loop system decreases the time-to-market of new ML-driven features.

Building a real-time human-in-the-loop system decreases the time-to-market of new ML-driven features and has significant impact on how prediction is integrated into the platform. We’d love to share the way we think about these problems.


Machine learning at Clara


First, a little background. Supervised ML algorithms require annotated training data. In the most basic setup, each data point takes the form (example, annotation); for instance, we may have the pair (“I’m copying my assistant to set something up.”, REQUEST_SCHEDULE). There are two key aspects to building a good data set:


      • Examples should be similar to what we expect to observe in our application. Schedule requests as in the example above would be challenging to find in say, a corpus of Wikipedia phrases or news articles.
      • Each example must be precisely annotated for the target application.

The more representative the training examples are and the better the annotations, the better prediction results will be. Obtaining such annotated data can be difficult and time consuming as tens of thousands of unique examples are often required. In practice, raw data examples are farmed out to contractors who perform tedious annotation tasks; fatigue often leads to numerous annotation errors and noisy data.


Our platform stands out in the following respects.


We design jointly for people and machines


Scheduling tasks are decomposed into subtasks that can be understood by both person and machine, tightly coupling software design and human process. The careful consideration of what exactly must be annotated often increases the tractability prediction models.


Our data quality is high because annotators “get it”


CRAs serve a dual role as annotators, often implicitly. As a result, the better our assistants perform at executing Clara’s tasks, the higher quality our training data will be. Further, data quality is top of mind across disciplines (such as UX) and teams at the company.


Small datasets become big ones, quickly


Human supervision means predictions need not be highly accurate at first, enabling fast bootstrapping of new learning-driven product features. We test new ideas quickly without spending significant resources building datasets and tuning algorithms. In other words, our machine learning efforts are agile.


Agile ML: A simple example


Not all of our ML features begin their life geared purely for automation. An example of this is our empathy prompts to assistants.



Text above the black horizontal line is an example message sent to Clara. The underlined yellow phrase is classified by our algorithm as referring to “illness” and from the first person. The suggested response below the black horizontal line is generated based on these parameters.


As shown in the image, we detect when an empathic response might be appropriate, generate a suggestion, and surface it to the assistant. CRAs are given the option to easily add the response on the outgoing message. When selected, this serves as an annotation for our dataset and drives a better customer experience. We can ship a feature like this well before it’s accurate enough to drive an automated response, significantly decreasing the time-cost of developing new ML-driven features and opening the door for experimentation and iteration.


Clara is growing


We’re excited to grow our platform and team. Find out more about open roles here:

Learn more about our CEO Maran Nelson here:

and read about our recent $7M series A announcement here:


We thank David A. Shamma and George B. Davis for their valuable advice and feedback on this article.