Negative results often yield interesting learnings and it’s unfortunate that they are infrequently written about. Here at Clara, one of our operating principles is to accelerate learning when buildingand we find it important to meditate on projects that didn’t always work out as planned. We share one such story with you here.
G Suite calendar sends out an invite email to attendees after an event is created. We’ve found that on occasion an attendee will respond to this email with a request or other details pertinent to scheduling. Clara may need this information to conduct further actions on behalf of the customer for the meeting.
Unfortunately, these invite emails are sent without a unique identifier that could be used to find the events that generated them.* Without the ability to associate these emails with the appropriate meeting model in our system, Clara may miss this potentially important information.
Associating emails with the right meeting model in our system can be challenging in general; consider the complexity of understanding a message that contains multiple meeting requests to Clara and multiple side conversations between customer and attendees. While this is a great part of the system to harness human intelligence, a purely automated solution is viable for the particular subproblem of event invitations.
It’s possible to scrape and parse the invite email text body to obtain key information (such as the event title, attendees, and time) and then search for the appropriate matching event in our database. We avoid these kinds of solutions since they have some likelihood of causing errors and introduce dependencies on a third party invite format, which is unpublished and subject to change at any time.
We opted to prepend our own unique meeting identifier to each event description. We had the question: can we hide the identifier so that the experience is seamless to the user? This is how we arrived at the simple watermarking technique we call invisible ink.
Unicode has four zero-width characters, namely
- zero-width space (u+200B)
- zero-width non-breaking space (u+FEFF)
- zero-width joiner (u+200D)
- zero-width non-joiner (u+200C)
The zero-width joiner is particularly neat: when used between two emojis, it represents an additional emoji character, effectively extending the emoji encoding length. Our idea was to overload these characters by using them to encode a 128-bit UUID4 string. Doing this is straightforward: we represent each hex element of the UUID as a pair of zero-width characters (i.e., 16 pairs).
With these characters on the event description, a UTF-8 compliant client should render them as zero-width and they will not be visible. Indeed, our initial investigation suggested that many of the email clients our customers use would support reading UTF-8 correctly, as well as a few we hand-tested. We also validated that these characters were transferred intact on the invite descriptions sent by Google.
Our event descriptions were tagged with invisible ink in production and the result seemed pretty cool: we were auto-associating these invite responses transparently, or so we thought.
It was only a short period of time before we soon discovered that the default iOS font presents these characters as question-mark-boxes. This is because the glyphs for these zero-width characters are not included with this font and so the default character for a missing glyph is shown. This was an awkward result, but the data was indeed valid and used by our system to automatically associate email responses to invites.
Since the events Clara generated “looked broken” we ultimately pulled invisible ink in favor of the human readable Reference ID: <ID> (Do not delete) which you see on Clara event descriptions today.
This project highlights that testing on all clients matters. This is particularly acute in Clara’s case as our product leverages email, which finds its way into many operating systems and clients. Despite the fact most of these support UTF-8 text, we still missed a large segment that was incompatible with the approach (from a customer perspective). Perhaps we were trying to be just a little bit too clever.
>> from invisible_ink import encode_watermark, decode_watermark
>>> encoded_text, uuid = encode_watermark(u’asdf’)
>>> print encoded_text
We’d love to hear if you’ve found a use for it!
*Office365 indeed provides the concept of EventMessage which could be used to solve this problem. To the best of our knowledge, a similar concept does not exist for G Suite.
Clara is growing
We’re excited to grow our platform and team. Find out more about open roles here: https://jobs.lever.co/claralabs
Learn more about our CEO Maran Nelson here:
and read about our recent $7M series A announcement here: