Gmail's "Smart Compose" feature should be considered harmful

Wednesday, February 27, 2019

In 2005, I got my invite to get a Gmail account. It was incredible, and I loved it, although I didn't really know why at the time. It was a combination of really great design so it was pleasant to use, the hype built up by the invite system, the perpetual feeling of getting something more as you watched your allotted storage slowly tick up, and quite a bit from the fact that it was the first email account I signed up for on my own. I had an email account before that, created by my parents through our ISP, but this one was mine, created by me, from an invite my friend gave me, and all my friends were also using Gmail - if they could get an invite. With that ability to also chat through your webmail client... it was mindblowing, and it eventually supplanted AOL Instant Messenger for my friends and me.

For the last 13 years, Gmail has been my primary email service. (I recently switched to a paid provider who respects my privacy. I'm willing to pay for privacy and security.) It has been instrumental in allowing me to communicate and stay in touch with family, friends, and acquaintances as we spread out across the globe. I have 65,000 emails in my account, 10,000 sent emails, and 9,000 chats with friends.

Along the way, they added features that helped me communicate more effectively. When they added the Google Labs feature to stop an email from sending if you say "find attached" or something and forget to attach a document... that saved me embarrassment many times, and it saved my contacts from frustration. When they added video calls in 2008, it made it easy to actually see my parents while I was away at college. Filtering gives us the ability to sort incoming messages and reduce information overload. Labels replaced the folders of other mail providers as a more natural fit for how we want to organize our information. The superior search features of Gmail have made it so that I have a (slow, asynchronous) external memory where I can look up past events. And not least of all, Gmail's spam filtering was incredibly effective at a time when most mail providers struggled against the tide.

This is all to say: Gmail, I have loved you, but "Smart Compose" and "Smart Reply" are strong deviations from the true value of Gmail.

The true value of Gmail has been in enhancing and facilitating communication, especially sincere communication. "Smart Reply" hinders this value by reducing the overall variety of responses, and "Smart Compose" biases the words that you send.

With "Smart Reply", Gmail shows you three possible replies below your email, encouraging you to select one to reply "efficiently". The problem is that this has the same effect as naming a number in a negotiation: it anchors you around those possibilities and reduces your creativity in responding. If someone asks you whether Monday or Tuesday works for you, it will offer the responses "Let's do Monday," "Monday works for me," or "Either day works for me" (this is an actual example from the announcement blog post). The net effect of this, I believe, is that people will be less likely to schedule on Tuesday, because the convenient options were all presented saying that Monday is better or that they are equal. For setting up a coffee date, this might not be a big deal, but what about for price negotiations? The models for "Smart Reply" are surely trained on real emails, so what if the model learns that men will negotiate more aggressively than women? If that makes its way into the "Smart Reply" feature, it will have material harm for any woman negotiating pricing over email by slightly biasing them toward less aggressive negotiations, reinforcing the status quo.

Similarl harms are baked into "Smart Compose", a feature which suggests the next word or phrase for you to type based on what you've typed so far. They've already had to remove pronouns because the system was biased toward men, so it is difficult to believe that it is unbiased in all other ways. What other harms are in the system that Google engineers simply have not detected yet?

And that's all just the active immediate harm from a particular message. There is also the more subtle shift from automating some of our communcations. What if the black box learns when your contacts' birthdays are, and suggest sending "happy birthday" to them on those days and fills it in for you? How long will it take to erode the well-wishing tradition in our society, replacing it with a mechanical button-press? The point of saying "happy birthday" isn't to just say it - it is to actually think about them and take your time to call them or message them with your well wishes.

Automating our communication is done in the name of efficiency but it is robbing us of the one thing that makes humans human: our language and our communication. It is causing direct harms, whether through something as benign as a "happy birthday" or through something as sinister as biasing negotiations or oppressing whole groups. I hope these harms were considered by the product managers at Google, but these are public harms, so the users and the public deserve to be made aware of whatever tradeoffs have been made and whatever protections are in place. In the meantime, we should all consider these features harmful and a net negative for our society.

If this post was enjoyable or useful for you, please share it! If you have comments, questions, or feedback, you can email my personal email. To get new posts and support my work, subscribe to the newsletter. There is also an RSS feed.

Want to become a better programmer? Join the Recurse Center!
Want to hire great programmers? Hire via Recurse Center!