A Multipart Mishap:
Adding plain text email support to Stop Slacking

July 23, 2018

Recently, I discovered that Stop Slacking, a free reply-by-email service I operate, wasn't processing plain text emails. This is a story about troubleshooting with a product design footnote.

Step 0: Never ignore your error messages

A user let me know that nothing happened when they tried to subscribe. Their subscription email looked fine so I reviewed my error message logs for another clue.

For weeks, I had been receiving error messages that said:

1java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String

A function called getBodyPart couldn't use the input it was receiving. getBodyPart is used in only one part of Stop Slacking – to parse incoming emails (subscription requests, replies to notifications, emails to me).

I had ignored the error messages. I wasn't familiar with the function getBodyPart because I hadn't written this bit of code. I told myself that if it was really a problem, I'd hear about it or see the problem myself (hah).

Why are only some emails causing an error? What's different about these emails?

Step 1: Recreate the error

In a Clojure REPL, I put an unsuccessful email through my email parsing function and received the error I expected:

1java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String

I have an error in an environment I can play with. Check.

Next, I can step through the function to see what my computer sees. (Fingers crossed the computers remember my empathy when they take over.)

Step 2: Diagnosing the problem

Here's the Clojure function my computer is trying to carry out when the error occurs:

1(defn parse-email
2  [email]
3  (-> (MimeMessage.
4      (Session/getDefaultInstance (java.util.Properties.))
5      (io/input-stream (.getBytes (:content email) "UTF-8")))
6      (.getContent)
7      (.getBodyPart 0)
8      (.getContent)))

You can see the .getBodyPart function in the middle.

Why does .getBodyPart throw an error when it gets a string? What are the successful emails giving .getBodyPart?

To see what is happening at each step, I've removed the last two instructions from this function and I'll add them back one at a time.

1(defn parse-email
2  [email]
3  (-> (MimeMessage.
4      (Session/getDefaultInstance (java.util.Properties.))
5      (io/input-stream (.getBytes (:content email) "UTF-8")))
6      (.getContent)))

Compare the outputs:

Input Output
Unsuccessful email "Hi, this is the body of the email. Such email text."
Successful email #object[javax.mail.internet.MimeMultipart 0x6af1733c "javax.mail.internet.MimeMultipart@6af1733c"]

When I input the unsuccessful email into parse-email, I get the message body. When I input the successful email, I get an object identifier back.

What happens when I follow the next instruction in my program, .getBodyPart?

1(defn parse-email
2  [email]
3  (-> (MimeMessage.
4      (Session/getDefaultInstance (java.util.Properties.))
5      (io/input-stream (.getBytes (:content msg) "UTF-8")))
6      (.getContent)
7      (.getBodyPart 0)))

The outputs:

Input Output
Unsuccessful email java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String
Successful email #object[javax.mail.internet.MimeBodyPart 0x39370e77 "javax.mail.internet.MimeBodyPart@39370e77"]

The unsuccessful email gets me the error that started this quest. The successful email gets another object. It's a different identifier this time, interesting.

If I had stopped manipulating the unsuccessful email in the last step, I would've had what I wanted.

I want to see how my successful email message makes it through this program. What happens when I add the second .getContent from my original function?

1(defn parse-email
2  [email]
3  (-> (MimeMessage.
4      (Session/getDefaultInstance (java.util.Properties.))
5      (io/input-stream (.getBytes (:content msg) "UTF-8")))
6      (.getContent)
7      (.getBodyPart 0)
8      (.getContent)))

The outputs:

Input Output
Unsuccessful email (Not applicable)
Successful email "Hi, this is the body of the email. Such email text."


In summary, the full set of instructions works for some emails but is too much for other emails.

What's the difference between these two types of emails?

Take a look at the emails themselves. (Note: These have been simplified for readability.)

The successful email:

 1Return-Path: <sender@example.com>
 2Subject: test email
 3To: recipient@example.com
 4Content-Type: multipart/alternative; boundary=\"123\"
 5--123
 6Content-Type: text/plain; charset=\"UTF-8\"
 7Hi, this is the body of the email. Such email text.
 8--123
 9Content-Type: text/html; charset=\"UTF-8\"
10Content-Transfer-Encoding: quoted-printable
11<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body><div>Hi, this is the body of the email. Such email text.</div><br><div></div></body></html>
12--123--

The unsuccessful email:

1Return-Path: <sender@example.com>
2Subject: test email
3To: recipient@example.com
4Content-Type: text/plain; charset=\"UTF-8\"
5Hi, this is the body of the email. Such email text.

You can see that the successful email starts with a Content-Type: multipart and contains two children, text/plain and text/html. The unsuccessful email has only Content-Type: text/plain.

This is the crux of the problem. My program expected only multipart emails, not plain text, or what I like to call "singlepart", emails.

Diagnosis complete.

Step 3: Resolve the problem.

Now that I knew the problem, I knew that I needed to detect whether an email was single or multipart and then parse accordingly.

Because I don't know which order the body parts will arrive in, my new function lists the body parts and their types so that I can select the longest of each type. For example, if there are two plain text parts in the email, I grab the longer one and parse it. If needed, I can select html and fall back on the plain text.

I've a newfound admiration for the folks that build email clients and email services because they must anticipate a forest full of sneaky edge cases just waiting to pounce.

Playing with email clients

I've discovered that Fastmail sends plain text emails by default and sends multipart emails in response to multipart emails.

You can write and send plain text emails from Gmail if you select "Plain text mode" from the little dropdown in the bottom right corner, to the right of the trash icon. This is handy for testing.

You can also examine the raw email in Gmail by selecting the "Show original" option from the top right dropdown, to the right of the reply arrow on an individual email message.

Now you can look at the email messages you've received and see if they're multipart messages or single part plain text messages.

It's cool that email supports a variety of options because we're all different people with different needs.

Product Design Footnote

Is it worthwhile to support plain text emails? Are you going to support every little edge case?

Plain text emails aren't just an edge case for Stop Slacking.

Folks who want to use lower bandwidth tools like Stop Slacking are highly likely to use plain text email because it is the lowest bandwidth type of email.

The primary purpose of Stop Slacking is to offer a lower bandwidth method for using Slack. I want to support my users in their efforts to use other low bandwidth tools.

This is probably the easiest product design decision I've ever made. :)

Drop me a line if you have any questions, have encountered odd types of emails in your work, or know of a set of raw emails that could be used for testing email parsers.