July 23, 2018
Recently, I discovered that Stop Slacking, a free reply-by-email service I operate, wasn't processing plain text emails. This is a story about troubleshooting with a product design footnote.
A user let me know that nothing happened when they tried to subscribe. Their subscription email looked fine so I reviewed my error message logs for another clue.
For weeks, I had been receiving error messages that said:
1java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String
A function called getBodyPart
couldn't use the input it was receiving. getBodyPart
is used in only one part of Stop Slacking – to parse incoming emails (subscription requests, replies to notifications, emails to me).
I had ignored the error messages. I wasn't familiar with the function getBodyPart
because I hadn't written this bit of code. I told myself that if it was really a problem, I'd hear about it or see the problem myself (hah).
Why are only some emails causing an error? What's different about these emails?
In a Clojure REPL, I put an unsuccessful email through my email parsing function and received the error I expected:
1java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String
I have an error in an environment I can play with. Check.
Next, I can step through the function to see what my computer sees. (Fingers crossed the computers remember my empathy when they take over.)
Here's the Clojure function my computer is trying to carry out when the error occurs:
You can see the .getBodyPart
function in the middle.
Why does .getBodyPart
throw an error when it gets a string? What are the successful emails giving .getBodyPart
?
To see what is happening at each step, I've removed the last two instructions from this function and I'll add them back one at a time.
Compare the outputs:
Input | Output |
---|---|
Unsuccessful email | "Hi, this is the body of the email. Such email text." |
Successful email | #object[javax.mail.internet.MimeMultipart 0x6af1733c "javax.mail.internet.MimeMultipart@6af1733c"] |
When I input the unsuccessful email into parse-email
, I get the message body.
When I input the successful email, I get an object identifier back.
What happens when I follow the next instruction in my program, .getBodyPart
?
The outputs:
Input | Output |
---|---|
Unsuccessful email | java.lang.IllegalArgumentException: No matching method found: getBodyPart for class java.lang.String |
Successful email | #object[javax.mail.internet.MimeBodyPart 0x39370e77 "javax.mail.internet.MimeBodyPart@39370e77"] |
The unsuccessful email gets me the error that started this quest. The successful email gets another object. It's a different identifier this time, interesting.
If I had stopped manipulating the unsuccessful email in the last step, I would've had what I wanted.
I want to see how my successful email message makes it through this program. What happens when I add the second .getContent
from my original function?
The outputs:
Input | Output |
---|---|
Unsuccessful email | (Not applicable) |
Successful email | "Hi, this is the body of the email. Such email text." |
In summary, the full set of instructions works for some emails but is too much for other emails.
What's the difference between these two types of emails?
Take a look at the emails themselves. (Note: These have been simplified for readability.)
The successful email:
1Return-Path: <sender@example.com> 2Subject: test email 3To: recipient@example.com 4Content-Type: multipart/alternative; boundary=\"123\" 5--123 6Content-Type: text/plain; charset=\"UTF-8\" 7Hi, this is the body of the email. Such email text. 8--123 9Content-Type: text/html; charset=\"UTF-8\" 10Content-Transfer-Encoding: quoted-printable 11<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body><div>Hi, this is the body of the email. Such email text.</div><br><div></div></body></html> 12--123--
The unsuccessful email:
You can see that the successful email starts with a Content-Type: multipart
and contains two children, text/plain
and text/html
. The unsuccessful email has only Content-Type: text/plain
.
This is the crux of the problem. My program expected only multipart emails, not plain text, or what I like to call "singlepart", emails.
Diagnosis complete.
Now that I knew the problem, I knew that I needed to detect whether an email was single or multipart and then parse accordingly.
Because I don't know which order the body parts will arrive in, my new function lists the body parts and their types so that I can select the longest of each type. For example, if there are two plain text parts in the email, I grab the longer one and parse it. If needed, I can select html and fall back on the plain text.
I've a newfound admiration for the folks that build email clients and email services because they must anticipate a forest full of sneaky edge cases just waiting to pounce.
I've discovered that Fastmail sends plain text emails by default and sends multipart emails in response to multipart emails.
You can write and send plain text emails from Gmail if you select "Plain text mode" from the little dropdown in the bottom right corner, to the right of the trash icon. This is handy for testing.
You can also examine the raw email in Gmail by selecting the "Show original" option from the top right dropdown, to the right of the reply arrow on an individual email message.
Now you can look at the email messages you've received and see if they're multipart messages or single part plain text messages.
It's cool that email supports a variety of options because we're all different people with different needs.
Is it worthwhile to support plain text emails? Are you going to support every little edge case?
Plain text emails aren't just an edge case for Stop Slacking.
Folks who want to use lower bandwidth tools like Stop Slacking are highly likely to use plain text email because it is the lowest bandwidth type of email.
The primary purpose of Stop Slacking is to offer a lower bandwidth method for using Slack. I want to support my users in their efforts to use other low bandwidth tools.
This is probably the easiest product design decision I've ever made. :)
Drop me a line if you have any questions, have encountered odd types of emails in your work, or know of a set of raw emails that could be used for testing email parsers.