This story contains no information of interest to anyone what-so-ever.
But it annoyed me, so let me vent.
I was already over my time-estimate, and I had a bug. It was strange, the system was sending out two emails and one had an exclamation mark; just randomly right in the footer – But the other was fine. After staring at my code and deciding that there was nothing wrong I decided to view-source in the browser – Yup, email had an exclamation mark right in the middle.
But I wasn’t putting it in there, and the code to attach the footer to each email was exactly the same. However only one email was displaying an exclamation mark.
See this project was in .php on a linux server – I’m not a php guru, nor a linux expert – previous to last month I had written approximately 50 lines of php code, and I’ve run linux, well never.
So I was convinced it was my bug, something I’d done wrong, something about the php_mail() function that I didn’t know about – It didn’t help that the server hadn’t been patched since 2004 and was still running php 4.2 so I couldn’t use any of the email libraries that kept being recommended in my google searches and promised to make my life easier.
So I entered a new test order into the system, and I got my two emails – One with an exclamation mark and one without, but this time the exclamation mark had moved. My previous testing at always shown it in the same place.
Test and measure, it’s the scientific method.
So I try entering a new order, this time adding exactly 4 characters to the address on the order form. This time the exclamation mark has moved exactly 4 places to the left.
Ah ha, I’ve got something! – I thought to myself. The system was no longer random, no longer chaotic, I had something deterministic – I had no idea why but the length of the email was causing an exclamation mark to be inserted
I modified my code output the email to the browser before sending it to php_mail() – No exclamation mark, formating all correct, everything fine. But when I recieved the email – exclamation mark.
It turns out that sendmail inserts an exclamation mark into an email if it receives a string that is too long without a line break. Why? I have no idea.
Because I was sending HTML I wasn’t bothering to insert line breaks ‘\n’ into my string before passing it to php_mail() – Instead I was using <br>
So some programmer, somewhere at sometime working on sendmail decided that if a string was too long they would split it up by inserting an ! character – This is probably some strange legacy thing from eons ago before HTML was even invented.
And only one of the emails was inserting an exclamation mark because the other was under the threshold for the maximum length before sendmail inserts an exclamation mark.
So that’s a boring story about how I lost 4 hours of my life and one of the stupid reasons why software is often over budget and over schedule.
!
#1 by Kristie on July 23, 2009 - 7:48 am
This was the most interesting one of your blogs I’ve read in ages
#2 by Jay Kint on July 23, 2009 - 10:20 am
That sucks. I’ve had that type of bug happen all too often. Especially when you don’t know the gotchas about the software you’re using.
It sounds like a case of Leaky Abstractions. (http://www.joelonsoftware.com/articles/LeakyAbstractions.html) Though I admit it’s been a while since I’ve read the article, it basically says that abstractions are never perfect and that we have to know something about the internals to be able to use the abstraction (in this case, a software mailer) properly.
Good you found it.
#3 by JoshuaSmyth on July 23, 2009 - 8:02 pm
Yup, definately leaky abstraction.
#4 by Dirk on July 27, 2009 - 12:48 pm
rotfl
Is that really documented in sendmail? Who defines “too long”?
I guess the gist of the story is not to send HTML mail. Ever.
#5 by Howard Baizer on September 3, 2009 - 11:52 am
Thank you. You saved me at least four hours.
#6 by Gary Barnett on September 4, 2009 - 8:34 am
Thanks man,
This post has saved me a couple of hours of pain!
#7 by Jerry Neff on September 28, 2009 - 2:18 pm
I am sorry you lost 4 hours of your life researching the rogue exclamation mark. However, if it helps I only lost a few minutes of my life due to the same problem thanks to your post. Thank you!
Later I found this specific issue, sendmail line length, is documented in the sendmail documentation.
jln
#8 by Cybernautix Web on January 19, 2010 - 10:57 am
Great work on finding this out. Saved loads of time for someone I know. Whoever built this in to sendmail needs an IQ test.
#9 by Ed on February 16, 2010 - 3:03 pm
This is so true. I wasted a few hours because of this same exact issue.
#10 by Damith on May 10, 2010 - 3:05 am
I applied this for message content variable then that issue solved. (in php )
$message = wordwrap($message, 70);
then call mail function…..and use this $message value
Thanks
Damith
#11 by Alex on May 10, 2010 - 5:48 am
Thank you soo much.
I already wasted like 3 hours and this helped me sort it in 5 minutes.
Cheers dude
#12 by Nick on June 25, 2010 - 6:27 am
Thanks! I wasted a couple of hours thinking it was to do with character encoding. You definitely saved me a couple more!