This story contains no information of interest to anyone what-so-ever.
But it annoyed me, so let me vent.
I was already over my time-estimate, and I had a bug. It was strange, the system was sending out two emails and one had an exclamation mark; just randomly right in the footer – But the other was fine. After staring at my code and deciding that there was nothing wrong I decided to view-source in the browser – Yup, email had an exclamation mark right in the middle.
But I wasn’t putting it in there, and the code to attach the footer to each email was exactly the same. However only one email was displaying an exclamation mark.
See this project was in .php on a linux server – I’m not a php guru, nor a linux expert – previous to last month I had written approximately 50 lines of php code, and I’ve run linux, well never.
So I was convinced it was my bug, something I’d done wrong, something about the php_mail() function that I didn’t know about – It didn’t help that the server hadn’t been patched since 2004 and was still running php 4.2 so I couldn’t use any of the email libraries that kept being recommended in my google searches and promised to make my life easier.
So I entered a new test order into the system, and I got my two emails – One with an exclamation mark and one without, but this time the exclamation mark had moved. My previous testing at always shown it in the same place.
Test and measure, it’s the scientific method.
So I try entering a new order, this time adding exactly 4 characters to the address on the order form. This time the exclamation mark has moved exactly 4 places to the left.
Ah ha, I’ve got something! – I thought to myself. The system was no longer random, no longer chaotic, I had something deterministic – I had no idea why but the length of the email was causing an exclamation mark to be inserted
I modified my code output the email to the browser before sending it to php_mail() – No exclamation mark, formating all correct, everything fine. But when I recieved the email – exclamation mark.
It turns out that sendmail inserts an exclamation mark into an email if it receives a string that is too long without a line break. Why? I have no idea.
Because I was sending HTML I wasn’t bothering to insert line breaks ‘\n’ into my string before passing it to php_mail() – Instead I was using <br>
So some programmer, somewhere at sometime working on sendmail decided that if a string was too long they would split it up by inserting an ! character – This is probably some strange legacy thing from eons ago before HTML was even invented.
And only one of the emails was inserting an exclamation mark because the other was under the threshold for the maximum length before sendmail inserts an exclamation mark.
So that’s a boring story about how I lost 4 hours of my life and one of the stupid reasons why software is often over budget and over schedule.
!
#1 by Kristie on July 23, 2009 - 7:48 am
This was the most interesting one of your blogs I’ve read in ages
#2 by Jay Kint on July 23, 2009 - 10:20 am
That sucks. I’ve had that type of bug happen all too often. Especially when you don’t know the gotchas about the software you’re using.
It sounds like a case of Leaky Abstractions. (http://www.joelonsoftware.com/articles/LeakyAbstractions.html) Though I admit it’s been a while since I’ve read the article, it basically says that abstractions are never perfect and that we have to know something about the internals to be able to use the abstraction (in this case, a software mailer) properly.
Good you found it.
#3 by JoshuaSmyth on July 23, 2009 - 8:02 pm
Yup, definately leaky abstraction.
#4 by Dirk on July 27, 2009 - 12:48 pm
rotfl
Is that really documented in sendmail? Who defines “too long”?
I guess the gist of the story is not to send HTML mail. Ever.
#5 by Howard Baizer on September 3, 2009 - 11:52 am
Thank you. You saved me at least four hours.
#6 by Gary Barnett on September 4, 2009 - 8:34 am
Thanks man,
This post has saved me a couple of hours of pain!
#7 by Jerry Neff on September 28, 2009 - 2:18 pm
I am sorry you lost 4 hours of your life researching the rogue exclamation mark. However, if it helps I only lost a few minutes of my life due to the same problem thanks to your post. Thank you!
Later I found this specific issue, sendmail line length, is documented in the sendmail documentation.
jln
#8 by Cybernautix Web on January 19, 2010 - 10:57 am
Great work on finding this out. Saved loads of time for someone I know. Whoever built this in to sendmail needs an IQ test.
#9 by Ed on February 16, 2010 - 3:03 pm
This is so true. I wasted a few hours because of this same exact issue.
#10 by Damith on May 10, 2010 - 3:05 am
I applied this for message content variable then that issue solved. (in php )
$message = wordwrap($message, 70);
then call mail function…..and use this $message value
Thanks
Damith
#11 by Alex on May 10, 2010 - 5:48 am
Thank you soo much.
I already wasted like 3 hours and this helped me sort it in 5 minutes.
Cheers dude
#12 by Nick on June 25, 2010 - 6:27 am
Thanks! I wasted a couple of hours thinking it was to do with character encoding. You definitely saved me a couple more!
#13 by Wendy on September 21, 2010 - 10:12 am
Another one of your happy customers
I too had a mysterious exclamation mark appearing – now I can safely skip this problem and move on to the actual problem I am trying to solve. Thank you!
#14 by Eugene on March 18, 2011 - 8:48 am
Sorry to be late, but long raw lines in email violate RFC 2822 2.1.1. Line Length Limits
Each line of characters MUST be no more than 998 characters, and SHOULD be no more than 78 characters, excluding the CRLF.
http://tools.ietf.org/html/rfc2822
So there you go, your messages were breaking RFC and sendmail is obligated to enforce RFC.
#15 by noodlekadoodle on May 28, 2011 - 9:15 am
Thanks for your story… I was on the same track as you, finding the exclamation point in my emails, changing the content to find that the exclamation point moved, printing to the screen to find there’s no exclamation point there. It probably would have taken a couple more hours to figure out that it was a lack of newlines causing the problem. So, thank you very much for posting your experience and saving me time.
#16 by chung on June 23, 2011 - 12:12 am
Thanks for the solution…
#17 by Janis on July 18, 2011 - 11:22 am
Thanks a lot! Saved me time too. I had seen this before when using sendmail from Perl, but had forgotted the exact reason and how it can be dealt with. In Perl, simply split the long string and insert prior to the (!)
print MAIL “\n”;
where MAIL is the variable for the sendmail handle.
Cheers
#18 by Mike on August 16, 2011 - 9:34 am
you just saved me multiple hours of pain and suffering!
and thank you for making the title search engine friendly by actually typing out “exclamation!”
#19 by Abey Babu on September 7, 2011 - 10:37 am
saved me a lot of hours . was #4 in my google search
Thanks,
Abey
#20 by Sean on September 30, 2011 - 8:06 am
MUCH appreciated!
Saved me quite a bit of grief
#21 by Joshua Smyth (Admin) on September 30, 2011 - 5:20 pm
Haha, cool an answer to the reason!
#22 by The Web Taylor on September 5, 2012 - 8:40 am
OMG!! Thank you so much! I have spent hours trying to figure out why a stupid exclamation mark was being added!
#23 by RandomDude on November 7, 2012 - 1:32 pm
Thanks for posting this. I have PL/SQL that generates an html table and sends email. To resolve this issue, I appended chr(13) and chr(10) at the end of each row. These are the ascii characters for carriage return and line feed. For example:
my_table := my_table || ” || x.val || ” || chr(13) || chr(10);
#24 by Piyush Raja on November 29, 2012 - 2:48 am
This solved my problem and saved me quite some time.
Thanks,
Piyush
#25 by Vetrivel on December 5, 2012 - 8:22 am
It is really helpful to me to trouble shooting the same kind of situation. Thanks for posting this!!! I believe it others as well in the future.
#26 by Patrick on December 7, 2012 - 2:02 pm
Thanks ! so much for help!ng fix this seemingly ! completely rando!m problem!
#27 by Grant on June 10, 2013 - 9:27 am
Discovered this issue after an HTML-encoded email I sent and thought it was character encoding. I wasted about an hour of my life but thanks to this article, I can move on – problem solved!