There are two possible reasons why it's J of all letters:
1. Microsoft? Gmail?? it must be a conspiracy!
or,
2. It's because in Windows, the Smiley character is created using the Wingdings font, where it occupies the same position that is occupied by J in textual fonts (like Arial).
In fact, when sending a smiley in an email, this is the HTML generated by Outlook:
<font size=2 face=Wingdings><span style='font-size:10.0pt;font-family:
Wingdings'>J</span></font>
Gmail, which doesn't recognize the non-standard Wingdings font, removes it, and is left with the famous J.