There are two possible reasons why it's J of all letters:

1. Microsoft? Gmail?? it must be a conspiracy!

or,

2. It's because in Windows, the Smiley character is created using the Wingdings font, where it occupies the same position that is occupied by J in textual fonts (like Arial).  

In fact, when sending a smiley in an email, this is the HTML generated by Outlook:

<font size=2 face=Wingdings><span style='font-size:10.0pt;font-family:
Wingdings'>J</span></font>

Gmail, which doesn't recognize the non-standard Wingdings font, removes it, and is left with the famous J.