I went to an ENT because I was snoring and it was disturbing my sleep. He took one look in my mouth and told me my tonsils were very swollen and possibly blocking my airways. I told him about the smelly yellowish things that would be stuck in my tonsils and he said it was the byproduct of an infection.
He put me on penicillin for three weeks (apparently it's hard to get rid of, so even if you've taken antibiotics for something else it probably wasn't long enough to get the infection cleared up) and my tonsils shrank dramatically. He warned me that the infection might return and said if it became bothersome enough I'd have to have my tonsils removed.
It did come back after a month or so but much more mild than before. My tonsils are less swollen and I only have those yellow things once or twice a week instead of every day.