In words like knife and knight, where did the "K" came from ?

Both words are very old , long before the modern English: 

The word knife is old ; here are its origins:

Origin: bef. 1100; Middle English knif, Old English cn?f; c. Dutch  knijf, German Kneif, Old Norse kn?fr] 

Even today , in Norwegian Danish and Swedish its kniv

The word knight is even older:

[Origin: bef. 900; Middle English ; Old English cniht  c. German, Dutch knecht ]

http://en.wikipedia.org/wiki/Knight#Etymology

So here we see that the letter c changed to k

See here : http://en.wikipedia.org/wiki/Old_english#The_alphabet

• c (except in the digraphs sc and cg): either /t?/ or /k/. The /t?/ pronunciation is sometimes written with a diacritic by modern editors: most commonly ?, sometimes ? or ç. Before a consonant letter the pronunciation is always /k/; word-finally after i it is always /t?/. Otherwise a knowledge of the historical linguistics of the word in question is needed to predict which pronunciation is needed. (See Old English phonology#The distribution of velars and palatals for details.)

yoavdothan's answer is a very good one.  Also, in Middle English, the -gh- in 'knight' was also pronounced, much like the -ch- in the German word knecht. 

<tangent> Reform English spelling?  Never.  Do so and we will lose a beautiful way of understanding and recognizing etymology. </tangent>

England was conquered by many peoples, such as the Romans, Normans, Saxons and each made some contribution to the English language. Most of the words in English having a silent "k" were introduced by the Saxons, a Germanic people.