Firstly, a cup of coffee at 100 degrees will take longer to get to 80 degrees than a cup of coffee that started at 90 degrees. The "exponential" aspect is that when you look at the temperature over time, the temperature will decrease asymptotically to room temperature, meaning that as it gets closer to room temperature, the temperature will change more slowly. This is due to the temperature difference between the coffee and the room (the temperature difference is the driving force of the exchange of energy). So, that first degree from 100 to 99 will be "faster" than the temperature drop from 90 to 89. There is no "acceleration" of the temperature change that gives advantage to starting out hotter.
The difference between "white" coffee and "black" coffee in terms of cooling effects is minimal. Yes, black bodies radiate away more heat, but the reality of the situation is such that it is inconsequential to the heat capacity of the coffee cup, the exposed surface area, and the environment.
But will it be better to add milk in five minutes, or should I add it now and wait five minutes? Given the volumes and factors involved... IT DOESN'T MATTER.