You’ll get different answers from different people, but I’m of the opinion that exclamation marks should be used in dialogue only, with a few rare exceptions.
Interesting point! I use them often, because they convey the sense of energy of the sentence. With a period, “interesting point” becomes stuffy and academic. Here, it’s like, “Hey! I agree with you on a topic that I find interesting.”
It really depends on your capabilities as a writer, Write it how you want to first, then comb over it and see what’s not necessary or excessive.
If you’re struggling to fix some of the excess exclamation points while cutting them out, you can always reword the sentence so the meaning isn’t quite as lost as it is with a period.