Issues with person on the other end being able to hear me when I make or receive a call

Any suggestions on how to fix an issue I've been having for a while with my Galaxy S5...When I answer an incoming call or make an outgoing call, the person on the other end cannot hear me unless I activate the speaker button on the phone. Extremely annoying as obviously, I am not able to talk on the phone in private. Any suggestions on how to fix this would be greatly appreciated! Thanks!!!