You are aware that you have the right to disagree with someone over something, you should know how to do it without being aggressive, and there is a chance that the fear of looking aggressive would make you so over-careful that you end up not speaking your mind properly.
The fact that you suppress your opinion is not right, and in fact can be mentally harmful. It can damage the potential constructive relationship that you could have with the person you have not shared your real/different opinion with.
But, the questions are :
how you can disagree without looking like a difficult person,
and how to speak your mind without looking like a diffident one?
And now in my 30s, I came to realize how important it is to find practical answers to these questions.