If you are unhappy, It’s up to you to decide and move ahead. It’s up to you to make a change!
We can't please everyone in life. If you know what you're doing is not wrong, If you're not hurting someone, If it's not really wrong then there is nothing to be worried about. Let's always give more and expect less.