There will always be those who find fault with what you do, how you do it, why you do it, and they will tell you how they would do it.
And that's the point. They aren't doing it.
If they aren't doing it, then what they say should not impact what you do.
Follow your dreams, stick with your plan, go after what you desire most, no matter what others may say.