I do agree with the fact that you should not be affected by the appearance of a person. Transcendentalism also confirms the above idea. There are three main reasons that we should not decide to do something according to his/her appearance; First, it may lead to a misjudgment about that person. Second is the fact of differences between appearance and one's real personality. Finally, it may make you ashamed!
The external appearance of a person who you are facing with may misguide you. That will lead to a misjudgment from your side. I would like to explain one of my memories.