To be prejudice you have to be able to prejudge. The root cause of that is often misinformation given to a person by someone else. Often prejudices are handed down from one generation to another.
Or by making generalizations. Say you meet one person of a particular religion and assume they are all like that? Suppose someone met someone from the KKK, who claim they are Christians, and base their opinion of all Christians on that one person?
To sum it up is ignorance and narrow-mindedness. If people can learn to rationally discuss things and learn to answer questions without be offended that the question was asked, we can all get along better.