Why do so many people care if Obama is Muslim?

The founding fathers came here mainly for religious freedom so what's the big deal what religion he is? Do people actually think that all Muslims are terrorists? I've heard a lot of stories about terrorists who call them selves Christian or Catholic or Atheist but I don't generalize my feelings about every one with those religious views based on what some crazy people have done in the name of there God. So why do people care so much if he is Muslim? I don't agree with every thing he does but those are political opinions. I don't think a persons religion should have anything to do with how good they are or aren't at there job. And to the people who do care can't they at least admit that this is prejudice? it seems like a way for people to complain about him when they're to lazy to actually pay attention to what he's doing.