A phoneme is a mental (as opposed to acoustic) construct which maps onto the smallest bit of sound which, if switched with another sound, will change the meaning of a word. One might say the word "pit" with a slight aspiration (a breathy sound) after the "p", or one might not. This will not change the meaning of the word.
However, if one changes it to a "b" one gets "bit", changing the "i" to an "o" gives "pot", and changing the "t" to an "n" gives "pin". So these are all phonemes. The two different "p"s are phones..allophones, actually, since they can replace each other. Another way of thinking of a phoneme is as the set of all its allophones.
Different languages consider different phones to be allophones, and that's just one of the things that makes pronunciation of a foreign language so interesting.