The allosteric site is the portion of a substrate which allows an allosteric inhibitor to engage allosteric change or allosteric interaction with the molecule (as you can see, 'allosteric' is a rather important term for the whole process). Allosteric sites must be differently shaped from active sites to make sure that the molecule is only modified and not activated.

Any and all non-active sites on a molecule are called allosteric sites.