A mineral is a naturally occuring, generally inorganic solid with a definite chemical composition that is fixed or varies within fixed limits, and a highly ordered internal atomic arrangement, which repeats itself in three dimensions.
Lets break this down:
Naturally occuring distinguishes between those things found in nature and those created in laboratories (synthetics) or by other man made means. This can be come confusing when minerals are formed in or on man made structures by natural processes, such as salt or calcium carbonate precipitating out of water in a gutter.
Inorganic means anything created by organism can not be considered a mineral. However, aragonite can be excreted by some organisms to create their shell, and this is often still considered a mineral. This might be because it is more convienient to say "this shell is made of the mineral aragonite" than it is to say "this shell is made of calcium carbonate with the crystalline structure of aragonite". This is a bit of a grey area, as many organic compounds are changed by inorganic processes, such that they may be unrecognizable as organic fragments or are reworked by temperature, pressure or time to create a new compounds or a change in the internal structure. Petroleum and coal are usually termed mineral fuels, ignoring their organic origin.
The solid part is fairly straight forward, excluding gases and liquids, which would be excluded by the ordered internal structure requirement as well. Water is not a mineral but ice can be if it has an ordered structure.
The chemical composition part is also fairly straight forward. For example,olivine is a mineral composed of the two end members forsterite, Mg2SiO4, and fayalite, Fe2SiO4. Thus olivine can be represented as a mixture of the two by writing its formula as (Mg, Fe)2SiO4. This varies within the fixed limits of being a mixture of the two end members. A solid compound with a composition that is variable and cannot be expressed by a chemical formula would not be considered a mineral.
A highly ordered internal structure describes a structure of atoms or ions that repeats in a 3-D geometric pattern. This is the definition of a crystalline solid, so often times crystalline replaces this part of the defintion of a mineral. Any solid lacking this internal structure is called amorphous. An example of this would be the volcanic glass obsidian. Although it may have the same chemical composition as quartz, it is not a mineral because it cooled very quickly, "freezing" in the lack of structure characteristic of liquids. Other phenomena cause a lack of structure, such as damage by radiation. This often appears as a halo around the radioactive mineral in the rock, although it can ususally only be seen in thin section. Many mineral-like compounds lacking structure are termed mineraloids. The repeatition of the interal structure of atoms in three dimensions creates the faces of a mineral.
Minerals generally have a set of physical characteristics, such as melting point, conductivity, malleability, etc, but these are often highly dependent on purity, so this is not part of the definition of a mineral in most mineralogy texts. Most minerals are not pure substances, and even if they were, things such as twinning and enantiomorphism cause changes to some physical characteristics. The physical characteristics that are fairly reliable in mineral identification are density, optical characteristics viewed with a petrographic microscope, hardness, crystal system, lustre, and cleavage.
The study of minerals is called mineralogy, and this often includes the study of synthesized rock forming compounds.
Reference: Klien, C., 2002, The 22nd Edition of the Manual of Mineral Science. John Wiley and Sons Inc., New York.