Until the 1820s, quill pens were the most commonly used writing implement in the world; these were made from feathers (usually goose feathers), and the nibs had to be cut, shaped, and sharpened. For this purpose one would use a small knife; this knife has been called a 'quill knife', 'desk knife', 'stationery knife', and most popularly, pen knife.
The original pen knives had a short blade at the end of a long handle - looking a bit like a scalpel or X-ACTO knife. Over the course of the centuries, many different designs have been used; for most of history, pen knives had very short, double-edged blades, usually with one side straight-edged and the other with a convex edge for careful shaping of the pen. These were often fitted with small sheaths; as part of a writing set, these knives had to be portable and remain sharp. In a bit of turn about, knives of almost the same form are sometimes sold today as 'pen knives', not for cutting pens, but because the long handle and short blade look something like a pen in form; a plastic pen cap functions as a cover.
In the 1800s folding pen knives became popular, and for safety reasons single-edged blades became standard. These fancy modern penknives were able to make up for this limitation by having multiple blades. At this point pen knives approached the form that we are familiar with today. They were small folding knives, with one to four blades; however, they were still, for the most part, shorter and thinner blades than you would find on the average Swiss army knife. Short blades gave more control, and narrow blades were able to make curved cuts more easily. One might carry both a delicate pen knife and a more sturdy pocket knife.
These days the term pen knife may be used for any pocket knife, although it is still most often used to refer to smaller pocket knives. While you can certainly still find pen knives intended for sharpening quills if you are willing to pay the price, very few people use true quills any more; if you like the look of a quill but do not care about authenticity, quills with metal nibs are available, saving you the need for any knife at all. If you do want to try a true quill, be warned that wood pulp paper tends to dull quills quickly, so a traditional quill has to be sharpened every few pages.