granularity, n.

In general, the quality of being composed of large particles.

In parallel computing, the quality of being composed of large chunks of code that can run in parallel. Also defined as the ratio of computation to communication.

Fine-grain parallelism:

• Low computation/communication ratio
• Small amounts of work done between communication
• Easier to load-balance

Coarse-grain parallelism:

• High computation/communication ratio
• Large amounts of work done between communication
• Harder to load-balance

Usually it is better create a coarse program because the overheads of communication and synchronization are relatively high (cpus sit idle while waiting), although it depends on the algorithm and hardware environment. For example, if you have 500 identical nodes and simply want to split up a large array of independent elements to be processed (once), program coarsely. If you have a heterogeneous environment with constantly changing requests, program finely to allow for load-balancing (e.g., a large database search).