Message Passing versus Shared Memory

Traditionally, communication among processes in a distributed system is based on the data-passing model. Message-passing systems or systems that support remote procedure calls (RPC's) adhere to this model. The data passing model logically and conveniently extends the underlying communication mechanism of the system; primitives such as Send and Receive are used for interprocess communication. In contrast to the data-passing model, the shared memory model provides processes in a system with a shared address space. Application programs can use this space in the same way they use normal local memory. That is, data in the shared space is accessed through Read and Write operations. As a result, applications can pass shared information by reference. The shared memory model is natural for distributed computations running on shared memory multiprocessors.

In some cases, applications using distributed shared memory can even outperform their message passing counterparts. This is possible for these reasons:

1. For shared memory algorithms that move data between hosts in large blocks, communication overhead is amortized over multiple memory accesses, reducing overall communication requirements if the application exhibits a sufficient degree of locality in its data accesses.
2. Many parallel applications execute in phases, where each computation phase is preceded by a data exchange phase. The time needed for the data-exchange phase is often dictated by the throughput limitations of the communications system. Distributed shared memory algorithms typically move data on demand as they are being accessed, eliminating the data-exchange phase, spreading the communication load over a longer period of time, and allowing for a greater degree of concurrency.
3. The total amount of memory may be increased proportionly, reducing paging and swapping activity.