BigMemory Max Explained

Nodes running JVMs can have a large amount of physical memory—16GB, 32GB, and more—but the long-standing problem of Java garbage collection (GC) limits the ability of all Java applications, including Terracotta software, to use that memory effectively. This drawback has limited Terracotta servers, for example, to using a small Java object heap as an in-memory store, backed by a limitless but slower disk store.

How BigMemory Improves Performance

The performance of Terracotta clients and server instances is affected by the amount of faulting required to make data available. In-memory data elements are fetched speedily because memory is very fast. Data elements that are not found in memory must be faulted in from disk, and sometimes from an even slower system of record, such as a database.

While disk-based storage slows applications down, it has the advantage of being limitless in size. In-memory storage is limited in size by system and hardware constraints, yet even this limit is difficult for Java heaps to reach due to the heavy cost imposed by GC. Full GC operations can slow a system to a crawl, and the larger the heap, the more often these operations are likely to occur. In most cases, heaps have been limited to about 2GB in size.

BigMemory allows Terracotta servers to expand memory storage in a way that bypasses the limitations resulting from Java GC. Using this off-heap memory gives the Terracotta cluster a number of important advantages:

Larger in-memory stores without the pauses of GC.
More locality of reference as more data is stored at the client.
Overall reduction in faulting from disk or database.
Low latencies as a result of more data available to applications at memory speed.
Fewer Terracotta server stripes required to efficiently handle the same amount of data.

Data stored in off-heap memory is stored in a cache, and therefore all data elements (keys and values) must be serializable. However, the costs imposed by serialization and deserialization are far outweighed by the performance gains noted above.

The following diagram illustrates how BigMemory adds a layer of off-heap memory storage that reduces faulting from the Terracotta server's disk yet remains outside of GC's domain.

BigMemory in the Terracotta Server

Using BigMemory With Terracotta Servers

BigMemory runs on each Terracotta server in a Terracotta Server Array. To use BigMemory, you must install and run a Terracotta enterprise kit (version 3.4.0 or higher) and a valid Terracotta license key that includes BigMemory.

BigMemory can use a 32-bit or 64-bit JVM (see Release and Platform Compatibility Information to locate supported platforms). 64-bit systems can operate with more memory than 32-bit systems. Running BigMemory on a 64-bit system allows for more off-heap memory to be allocated.

NOTE: Using a 32-bit JVM
The amount of heap-offload you can achieve is limited by addressable memory. For a 32-bit process model, the maximum virtual address size of the process is typically 4GB, though most 32-bit operating systems have a 2GB limit. The maximum heap size available to Java is lower still due to particular OS limitations, other operations that may run on the machine (such as mmap operations used by certain APIs), and various JVM requirements for loading shared libraries and other code. A useful rule to observe is to allocate no more to off-heap memory than what is left over after `-Xmx` is set. For example, if you set `-Xmx3G`, then off-heap should be no more than 1GB. Breaking this rule may not cause an OOME on startup, but one is likely to occur at some point during the JVM's life.

See the configuration page for information on how to configure BigMemory on the Terracotta Server Array.