Quick Links

Terracotta.org Home
FAQ
Technical FAQ
Troubleshooting Guide

Community Resources

Forums
Mailing Lists
JIRA
Paste Bin
Source Repository (svn)

Unknown macro: {builder-pagetitle}

Please enumerate the various Terracotta Caching products.

• STANDALONE CACHE
  • Ehcache OSS is the leading open source caching library and is available from http://ehcache.org
  • Terracotta Ehcache DX is a commercial product. It offers a Management Console, 24x7 Support, Legal Indemnification etc in addition to the same Ehcache OSS code base.
• DISTRIBUTED CACHE
  • RMI Based Server Less Replication:
    • Terracotta Ehcache DX is a commercial product. It offers a Management Console, 24x7 Support, Legal Indemnification etc. over Ehcache OSS at ehcache.org.
  • Server Based - Higher-Scale, Higher-HA, Higher-Correctness Distributed Cache:
    • Server Based Distributed Cache - Terracotta EX is the commercial version. It offers an Operator-Console, 24x7 Support, Legal Indemnification etc. over Terracotta ES at terracotta.org.
    • Server Based Distributed Cache - Terracotta FX is also a commercial version. Over Terracotta Ehcache EX, it additionally offers Server Striping for massive Scale.
• HIBERNATE SECOND LEVEL CACHE:
  • Terracotta provides a Hibernate plugin - which then clusters the Hibernate 2nd Level Cache.

(Alex's version of the above here

• Ehcache OSS is the leading open source caching library and is available from http://ehcache.org
  • Single node caching
  • Rich caching API suitable for a wide range of applications
  • RMI, JGroups and JMS based clustered cache replication options
  • Additional Cache APIs such as JCache and Web Caching
• Terracotta Ehcache DX is a commercial product that builds on Ehcache OSS.
  • Includes all Ehcache OSS features
  • Web-based management server and console
  • HTTP-based monitoring and management APIs
  • 24x7 support
  • Legal indemnification
• Terracotta Ehcache EX is a commercial product that introduces Terracotta integration to Ehcache.
  • Includes all Ehcache DX functionality and support
  • Includes integrated Ehcache Terracotta clustering
    • Integrates the Terracotta Server Array
    • Support for higher scale, high availability, data coherency
    • Operator console
• Terracotta Ehcache FX is a commercial product that provides even greater scale-out for high-performance uses cases with Ehcache and Terracotta clustered integration
  • Includes all Ehcache EX functionality and support
  • Adds support for Terracotta Server Striping for massive scale
• Terracotta for Hibernate is a commercial product that provides a Hibernate Second Level Cache Provider
  • Hibernate 3.2 and 3.3 support
  • Hibernate developer console
  • Terracotta server integration

What do you mean by the Caching Continuum?

• As described in the prior section, Terracotta Ehcache can be deployed as a standalone cache all the way to a massively scalable distributed cache with Terracotta FX.
• One can additionally deploy an application using Ehcache to a public cloud (via the EC2 AMI) or a private cloud (using Terracotta's partnership and best practices with VMWare).
• Hence the continuum from standalone to massively scalable on commodity infrastructure and/or using public or private clouds.
• The interface used by developers with Ehcache across this continuum is identical. Developers plug in different configurations and use different deployment modes to achieve these points on the continuum but the API remains the same from a code perspective.

I understand that there are are multiple deployment modes to Terracotta Ehcache-DX/EX/FX - please enumerate.

• Ehcache OSS and Terracotta Ehcache DX can be deployed as a standalone cache or as a peer-peer (with RMI) distributed cache. Ehcache also provides a "RESTful" cache server deployment that is deployed as a WAR or as it's own server. The "RESTful" cache server can be accessed over HTTP by heterogeneous client applications (LAMP, PHP, C, Java, .NET etc).
• Terracotta Ehcache-EX/FX can additionally be deployed with Terracotta in either Express Mode or Custom Mode.
  • Express Mode is simpler and will satisfy most users. Keys and values put into the cache must be Serializable and may be copied as they are put in and out of the cache, such that there is no preserved object identity (this is similar to the peer-peer RMI distributed cache functionality.
  • Custom Mode is for advanced users. Within the Custom mode, you can choose either the "serialized" behavior or an additional "identity" mode. Identity mode maintains the object identity using Terracotta shared object technology. This allows you to lock the object on one node and have other nodes lock on the same object, as if they were in the same JVM. In addition, greater extensibility is available as you can easily mix and match other TIMs and traditional DSO along with the distributed cache solution.

What are the main components to tuning Distributed Terracotta Ehcache-DX/EX/FX?

See http://www.ehcache.org/documentation/faq.html and the table below:

Area Of Concern	Specific Concern	Why the Concern?	Tool-to-use	How-to-use-the-Tool
Application Architecture	Cache Strategy	Are you making the right decision in terms of choosing data that ought to be cached?	Developer Console Plugin for Ehcache	Dev Console highlights cache hit-ratios. There might be a case for not caching data which has very low hit-ratios (sometimes described as the "LONG-TAIL" problem). This is as against Data with a "HOT-SET" which implies heavy re-use and is an ideal candidate for caching. Similarly if data is being heavily written to (e.g. close to 50-50 read/write ratio), then unless there a write-behind to the Data-Store employed - it may not be a good candidate for caching. So typically, you get better returns if the data has a HOT-SET and is frequently accessed before expiration and the READ-WRITE ratio is skewed towards READs.
Application Architecture	Cache Strategy	For data chosen to be cached, is the right caching strategy adopted?	Developer Console - Ehcache Plugin	Look at the Cache Hit Ratios. For caches, choose between standalone OR distributed deployments based on amount of DB-offload needed and correctness. In some cases, simply add this element to ehcache.xml (<terracotta clustered="false"/>) to leave it STANDALONE. Choose between serialized/Express or identity-based Caches based on cache-correctness requirements.
Application Architecture	Locality of Reference	Poor locality implies a lot of flushing and faulting of Data (EX, FX) from the Terracotta server if the cache size exceeds size of the Client JVM Heap	Developer Console Plugin/Logs	Display Fault/Flush Rate and Broadcast Rate. If these are unacceptably high, then one may need increased Heap on the Terracotta Server and/or need Terracotta FX (i.e. more Server Stripes), so data unavailable on the client JVM is just 1 network-hop away at the Terracotta Server JVM Heap (instead of network-hop + disk-hop on the Terracotta Server). Partitioning/Sharding may be an additional option (see http://ehcache.org/documentation/cache_server.html) for some partitioning topologies with routing intelligence.
Application Architecture	Cache Deployment	Does the Cache need to be in-process or does it need to be shared across Disparate client Environments?	n/a	If sharing across heterogeneous stacks/ client applications is a requirement, consider deploying Ehcache as a "server" that exports a remoting API. REST is recommended since it is lighter-weight than SOAP. Note that the cost for cross platform access to the cache is latency in that each cache I/O operation is over the network.
Application Architecture	Cache Deployment	If Ehcache DX does Peer-Peer replication need to be Synchronous or Asynchronous?	n/a	Asynchronous may work if correctness needs across JVM Processes are not stringent and one gains better latency of write operations. The highest level of inter-thread and inter-JVM-process correctness is afforded by Terracotta Ehcache EX or FX in "identity-mode" - basically ensures that Java Memory Model semantics are honored.
Memory	Cache TTL/TTI	Are the TTL/TTI chosen for the cache commensurate with the cache freshness requirements?	Developer Console Plugin for Ehcache	Dev Console highlights cache hit-ratios including hits and misses. Making the TTL/TTI too aggressive implies greater eviction activity and hence less offload. Making it too lax implies greater memory consumption for the cache.
Memory	Cache Size	Both Size Based and Memory Based Policy management available for cache sizes	Developer Console	Displays Cache Sizes. Ensure that maxElementsInMemory and maxElementsOnDisk are chosen correctly. Ensure Virtual Memory Manager on Client-JVM and Server-JVM are tuned correctly - see link1 and link2
Memory	Garbage Collection on Client and Terracotta Server JVM	Excessive GC (especially pauseful) translates to direct latency and scale loss.	JStat, -verbose:gc and other GC profiling Tools	See link1 and link2
Memory	Distributed Garbage Collection on Terracotta Server JVM	Terracota Dev Console	Displays frequency of DGC and time take for various stages within a DGC cycle	See link1 and link2
Cache Attributes	Cache Attributes if set incorrectly can ruin Functional, Latency and Scale characteristics	Visual inspecton of ehcache.xml		Click here for Ehcache Attribute do's/don'ts

Ehcache - What are some do's/don'ts with regards to cache attributes in ehcache.xml?

• All Disk Related Attributes: diskStore/ overflowToDisk/ diskPersistent/ diskExpiryThreadIntervalSeconds/ diskSpoolBufferSizeMB -
  • These make sense only for Ehcache DX. Choose these attributes such that I/O wait on each node is minimized. In some cases consider expanding diskSpoolBufferSizeMB (default is 30MB) if ample memory is available. One could also consider RAM-based File-System.
  • For Ehcache EX and FX - ensure that these are not used at all, since client JVM either materializes the entire cache or a subset of overall cache data which is all held at Terracotta server JVM heap and/or the disk backing it. To tune Disk I/O on the Terracotta Server, ensure that the Machine has ample RAM so that BerkeleyDB blocks are cached in RAM on disk access (BerkeleyDB runs in process on the Terracotta Server) or employ Terracotta FX so that all data fits within heap on the array of Terracotta Servers.

• Eventing: cacheEventListenerFactory - leave it unset if not interested in Cache Events. There is a small performance cost to intercepting and raising events. Alternatively, use the Ehcache "quiet" API e.g. getQuiet, putQuiet.
• Cache-Sizing: - Ensure that maxElementsInMemory (number of elements in Memory before Terracotta Memory Manager/Policy runs) and maxElementsOnDisk (total number of elements in the cache) are set correctly. Note that you can also simply rely on Terracotta Virtual Memory Manager to keep heap utilization on the client JVM and the Terracotta Server JVM in check and avoid having to set cache sizes explicitly/correctly.
• Liveness: Ensure that the cache liveness settings match application requirements around freshness of data. Typically, set eternal = false, unless you rely on an external cache-clear and set timeToIdleSeconds/ timeToLiveSeconds for greater fine-grained control. TTI and TTI can also be set on a per element basis, although that incurs greater clustered object overhead.
• Memory Policy: clearOnFlush/ memoryStoreEvictionPolicy - Ignore this in case of Terracotta EX and FX, since tuning Virtual Memory Manager is the way to specify Memory based Sizing Policy in case of EX/FX
• HA: bootstrapCacheLoaderFactory -
  • In the case of Terracotta DX, this option allows a Cache to get boot-strapped from a peer JVM.
  • Do not set these in case of Terracotta EX and FX. Since caches on the client-JVM are "boot-strapped automatically" - i.e. "faulted-in" from the Terracotta Server JVM.
• Correctness: coherentReads=true|false - indicates whether this cache should have coherent reads with guaranteed correctness semantics. For read-only payloads, or payloads where stale-reads are allowed, coherentReads = false is an appropriate way to enjoy better read latencies.

Any specific Hibernate 2nd Level Cache Tuning Tips?

In addition to some of the generic tips delineated earlier on the Wiki (Garbage Collection Tuning, Memory Manager Tuning, DGC tuning etc.), this table presents some hibernate specific tuning tips. Also look at http://www.terracotta.org/web/display/orgsite/Testing+and+Tuning+Terracotta+for+Hibernate

Area of Concern	Specific Tuning Tip	Rationale
Query Cache Regions	Turn off query cache if not being used	Any change to a related table invalidates any and all query cache-elements that reference that table - hence, more cache misses. So the invalidation overhead often detracts significantly from the value that the Query Cache brings in the first place. Also, exactly one timestamp cache utilized by all query cache instances. Timestamp cache has a single coarse lock for all lookups (during query caching) and insert/update/delete SQL execution. Under load, this lock can easily become a bottleneck. If the timestamp of the last update on a table is greater than the time the query results were cached, then the entry is removed and the lookup is a cache miss. If possible, use a Criteria-based cached query lookup on an immutable natural key column, as it bypasses the table timestamp invalidation logic. Note that this lock has an impact on all insert/update/delete actions so will affect performance even if no queries are being cached!
Timestamp Cache Region	Ensure that the timestamp cache does not evict entries before the query cache.	In general, it is recommended that the timestamp cache be eternal or evicted on a "time to live" longer than the query cache "time to live". The timestamp cache should not use "time to idle" based eviction.
Concurrency Strategy	Use the right concurrency strategy for the right isolation levels.	Read Only - Only useful for data which is read, but never updated (mainly reference data). Non Strict Read Write - Used for data that hardly ever changes. Does not lock the cache, hence makes no guarantee that the item returned from the cache is the latest version available in the DB. Read/Write - Used for caching data that is sometimes updated while maintaining the semantics of "read committed" isolation level. If database is set to "repeatable read" this concurrency strategy almost maintains the semantics. Repeatable read isolation is compromised in the case of concurrent writes. Transactional - Synchronous concurrency strategy and least scalable among all. Available in a managed environment only, it guarantees full transactional isolation up to repeatable read, if required (not supported by Terracotta currently). If application can tolerate somewhat inconsistent views of data, and the data does not change frequently, consider changing the cache concurrency strategy from READ_WRITE to NONSTRICT_READ_WRITE to boost performance.
Liveness	The choice of directives to control cache element liveness needs to be made judiciously	The cause for symptoms such as high cache miss rate, high rate of flushing and faulting is often unoptimized cache eviction parameters. So, prune the cache size appropriately using any of the cache eviction policies and adjust eviction parameters for regions containing entities with high access rates to evict less frequently. Turn off eviction for often-used, long-lived data. However, too high TTI/TTL can result in stale data for data that gets mutated. Select from a combination of Time Based (Time to Idle: TTI, Time to Live: TTL) directives or Count Based directives (Target Max In-Memory Count, Target Max Total Count) to meet liveness requirements whilst still minimizing overhead.
Liveness (for Orphans)	Tune Orphan Eviction separately if defaults do not work	Advanced usage only Orphan elements are those elements which are not in the JVM heap of any Terracotta L1 node (i.e. client JVM). Disabling orphan eviction will prevent the clearing of expired orphan elements from the L2 heap until they are faulted into an L1 node. <orphan eviction> element in the config contains settings controlling the TTI/TTL based eviction of orphan elements. The orphan eviction period defines the number of regular (local) eviction cycles that occur between successive orphan evictions. Default period is 4 cycles.
Connection Pools	Use one of the supported connection pools, such as C3P0 or DBCP, and set the number of connections to an optimal amount	If this is not tuned right, Hibernate I/O to the database will suffer, which will in turn impact Application Latency and Throughput.
Statistics	Disable statistics gathering and logging during performance tests and in production	High overhead - only to be used for debugging.
Others	Enable local key caching	Terracotta for Hibernate can cache a "hotset" of serialized keys on clients to add locality-of-reference, a feature suitable for read-only cases. Note that the set of keys must be small enough for available memory.