Home > > > TiDB Out of Memory (OOM) Errors: Root Cause Analysis & Memory Tuning Guide

TiDB Out of Memory (OOM) Errors: Root Cause Analysis & Memory Tuning Guide

Author : Mafiree Team | Published On : 04 Sep 2026

Overview

TiDB Out of Memory errors have two distinct forms, and figuring out which one occurred is the essential first step in troubleshooting. The immediate fix is usually a simple restart, but understanding the actual cause is what prevents recurrence.

There Are Two Different TiDB OOM Situations

In the first case, the Linux operating system runs out of available memory and kills the tidb-server process, dropping active connections and terminating running queries before TiDB restarts - this is the more serious scenario since it affects the entire server. In the second case, TiDB's own internal memory controller cancels a single runaway query (returning an "Out of Memory Quota" error) while the node itself stays healthy. This is the protection system functioning as intended, though the underlying cause still deserves investigation.

How TiDB Memory Controls Work

TiDB's memory thresholds operate in stages: 0–70% is the normal operating range; 70–80% is a good point for early-warning alerts by setting tidb_memory_usage_alarm_ratio to 0.7; and 80%+ is the default tidb_server_memory_limit threshold where TiDB begins killing queries.

First: Confirm What Actually Happened

Before changing any configuration, determine which type of OOM occurred. Check OS logs with dmesg -T | grep tidb-server for oom-killer activity, look at the "Welcome to TiDB" restart line in tidb.log as a timestamp anchor, and cross-check the Grafana memory usage graph under TiDB → Server → Memory Usage - a pattern that climbs steadily, drops to zero, then climbs again typically signals a process restart.

What Usually Causes TiDB OOM Errors?

The most common root causes are memory-heavy queries with large intermediate result sets (the most frequent cause), too many concurrent sessions collectively exhausting available RAM (common), memory not being released properly over time (occasional), and under-provisioned hardware (less common). The first two account for most production incidents.

SQL Patterns That Consume Large Amounts of Memory

HashJoin against a large table: builds a huge in-memory hash table on the inner side and can consume gigabytes. Checking EXPLAIN for large estRows on the inner side and hinting toward MergeJoin can reduce memory use, since MergeJoin works with sorted streams instead of a hash table.

HashAgg on large grouped datasets: builds per-worker hash tables for grouped data, which can be replaced with StreamAgg for lower memory use by processing sorted rows.

Stale statistics causing bad query plans: outdated statistics can cause the optimizer to badly misjudge row counts (e.g., estimating 1,000 rows for a 10-million-row scan) and pick memory-hungry algorithms. Checking SHOW STATS_HEALTHY and running ANALYZE TABLE regularly on high-write tables helps.

Large transactions holding too much memory: TiDB caches all writes in memory before commit, so a transaction touching millions of rows can use two to three times the actual data size. Breaking bulk deletes/updates into smaller batches (with a brief 50–100ms sleep between rounds) or using tidb_dml_type = "bulk" or non-transactional DML mitigates this.

TiDB Memory Parameters Worth Configuring

TiDB memory protection operates in layers — per-query, per-instance, and OS/cgroup limits. Four parameters matter most:

  • tidb_mem_quota_query (default 1GB, session/global): caps memory per query.

  • tidb_mem_oom_action (default CANCEL, session/global): CANCEL terminates the offending query and is right for production; LOG lets the query continue while logging the event, useful only temporarily for investigation.

  • tidb_server_memory_limit (default 80% of system memory, v6.5.0+): caps the whole TiDB process; should be set explicitly in hybrid deployments sharing a server with other workloads.

  • tidb_memory_usage_alarm_ratio (default 0.8, global): triggers diagnostic collection; setting it to 0.7 gives earlier warning.

TiDB Disk Spill: Let TiDB Use Disk Instead of RAM

Disk spill lets TiDB offload intermediate execution data (from Sort, MergeJoin, HashJoin, HashAgg, or TopN operators) to disk under memory pressure rather than crashing, governed by tidb_mem_quota_query, tidb_enable_tmp_storage_on_oom, tmp-storage-path, and tmp-storage-quota. Spill support has improved across versions, especially for HashAgg, so behavior should be verified per release. A dedicated tmp-storage-path is recommended, but spill is a safety net, not a substitute for query optimization.

View TiDB Memory Usage Through INFORMATION_SCHEMA

TiDB exposes MEMORY_USAGE, CLUSTER_MEMORY_USAGE, MEMORY_USAGE_OPS_HISTORY, and CLUSTER_MEMORY_USAGE_OPS_HISTORY, with the OPS_HISTORY tables retaining the latest 50 records per instance for post-incident analysis.

Production Readiness Checklist

  • Memory configuration: explicit tidb_server_memory_limit, workload-based tidb_mem_quota_query, tidb_mem_oom_action set to CANCEL, appropriate alarm ratio.

  • Query optimization: review with EXPLAIN ANALYZE, check HashJoin/HashAgg, refresh stale statistics, monitor high-memory queries.

  • Transaction management: batch large DELETE/UPDATE operations, avoid unnecessary million-row transactions, evaluate bulk/non-transactional DML.

  • Disk spill: dedicated temp storage path, sufficient disk space, monitor spill usage.

  • Infrastructure: avoid undersized hardware, account for co-located services, configure cgroup boundaries in hybrid deployments.

If recurring OOM issues persist despite these steps, bringing in DBA consulting services can provide the deeper diagnostic expertise needed to resolve them. 

Conclusion

TiDB OOM errors are rarely random - they trace back to specific SQL patterns, undersized memory limits, or mismatched defaults. Confirming whether the OS or TiDB's own memory manager caused the event is the first troubleshooting step; from there, tuning tidb_mem_quota_query and tidb_server_memory_limit, optimizing costly queries, and enabling disk spill as a safety net prevents most repeat incidents.