MySQL Server层的 max_connections 和引擎层的 innodb_thread_concurrency

news/2024/12/22 10:53:23/

MySQL 数据库服务端是由 Server 层 和 引擎层组成

  • Server 层包括连接器、查询缓存、分析器、优化器、执行器等,涵盖 MySQL
    的大多数核心服务功能,以及所有的内置函数(如日期、时间、数学和加密函数等),所有跨存储引擎的功能都在这一层实现,比如存储过程、触发器、视图等。
  • 存储引擎层负责数据的存储和提取。其架构模式是插件式的,支持 InnoDB、MyISAM、Memory 等多个存储引擎。从 MySQL 5.5.5 版本开始InnoDB成为了默认存储引擎。

max_connections: Server层的最大连接并发数

通过 mysql client 或者 驱动程序连接 MySQL时,每创建一个连接,MySQL 的 Server层 就会创建一个线程来处理该连接(实际情况更可能是 MySQL server 是用 thread pool 线程池来处理连接请求的:当客户端1的连接断开时,对应的线程进入线程池而不是销毁;客户端2马上建立连接就会发现客户端1用过的THREAD_OS_ID在处理客户端2的连接;站在操作系统角度看对于建立和断开链接不会存在线程的创建和销毁,当然前提是线程池里有空闲线程);此处的线程就是通过 show processlist 查看到的列表,你可能会有疑问 show processlist 的 id 和操作系统 ps 查看到的进程或线程id 有什么关系? 其实是一一对应的,比如 show processlist 的 id 844227 就是 performance_schema.threads 表的列 PROCESSLIST_ID 从而可以得知对应的 THREAD_OS_ID 为6789,通过 ps -eLf 就可以看到该值。(具体可以并发处理多少连接是由环境变量 max_connections 决定的,这是 MySQL Server层的行为还未进入引擎层,也就是说和环境变量 innodb_thread_concurrency 还没有关系呢) 具体如下:

	mysql> show processlist;+--------+------+-----------+------+---------+------+----------+------------------+| Id     | User | Host      | db   | Command | Time | State    | Info             |+--------+------+-----------+------+---------+------+----------+------------------+| 844227 | root | localhost | NULL | Sleep   |  530 |          | NULL             || 844228 | root | localhost | NULL | Query   |    0 | starting | show processlist || 844230 | root | localhost | NULL | Sleep   |  505 |          | NULL             || 844231 | root | localhost | NULL | Sleep   |  959 |          | NULL             || 844232 | root | localhost | NULL | Sleep   |  930 |          | NULL             || 844233 | root | localhost | NULL | Sleep   |  845 |          | NULL             || 844234 | root | localhost | NULL | Sleep   |  834 |          | NULL             || 844236 | root | localhost | NULL | Sleep   |  836 |          | NULL             || 844237 | root | localhost | NULL | Sleep   |  583 |          | NULL             |+--------+------+-----------+------+---------+------+----------+------------------+9 rows in set (0.00 sec)mysql> mysql> //show processlist 为9个连接,对应的Threads_connected 也为9,mysql> //同时 Threads_cached 为0代表线程池无空闲线程;第10个连接进来时会创建新的线程mysql> SHOW STATUS LIKE 'Threads%'; +-------------------+-------+| Variable_name     | Value |+-------------------+-------+| Threads_cached    | 0     || Threads_connected | 9     || Threads_created   | 2477  || Threads_running   | 1     |+-------------------+-------+4 rows in set (0.01 sec)mysql>mysql> select *  from performance_schema.threads  where type = 'FOREGROUND' and name like '%conn%' ;+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+| THREAD_ID | NAME                      | TYPE       | PROCESSLIST_ID | PROCESSLIST_USER | PROCESSLIST_HOST | PROCESSLIST_DB | PROCESSLIST_COMMAND | PROCESSLIST_TIME | PROCESSLIST_STATE | PROCESSLIST_INFO                                                                            | PARENT_THREAD_ID | ROLE | INSTRUMENTED | HISTORY | CONNECTION_TYPE | THREAD_OS_ID |+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+|    844252 | thread/sql/one_connection | FOREGROUND |         844227 | root             | localhost        | NULL           | Sleep               |              581 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |         6789 ||    844253 | thread/sql/one_connection | FOREGROUND |         844228 | root             | localhost        | NULL           | Query               |                0 | Sending data      | select *  from performance_schema.threads  where type = 'FOREGROUND' and name like '%conn%' |             NULL | NULL | YES          | YES     | Socket          |        28020 ||    844255 | thread/sql/one_connection | FOREGROUND |         844230 | root             | localhost        | NULL           | Sleep               |              556 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28276 ||    844256 | thread/sql/one_connection | FOREGROUND |         844231 | root             | localhost        | NULL           | Sleep               |             1010 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28036 ||    844257 | thread/sql/one_connection | FOREGROUND |         844232 | root             | localhost        | NULL           | Sleep               |              981 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28356 ||    844258 | thread/sql/one_connection | FOREGROUND |         844233 | root             | localhost        | NULL           | Sleep               |              896 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |         6785 ||    844259 | thread/sql/one_connection | FOREGROUND |         844234 | root             | localhost        | NULL           | Sleep               |              885 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28277 ||    844261 | thread/sql/one_connection | FOREGROUND |         844236 | root             | localhost        | NULL           | Sleep               |              887 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28222 ||    844262 | thread/sql/one_connection | FOREGROUND |         844237 | root             | localhost        | NULL           | Sleep               |              634 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28360 |+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+9 rows in set (0.00 sec)mysql>[root@dbs-test-rob ~]# //看最后一行的线程的启动时间 16:43 可知,这就是在线程池中无空闲线程时为接下来的新建连接(第10个连接)而创建的线程[root@dbs-test-rob ~]# ps -eLf | grep mysqld mysql     4105     1 28020  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28036  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28222  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28276  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28277  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28356  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 28360  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1  6785  0   37 9月08 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1  6789  0   37 9月08 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pidmysql     4105     1 23536  0   37 16:43 ?        00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid[root@dbs-test-rob ~]#

innodb_thread_concurrency: InnoDB 引擎层的最大并发执行的线程数

innodb_thread_concurrency:InnoDB tries to keep the number of operating system threads concurrently inside InnoDB less than or equal to the limit given by this variable (InnoDB uses operating system threads to process user transactions). Once the number of threads reaches this limit, additional threads are placed into a wait state within a “First In, First Out” (FIFO) queue for execution(一旦数量达到了限制,额外的线程被放到了 FIFO队列里). Threads waiting for locks are not counted in the number of concurrently executing threads.

innodb_thread_sleep_delay:Defines how long InnoDB threads sleep before joining the InnoDB queue, in microseconds. The default value is 10000. A value of 0 disables sleep. You can set innodb_adaptive_max_sleep_delay to the highest value you would allow for innodb_thread_sleep_delay, and InnoDB automatically adjusts innodb_thread_sleep_delay up or down depending on current thread-scheduling activity. This dynamic adjustment helps the thread scheduling mechanism to work smoothly during times when the system is lightly loaded or when it is operating near full capacity.

innodb_concurrency_tickets:Determines the number of threads that can enter InnoDB concurrently. A thread is placed in a queue when it tries to enter InnoDB if the number of threads has already reached the concurrency limit. When a thread is permitted to enter InnoDB, it is given a number of “ tickets” equal to the value of innodb_concurrency_tickets, and the thread can enter and leave InnoDB freely until it has used up its tickets. After that point, the thread again becomes subject to the concurrency check (and possible queuing) the next time it tries to enter InnoDB. The default value is 5000.

With a small innodb_concurrency_tickets value, small transactions that only need to process a few rows compete fairly with larger transactions that process many rows. The disadvantage of a small innodb_concurrency_tickets value is that large transactions must loop through the queue many times before they can complete, which extends the amount of time required to complete their task.(大事务在完成之前必须多次通过队列,这增大了他们完成事务的总时间)

With a large innodb_concurrency_tickets value, large transactions spend less time waiting for a position at the end of the queue (controlled by innodb_thread_concurrency) and more time retrieving rows. Large transactions also require fewer trips through the queue to complete their task. The disadvantage of a large innodb_concurrency_tickets value is that too many large transactions running at the same time can starve smaller transactions by making them wait a longer time before executing.

With a nonzero innodb_thread_concurrency value, you may need to adjust the innodb_concurrency_tickets value up or down to find the optimal balance between larger and smaller transactions. The SHOW ENGINE INNODB STATUS report shows the number of tickets remaining(剩下的tickets) for an executing transaction in its current pass through the queue. This data may also be obtained from the TRX_CONCURRENCY_TICKETS column of the Information Schema INNODB_TRX table.

进一步的思考:
如果 MySQL 系统中,突然慢查询陡增或者普通的一个查询也需要耗时很久,有可能是并发执行的线程达到了 innodb_thread_concurrency :因为并发执行的线程达到了 innodb_thread_concurrency 后,再有新连接过来执行一个哪怕最简单的按ID查询,该查询也会需要很长时间才能返回;因为只能等到“正在执行的线程”把自己的 tickets(innodb_concurrency_tickets)消耗完了,才会换出;进而从FIFO中取出一个待执行的 thread 比如按照ID的查询

mysql> show engine innodb status \G
*************************** 1. row ***************************Type: InnoDBName:
Status:
=====================================
2023-09-18 10:51:17 0x7f1c3171f700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 19 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 494 srv_active, 0 srv_shutdown, 49571163 srv_idle
srv_master_thread log flush and writes: 49571657
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 152
OS WAIT ARRAY INFO: signal count 129
RW-shared spins 0, rounds 194, OS waits 84
RW-excl spins 0, rounds 142, OS waits 0
RW-sx spins 15, rounds 193, OS waits 1
Spin rounds per wait: 194.00 RW-shared, 142.00 RW-excl, 12.87 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 21627
Purge done for trx's n:o < 21625 undo n:o < 0 state: running but idle
History list length 0
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421234659759952, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421234659760864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
515 OS file reads, 2656 OS file writes, 587 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:insert 0, delete mark 0, delete 0
discarded operations:insert 0, delete mark 0, delete 0
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 1 buffer(s)
Hash table size 34673, node heap has 1 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number 13325540
Log flushed up to   13325540
Pages flushed up to 13325540
Last checkpoint at  13325531
0 pending log flushes, 0 pending chkp writes
399 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 137428992
Dictionary memory allocated 239644
Buffer pool size   8191
Free buffers       7657
Database pages     532
Old database pages 211
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 472, created 60, written 2191
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 532, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=4105, Main thread ID=139759309551360, state: sleeping
Number of rows inserted 20397, updated 78, deleted 3, read 24652
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================1 row in set (0.00 sec)
mysql>
mysql>
mysql> select trx_id,trx_state,trx_query,trx_operation_state,trx_concurrency_tickets from information_schema.innodb_trx \G
*************************** 1. row ***************************trx_id: 162612trx_state: RUNNINGtrx_query: insert into testti3 select * from testti3trx_operation_state: NULL
trx_concurrency_tickets: 10
*************************** 2. row ***************************trx_id: 422212176322720trx_state: RUNNINGtrx_query: insert into testui select * from testuitrx_operation_state: sleeping before entering InnoDB
trx_concurrency_tickets: 0
2 rows in set (0.32 sec)从trx_operation_state中可以看到他们不断的在进行轮换的进入的innodb层次,同时我们还能看到
活跃事物trx_concurrency_tickets这个tickets不断的减少,而处于sleeping before entering InnoDB
的事物其trx_concurrency_tickets为0。

http://www.ppmy.cn/news/1122998.html

相关文章

Cannot find module ‘core-js/modules/es6.regexp.constructor‘

npm run dev 之后报如下错误 解决方法&#xff1a;npm install core-js2 如果超时或者下载时间慢可以尝试 用cnpm install core-js2

AFFiNE:下一代全能知识库工具

安装使用 AFFiNE 最简单的方法是使用 Docker 一键启动&#xff1a; docker pull ghcr.io/toeverything/affine:nightly-latest docker run -p 4200:3000 --name affine -d ghcr.io/toeverything/affine:nightly-latest 然后在浏览器中打开 http://localhost:4200 即可。 可…

windows 深度学习环境部署

1. 根据显卡配置安装适合的CUDA,查看显卡配置可在显卡控制面板上查看,安装是否成功可通过nvidia-smi查看&#xff1b;注意安装路径 https://developer.nvidia.com/cuda-toolkit-archive 2. 根据cuda安装合适的cudnn&#xff0c;需要注册NVIDIA cuDNN Archive | NVIDIA Devel…

运动项目记录

桌上足球 规则讲解 - 发球、死球规则运球&控球前锋、中锋、后卫、守门员 共11人。

Android问题笔记 - kotlin中使用Java接口,报错Parameter specified as non-null is null 快速解决

点击跳转>Unity3D特效百例点击跳转>案例项目实战源码点击跳转>游戏脚本-辅助自动化点击跳转>Android控件全解手册点击跳转>Scratch编程案例点击跳转>软考全系列 &#x1f449;关于作者 专注于Android/Unity和各种游戏开发技巧&#xff0c;以及各种资源分享&…

【LeetCode75】第六十三题 不同路径

目录 题目&#xff1a; 示例&#xff1a; 分析&#xff1a; 代码&#xff1a; 题目&#xff1a; 示例&#xff1a; 分析&#xff1a; 题目给我们返回地图的长和宽。问我们从地图的左上走到右下有几种方法。我们只能往下走或是往右走。 这个算是简单的二维动态规划题了。 …

Linux离线安装elasticsearch|header|kibna插件最详细

1.准备软件安装包 [hadoophost152 elasticsearch]$ ll -rw-r--r--. 1 hadoop hadoop 515807354 9月 23 23:40 elasticsearch-8.1.1-linux-x86_64.tar.gz -rw-r--r--. 1 hadoop hadoop 1295593 9月 23 23:48 elasticsearch-head-master.tar.gz -rw-r--r--. 1 hadoop hadoop…

【网络八股】TCP八股

网络八股 请简述TCP/IP模型中每层的作用&#xff0c;典型协议和典型设备介绍一下三次握手的过程介绍一下四次挥手的过程必须三次握手吗&#xff0c;两次不行吗&#xff1f;为什么ACK数据包消耗TCP的序号吗三次握手中可以携带应用层数据吗四次挥手时&#xff0c;可以携带应用层数…