我们的环境是 mysql 社区版 5.7.21 MGR-single Primary，环境说明：32G内存，8CPU，centos7.4，glibc-2.17,gcc-c++-4.8.5,gcc-4.8.5
问题说明：集群环境运行一直很好，但是上周三台机器同时崩溃，均报以下错误
" /usr/local/mysql/bin/mysqld': malloc(): memory corruption: 0x00007ff660010d90 "
等到数据库重启后，三台机器一执行"start group_replication;"就crash
个人解决方案如下：
1. 观察当时binlog，并没有异常的SQL，当时的内存、磁盘、inode充足
2. 查看依赖环境，重新安装依赖环境，也不行
3. 重新安装数据库（rpm、源码、glibc）方式都用过，不好使
4. 替换为5.7.24版本也不好使日志比较长，放在回复区配置文件信息
[mysqld]
basedir=/usr/local/mysql
datadir=/var/lib/mysqlserver_id=1
port=3306
socket=/var/run/mysqld/mysql.sock
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-storage-engine=INNODBskip-name-resolve#Optimize
max_connections = 500
innodb_flush_log_at_trx_commit = 1
innodb_buffer_pool_size = 16G
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES,NO_ZERO_DATE,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO
explicit_defaults_for_timestamp=truelog-bin=binlog
log_bin_trust_function_creators=1
binlog_format=ROW
expire_logs_days=99
relay-log=s3-relay-binsync_binlog=0
slow-query-log=1
slow-query-log-file=/var/log/mysql/slow-queries.log
long_query_time=0.5
#log-queries-not-using-indexes
log-error=/var/log/mysql/mysql.err
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery = ON
binlog_checksum=NONE
log_slave_updates=ONplugin-load=group_replication.so
transaction_write_set_extraction = XXHASH64
loose-group_replication_group_name = 33847171-5f22-11e7-b03b-080027d9d784loose-group_replication_start_on_boot = off
loose-group_replication_local_address = 192.168.96.3:13306
loose-group_replication_group_seeds = 192.168.96.3:13306,192.168.96.4:13306,192.168.96.5:13306
loose-group_replication_bootstrap_group= off
loose-group_replication_single_primary_mode=TRUE
loose-group_replication_enforce_update_everywhere_checks=FALSEgroup_replication_allow_local_disjoint_gtids_join=ON
group-replication-auto-increment-increment=3
[client]
default-character-set=utf8mb4
port=3306
socket = /var/run/mysqld/mysql.sock
错误复现,以下数据库为刚刚初始化后为做任何操作的数据库mysql -p123456
Logging to file '/usr/local/mysql/mysql_client.log'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 253
Server version: 5.7.24-log MySQL Community Server (GPL)Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trade of Oracle Corporation and/or its
affiliates. Other names may be trades of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> start group_replication;
ERROR 2013 (HY000): Lost connection to MySQL server during query
问题说明：集群环境运行一直很好，但是上周三台机器同时崩溃，均报以下错误
" /usr/local/mysql/bin/mysqld': malloc(): memory corruption: 0x00007ff660010d90 "
等到数据库重启后，三台机器一执行"start group_replication;"就crash
个人解决方案如下：
1. 观察当时binlog，并没有异常的SQL，当时的内存、磁盘、inode充足
2. 查看依赖环境，重新安装依赖环境，也不行
3. 重新安装数据库（rpm、源码、glibc）方式都用过，不好使
4. 替换为5.7.24版本也不好使日志比较长，放在回复区配置文件信息
[mysqld]
basedir=/usr/local/mysql
datadir=/var/lib/mysqlserver_id=1
port=3306
socket=/var/run/mysqld/mysql.sock
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-storage-engine=INNODBskip-name-resolve#Optimize
max_connections = 500
innodb_flush_log_at_trx_commit = 1
innodb_buffer_pool_size = 16G
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES,NO_ZERO_DATE,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO
explicit_defaults_for_timestamp=truelog-bin=binlog
log_bin_trust_function_creators=1
binlog_format=ROW
expire_logs_days=99
relay-log=s3-relay-binsync_binlog=0
slow-query-log=1
slow-query-log-file=/var/log/mysql/slow-queries.log
long_query_time=0.5
#log-queries-not-using-indexes
log-error=/var/log/mysql/mysql.err
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery = ON
binlog_checksum=NONE
log_slave_updates=ONplugin-load=group_replication.so
transaction_write_set_extraction = XXHASH64
loose-group_replication_group_name = 33847171-5f22-11e7-b03b-080027d9d784loose-group_replication_start_on_boot = off
loose-group_replication_local_address = 192.168.96.3:13306
loose-group_replication_group_seeds = 192.168.96.3:13306,192.168.96.4:13306,192.168.96.5:13306
loose-group_replication_bootstrap_group= off
loose-group_replication_single_primary_mode=TRUE
loose-group_replication_enforce_update_everywhere_checks=FALSEgroup_replication_allow_local_disjoint_gtids_join=ON
group-replication-auto-increment-increment=3
[client]
default-character-set=utf8mb4
port=3306
socket = /var/run/mysqld/mysql.sock
错误复现,以下数据库为刚刚初始化后为做任何操作的数据库mysql -p123456
Logging to file '/usr/local/mysql/mysql_client.log'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 253
Server version: 5.7.24-log MySQL Community Server (GPL)Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trade of Oracle Corporation and/or its
affiliates. Other names may be trades of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> start group_replication;
ERROR 2013 (HY000): Lost connection to MySQL server during query
======= Backtrace: =========
/lib64/libc.so.6(+0x82c96)[0x7ff6b8e0ac96]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x7ff6b8e0d1ac]
/lib64/libstdc++.so.6(_Znwm+0x1d)[0x7ff6b96cbecd]
/lib64/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x7ff6b972aa19]
/lib64/libstdc++.so.6(_ZNSs12_S_constructIPKcEEPcT_S3_RKSaIcESt20forward_iterator_tag+0x21)[0x7ff6b972c2a1]
/lib64/libstdc++.so.6(_ZNSsC2EPKcRKSaIcE+0x38)[0x7ff6b972c6d8]
/usr/local/mysql/lib/plugin/group_replication.so(_Z24get_ipv4_local_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x86b)[0x7ff67c37c3cb]
/usr/local/mysql/lib/plugin/group_replication.so(_Z32get_ipv4_local_private_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x5c)[0x7ff67c37e27c]
/usr/local/mysql/lib/plugin/group_replication.so(_Z21fix_parameters_syntaxR24Gcs_interface_parameters+0x565)[0x7ff67c3837c5]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN18Gcs_xcom_interface10initializeERK24Gcs_interface_parameters+0x270)[0x7ff67c391960]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN14Gcs_operations9configureERK24Gcs_interface_parameters+0x98)[0x7ff67c3a9e48]
/usr/local/mysql/lib/plugin/group_replication.so(_Z29configure_group_communicationP23st_server_ssl_variables+0xb3a)[0x7ff67c3b942a]
/usr/local/mysql/lib/plugin/group_replication.so(_Z26initialize_plugin_and_join25enum_plugin_con_isolationP29Delayed_initialization_thread+0x1f2)[0x7ff67c3ba702]
/usr/local/mysql/lib/plugin/group_replication.so(_Z30plugin_group_replication_startv+0x5a3)[0x7ff67c3bae43]
/usr/local/mysql/bin/mysqld(_Z23group_replication_startv+0x8f)[0xe8871f]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x2acc)[0xd1956c]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x40d)[0xd1bded]
/usr/local/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x11a5)[0xd1d015]
/usr/local/mysql/bin/mysqld(_Z10do_commandP3THD+0x194)[0xd1dec4]
/usr/local/mysql/bin/mysqld(handle_connection+0x29c)[0xdef52c]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x174)[0xf74ee4]
/lib64/libpthread.so.0(+0x7dd5)[0x7ff6ba3ccdd5]
/lib64/libc.so.6(clone+0x6d)[0x7ff6b8e85ead]
======= Memory map: ========
00400000-01aad000 r-xp 00000000 fd:00 1073789263 /usr/local/mysql/bin/mysqld
01cac000-01e4f000 rw-p 016ac000 fd:00 1073789263 /usr/local/mysql/bin/mysqld
01e4f000-01f0e000 rw-p 00000000 00:00 0
037f1000-03812000 rw-p 00000000 00:00 0 [heap]
03812000-055f7000 rw-p 00000000 00:00 0 [heap]
7ff654000000-7ff654021000 rw-p 00000000 00:00 0
7ff654021000-7ff658000000 ---p 00000000 00:00 0
7ff65c000000-7ff65c021000 rw-p 00000000 00:00 0
7ff65c021000-7ff660000000 ---p 00000000 00:00 0
7ff660000000-7ff660021000 rw-p 00000000 00:00 0
7ff660021000-7ff664000000 ---p 00000000 00:00 0
7ff664000000-7ff664021000 rw-p 00000000 00:00 0
7ff664021000-7ff668000000 ---p 00000000 00:00 0
7ff668000000-7ff668021000 rw-p 00000000 00:00 0
7ff668021000-7ff66c000000 ---p 00000000 00:00 0
7ff66f7ff000-7ff66f800000 ---p 00000000 00:00 0
7ff66f800000-7ff670000000 rw-p 00000000 00:00 0
7ff670000000-7ff670024000 rw-p 00000000 00:00 0
7ff670024000-7ff674000000 ---p 00000000 00:00 0
7ff674000000-7ff674021000 rw-p 00000000 00:00 0
7ff674021000-7ff678000000 ---p 00000000 00:00 0
7ff678000000-7ff678024000 rw-p 00000000 00:00 0
7ff678024000-7ff67c000000 ---p 00000000 00:00 0
7ff67c2c9000-7ff67c452000 r-xp 00000000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c452000-7ff67c652000 ---p 00189000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c652000-7ff67c660000 rw-p 00189000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c660000-7ff67e7fd000 rw-p 00000000 00:00 0
7ff67e7fd000-7ff67e7fe000 ---p 00000000 00:00 0
7ff67e7fe000-7ff67effe000 rw-p 00000000 00:00 0
7ff67effe000-7ff67efff000 ---p 00000000 00:00 0
7ff67efff000-7ff67f7ff000 rw-p 00000000 00:00 0
7ff67f7ff000-7ff67f800000 ---p 00000000 00:00 0
7ff67f800000-7ff680000000 rw-p 00000000 00:00 0
7ff680000000-7ff680024000 rw-p 00000000 00:00 0
7ff680024000-7ff684000000 ---p 00000000 00:00 0
7ff684000000-7ff684021000 rw-p 00000000 00:00 0
7ff684021000-7ff688000000 ---p 00000000 00:00 0
7ff6887f9000-7ff6887fa000 ---p 00000000 00:00 0
7ff6887fa000-7ff688ffa000 rw-p 00000000 00:00 0
7ff688ffa000-7ff688ffb000 ---p 00000000 00:00 0
7ff688ffb000-7ff6897fb000 rw-p 00000000 00:00 0
7ff6897fb000-7ff6897fc000 ---p 00000000 00:00 0
7ff6897fc000-7ff689ffc000 rw-p 00000000 00:00 0
7ff689ffc000-7ff689ffd000 ---p 00000000 00:00 0
7ff689ffd000-7ff68a7fd000 rw-p 00000000 00:00 0
7ff68a7fd000-7ff68a7fe000 ---p 00000000 00:00 0
7ff68a7fe000-7ff68affe000 rw-p 00000000 00:00 0
7ff68affe000-7ff68afff000 ---p 00000000 00:00 0
7ff68afff000-7ff68b7ff000 rw-p 00000000 00:00 0
7ff68b7ff000-7ff68b800000 ---p 00000000 00:00 0
7ff68b800000-7ff68c000000 rw-p 00000000 00:00 0
7ff68c000000-7ff68c021000 rw-p 00000000 00:00 0
7ff68c021000-7ff690000000 ---p 00000000 00:00 0
7ff690278000-7ff690279000 ---p 00000000 00:00 0
7ff690279000-7ff6902ba000 rw-p 00000000 00:00 0
7ff6902ba000-7ff6902bb000 ---p 00000000 00:00 0
7ff6902bb000-7ff6902fc000 rw-p 00000000 00:00 0
7ff6902fc000-7ff6902fd000 ---p 00000000 00:00 0
7ff6902fd000-7ff6905ff000 rw-p 00000000 00:00 0
7ff6905ff000-7ff690600000 ---p 00000000 00:00 0
7ff690600000-7ff690e00000 rw-p 00000000 00:00 0
7ff690e00000-7ff690e01000 ---p 00000000 00:00 0
7ff690e01000-7ff691827000 rw-p 00000000 00:00 0
7ff691827000-7ff691828000 ---p 00000000 00:00 0
7ff691828000-7ff692028000 rw-p 00000000 00:00 0
7ff692028000-7ff692029000 ---p 00000000 00:00 0
7ff692029000-7ff692829000 rw-p 00000000 00:00 0
7ff692829000-7ff69282a000 ---p 00000000 00:00 0
7ff69282a000-7ff69302a000 rw-p 00000000 00:00 0
7ff69302a000-7ff69302b000 ---p 00000000 00:00 0
7ff69302b000-7ff69382b000 rw-p 00000000 00:00 0
7ff69382b000-7ff69382c000 ---p 00000000 00:00 0
7ff69382c000-7ff69402c000 rw-p 00000000 00:00 0
7ff69402c000-7ff69402d000 ---p 00000000 00:00 0
7ff69402d000-7ff69482d000 rw-p 00000000 00:00 0
7ff69482d000-7ff69482e000 ---p 00000000 00:00 0
7ff69482e000-7ff69502e000 rw-p 00000000 00:00 0
7ff69502e000-7ff69502f000 ---p 00000000 00:00 0
7ff69502f000-7ff69582f000 rw-p 00000000 00:00 0
7ff69582f000-7ff695830000 ---p 00000000 00:00 0
7ff695830000-7ff696030000 rw-p 00000000 00:00 0
7ff696030000-7ff696031000 ---p 00000000 00:00 0
7ff696031000-7ff696831000 rw-p 00000000 00:00 0
7ff696831000-7ff696832000 ---p 00000000 00:00 0
7ff696832000-7ff697184000 rw-p 00000000 00:00 0
7ff6971a1000-7ff6b08cc000 rw-p 00000000 00:00 0
7ff6b08cc000-7ff6b08cd000 ---p 00000000 00:00 0
7ff6b08cd000-7ff6b8b85000 rw-p 00000000 00:00 0
7ff6b8b85000-7ff6b8b87000 r-xp 00000000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8b87000-7ff6b8d86000 ---p 00002000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d86000-7ff6b8d87000 r--p 00001000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d87000-7ff6b8d88000 rw-p 00002000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d88000-7ff6b8f4a000 r-xp 00000000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b8f4a000-7ff6b914a000 ---p 001c2000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b914a000-7ff6b914e000 r--p 001c2000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b914e000-7ff6b9150000 rw-p 001c6000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b9150000-7ff6b9155000 rw-p 00000000 00:00 0
7ff6b9155000-7ff6b916a000 r-xp 00000000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b916a000-7ff6b9369000 ---p 00015000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b9369000-7ff6b936a000 r--p 00014000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b936a000-7ff6b936b000 rw-p 00015000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b936b000-7ff6b946c000 r-xp 00000000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b946c000-7ff6b966b000 ---p 00101000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966b000-7ff6b966c000 r--p 00100000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966c000-7ff6b966d000 rw-p 00101000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966d000-7ff6b9756000 r-xp 00000000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b9756000-7ff6b9955000 ---p 000e9000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b9955000-7ff6b995d000 r--p 000e8000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b995d000-7ff6b995f000 rw-p 000f0000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b995f000-7ff6b9974000 rw-p 00000000 00:00 0
7ff6b9974000-7ff6b997b000 r-xp 00000000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b997b000-7ff6b9b7a000 ---p 00007000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7a000-7ff6b9b7b000 r--p 00006000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7b000-7ff6b9b7c000 rw-p 00007000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7c000-7ff6b9b7e000 r-xp 00000000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9b7e000-7ff6b9d7e000 ---p 00002000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d7e000-7ff6b9d7f000 r--p 00002000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d7f000-7ff6b9d80000 rw-p 00003000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d80000-7ff6b9d88000 r-xp 00000000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
======= Backtrace: =========
/lib64/libc.so.6(+0x82c96)[0x7ff6b8e0ac96]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x7ff6b8e0d1ac]
/lib64/libstdc++.so.6(_Znwm+0x1d)[0x7ff6b96cbecd]
/lib64/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x7ff6b972aa19]
/lib64/libstdc++.so.6(_ZNSs12_S_constructIPKcEEPcT_S3_RKSaIcESt20forward_iterator_tag+0x21)[0x7ff6b972c2a1]
/lib64/libstdc++.so.6(_ZNSsC2EPKcRKSaIcE+0x38)[0x7ff6b972c6d8]
/usr/local/mysql/lib/plugin/group_replication.so(_Z24get_ipv4_local_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x86b)[0x7ff67c37c3cb]
/usr/local/mysql/lib/plugin/group_replication.so(_Z32get_ipv4_local_private_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x5c)[0x7ff67c37e27c]
/usr/local/mysql/lib/plugin/group_replication.so(_Z21fix_parameters_syntaxR24Gcs_interface_parameters+0x565)[0x7ff67c3837c5]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN18Gcs_xcom_interface10initializeERK24Gcs_interface_parameters+0x270)[0x7ff67c391960]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN14Gcs_operations9configureERK24Gcs_interface_parameters+0x98)[0x7ff67c3a9e48]
/usr/local/mysql/lib/plugin/group_replication.so(_Z29configure_group_communicationP23st_server_ssl_variables+0xb3a)[0x7ff67c3b942a]
/usr/local/mysql/lib/plugin/group_replication.so(_Z26initialize_plugin_and_join25enum_plugin_con_isolationP29Delayed_initialization_thread+0x1f2)[0x7ff67c3ba702]
/usr/local/mysql/lib/plugin/group_replication.so(_Z30plugin_group_replication_startv+0x5a3)[0x7ff67c3bae43]
/usr/local/mysql/bin/mysqld(_Z23group_replication_startv+0x8f)[0xe8871f]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x2acc)[0xd1956c]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x40d)[0xd1bded]
/usr/local/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x11a5)[0xd1d015]
/usr/local/mysql/bin/mysqld(_Z10do_commandP3THD+0x194)[0xd1dec4]
/usr/local/mysql/bin/mysqld(handle_connection+0x29c)[0xdef52c]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x174)[0xf74ee4]
/lib64/libpthread.so.0(+0x7dd5)[0x7ff6ba3ccdd5]
/lib64/libc.so.6(clone+0x6d)[0x7ff6b8e85ead]
======= Memory map: ========
00400000-01aad000 r-xp 00000000 fd:00 1073789263 /usr/local/mysql/bin/mysqld
01cac000-01e4f000 rw-p 016ac000 fd:00 1073789263 /usr/local/mysql/bin/mysqld
01e4f000-01f0e000 rw-p 00000000 00:00 0
037f1000-03812000 rw-p 00000000 00:00 0 [heap]
03812000-055f7000 rw-p 00000000 00:00 0 [heap]
7ff654000000-7ff654021000 rw-p 00000000 00:00 0
7ff654021000-7ff658000000 ---p 00000000 00:00 0
7ff65c000000-7ff65c021000 rw-p 00000000 00:00 0
7ff65c021000-7ff660000000 ---p 00000000 00:00 0
7ff660000000-7ff660021000 rw-p 00000000 00:00 0
7ff660021000-7ff664000000 ---p 00000000 00:00 0
7ff664000000-7ff664021000 rw-p 00000000 00:00 0
7ff664021000-7ff668000000 ---p 00000000 00:00 0
7ff668000000-7ff668021000 rw-p 00000000 00:00 0
7ff668021000-7ff66c000000 ---p 00000000 00:00 0
7ff66f7ff000-7ff66f800000 ---p 00000000 00:00 0
7ff66f800000-7ff670000000 rw-p 00000000 00:00 0
7ff670000000-7ff670024000 rw-p 00000000 00:00 0
7ff670024000-7ff674000000 ---p 00000000 00:00 0
7ff674000000-7ff674021000 rw-p 00000000 00:00 0
7ff674021000-7ff678000000 ---p 00000000 00:00 0
7ff678000000-7ff678024000 rw-p 00000000 00:00 0
7ff678024000-7ff67c000000 ---p 00000000 00:00 0
7ff67c2c9000-7ff67c452000 r-xp 00000000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c452000-7ff67c652000 ---p 00189000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c652000-7ff67c660000 rw-p 00189000 fd:00 3222200603 /usr/local/mysql/lib/plugin/group_replication.so
7ff67c660000-7ff67e7fd000 rw-p 00000000 00:00 0
7ff67e7fd000-7ff67e7fe000 ---p 00000000 00:00 0
7ff67e7fe000-7ff67effe000 rw-p 00000000 00:00 0
7ff67effe000-7ff67efff000 ---p 00000000 00:00 0
7ff67efff000-7ff67f7ff000 rw-p 00000000 00:00 0
7ff67f7ff000-7ff67f800000 ---p 00000000 00:00 0
7ff67f800000-7ff680000000 rw-p 00000000 00:00 0
7ff680000000-7ff680024000 rw-p 00000000 00:00 0
7ff680024000-7ff684000000 ---p 00000000 00:00 0
7ff684000000-7ff684021000 rw-p 00000000 00:00 0
7ff684021000-7ff688000000 ---p 00000000 00:00 0
7ff6887f9000-7ff6887fa000 ---p 00000000 00:00 0
7ff6887fa000-7ff688ffa000 rw-p 00000000 00:00 0
7ff688ffa000-7ff688ffb000 ---p 00000000 00:00 0
7ff688ffb000-7ff6897fb000 rw-p 00000000 00:00 0
7ff6897fb000-7ff6897fc000 ---p 00000000 00:00 0
7ff6897fc000-7ff689ffc000 rw-p 00000000 00:00 0
7ff689ffc000-7ff689ffd000 ---p 00000000 00:00 0
7ff689ffd000-7ff68a7fd000 rw-p 00000000 00:00 0
7ff68a7fd000-7ff68a7fe000 ---p 00000000 00:00 0
7ff68a7fe000-7ff68affe000 rw-p 00000000 00:00 0
7ff68affe000-7ff68afff000 ---p 00000000 00:00 0
7ff68afff000-7ff68b7ff000 rw-p 00000000 00:00 0
7ff68b7ff000-7ff68b800000 ---p 00000000 00:00 0
7ff68b800000-7ff68c000000 rw-p 00000000 00:00 0
7ff68c000000-7ff68c021000 rw-p 00000000 00:00 0
7ff68c021000-7ff690000000 ---p 00000000 00:00 0
7ff690278000-7ff690279000 ---p 00000000 00:00 0
7ff690279000-7ff6902ba000 rw-p 00000000 00:00 0
7ff6902ba000-7ff6902bb000 ---p 00000000 00:00 0
7ff6902bb000-7ff6902fc000 rw-p 00000000 00:00 0
7ff6902fc000-7ff6902fd000 ---p 00000000 00:00 0
7ff6902fd000-7ff6905ff000 rw-p 00000000 00:00 0
7ff6905ff000-7ff690600000 ---p 00000000 00:00 0
7ff690600000-7ff690e00000 rw-p 00000000 00:00 0
7ff690e00000-7ff690e01000 ---p 00000000 00:00 0
7ff690e01000-7ff691827000 rw-p 00000000 00:00 0
7ff691827000-7ff691828000 ---p 00000000 00:00 0
7ff691828000-7ff692028000 rw-p 00000000 00:00 0
7ff692028000-7ff692029000 ---p 00000000 00:00 0
7ff692029000-7ff692829000 rw-p 00000000 00:00 0
7ff692829000-7ff69282a000 ---p 00000000 00:00 0
7ff69282a000-7ff69302a000 rw-p 00000000 00:00 0
7ff69302a000-7ff69302b000 ---p 00000000 00:00 0
7ff69302b000-7ff69382b000 rw-p 00000000 00:00 0
7ff69382b000-7ff69382c000 ---p 00000000 00:00 0
7ff69382c000-7ff69402c000 rw-p 00000000 00:00 0
7ff69402c000-7ff69402d000 ---p 00000000 00:00 0
7ff69402d000-7ff69482d000 rw-p 00000000 00:00 0
7ff69482d000-7ff69482e000 ---p 00000000 00:00 0
7ff69482e000-7ff69502e000 rw-p 00000000 00:00 0
7ff69502e000-7ff69502f000 ---p 00000000 00:00 0
7ff69502f000-7ff69582f000 rw-p 00000000 00:00 0
7ff69582f000-7ff695830000 ---p 00000000 00:00 0
7ff695830000-7ff696030000 rw-p 00000000 00:00 0
7ff696030000-7ff696031000 ---p 00000000 00:00 0
7ff696031000-7ff696831000 rw-p 00000000 00:00 0
7ff696831000-7ff696832000 ---p 00000000 00:00 0
7ff696832000-7ff697184000 rw-p 00000000 00:00 0
7ff6971a1000-7ff6b08cc000 rw-p 00000000 00:00 0
7ff6b08cc000-7ff6b08cd000 ---p 00000000 00:00 0
7ff6b08cd000-7ff6b8b85000 rw-p 00000000 00:00 0
7ff6b8b85000-7ff6b8b87000 r-xp 00000000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8b87000-7ff6b8d86000 ---p 00002000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d86000-7ff6b8d87000 r--p 00001000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d87000-7ff6b8d88000 rw-p 00002000 fd:00 31759 /usr/lib64/libfreebl3.so
7ff6b8d88000-7ff6b8f4a000 r-xp 00000000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b8f4a000-7ff6b914a000 ---p 001c2000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b914a000-7ff6b914e000 r--p 001c2000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b914e000-7ff6b9150000 rw-p 001c6000 fd:00 31765 /usr/lib64/libc-2.17.so
7ff6b9150000-7ff6b9155000 rw-p 00000000 00:00 0
7ff6b9155000-7ff6b916a000 r-xp 00000000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b916a000-7ff6b9369000 ---p 00015000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b9369000-7ff6b936a000 r--p 00014000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b936a000-7ff6b936b000 rw-p 00015000 fd:00 1801938 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7ff6b936b000-7ff6b946c000 r-xp 00000000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b946c000-7ff6b966b000 ---p 00101000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966b000-7ff6b966c000 r--p 00100000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966c000-7ff6b966d000 rw-p 00101000 fd:00 1400359 /usr/lib64/libm-2.17.so
7ff6b966d000-7ff6b9756000 r-xp 00000000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b9756000-7ff6b9955000 ---p 000e9000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b9955000-7ff6b995d000 r--p 000e8000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b995d000-7ff6b995f000 rw-p 000f0000 fd:00 85 /usr/lib64/libstdc++.so.6.0.19
7ff6b995f000-7ff6b9974000 rw-p 00000000 00:00 0
7ff6b9974000-7ff6b997b000 r-xp 00000000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b997b000-7ff6b9b7a000 ---p 00007000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7a000-7ff6b9b7b000 r--p 00006000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7b000-7ff6b9b7c000 rw-p 00007000 fd:00 58988 /usr/lib64/librt-2.17.so
7ff6b9b7c000-7ff6b9b7e000 r-xp 00000000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9b7e000-7ff6b9d7e000 ---p 00002000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d7e000-7ff6b9d7f000 r--p 00002000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d7f000-7ff6b9d80000 rw-p 00003000 fd:00 1387625 /usr/lib64/libdl-2.17.so
7ff6b9d80000-7ff6b9d88000 r-xp 00000000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
7ff6b9d80000-7ff6b9d88000 r-xp 00000000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
7ff6b9d88000-7ff6b9f87000 ---p 00008000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
7ff6b9f87000-7ff6b9f88000 r--p 00007000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
7ff6b9f88000-7ff6b9f89000 rw-p 00008000 fd:00 31769 /usr/lib64/libcrypt-2.17.so
7ff6b9f89000-7ff6b9fb7000 rw-p 00000000 00:00 0
7ff6b9fb7000-7ff6b9fc1000 r-xp 00000000 fd:00 167179 /usr/lib64/libnuma.so.1
7ff6b9fc1000-7ff6ba1c1000 ---p 0000a000 fd:00 167179 /usr/lib64/libnuma.so.1
7ff6ba1c1000-7ff6ba1c2000 r--p 0000a000 fd:00 167179 /usr/lib64/libnuma.so.1
7ff6ba1c2000-7ff6ba1c3000 rw-p 0000b000 fd:00 167179 /usr/lib64/libnuma.so.1
7ff6ba1c3000-7ff6ba1c4000 r-xp 00000000 fd:00 195911 /usr/lib64/libaio.so.1.0.1
7ff6ba1c4000-7ff6ba3c3000 ---p 00001000 fd:00 195911 /usr/lib64/libaio.so.1.0.1
7ff6ba3c3000-7ff6ba3c4000 r--p 00000000 fd:00 195911 /usr/lib64/libaio.so.1.0.1
7ff6ba3c4000-7ff6ba3c5000 rw-p 00001000 fd:00 195911 /usr/lib64/libaio.so.1.0.1
7ff6ba3c5000-7ff6ba3dc000 r-xp 00000000 fd:00 58984 /usr/lib64/libpthread-2.17.so
7ff6ba3dc000-7ff6ba5db000 ---p 00017000 fd:00 58984 /usr/lib64/libpthread-2.17.so
7ff6ba5db000-7ff6ba5dc000 r--p 00016000 fd:00 58984 /usr/lib64/libpthread-2.17.so
7ff6ba5dc000-7ff6ba5dd000 rw-p 00017000 fd:00 58984 /usr/lib64/libpthread-2.17.so
7ff6ba5dd000-7ff6ba5e1000 rw-p 00000000 00:00 0
7ff6ba5e1000-7ff6ba603000 r-xp 00000000 fd:00 1152710 /usr/lib64/ld-2.17.so
7ff6ba605000-7ff6ba607000 rw-p 00000000 00:00 0
7ff6ba607000-7ff6ba608000 rw-s 00000000 00:0b 680583 /[aio] (deleted)
7ff6ba608000-7ff6ba60b000 rw-s 00000000 00:0b 680582 /[aio] (deleted)
7ff6ba60b000-7ff6ba60e000 rw-s 00000000 00:0b 680581 /[aio] (deleted)
7ff6ba60e000-7ff6ba611000 rw-s 00000000 00:0b 680580 /[aio] (deleted)
7ff6ba611000-7ff6ba614000 rw-s 00000000 00:0b 680579 /[aio] (deleted)
7ff6ba614000-7ff6ba617000 rw-s 00000000 00:0b 680578 /[aio] (deleted)
7ff6ba617000-7ff6ba61a000 rw-s 00000000 00:0b 680577 /[aio] (deleted)
7ff6ba61a000-7ff6ba61d000 rw-s 00000000 00:0b 680576 /[aio] (deleted)
7ff6ba61d000-7ff6ba7ef000 rw-p 00000000 00:00 0
7ff6ba7ef000-7ff6ba7f7000 rw-p 00000000 00:00 0
7ff6ba7f7000-7ff6ba7fa000 rw-s 00000000 00:0b 680575 /[aio] (deleted)
7ff6ba7fa000-7ff6ba7fd000 rw-s 00000000 00:0b 680574 /[aio] (deleted)
7ff6ba7fd000-7ff6ba800000 rw-s 00000000 00:0b 680573 /[aio] (deleted)
7ff6ba800000-7ff6ba801000 rw-s 00000000 00:0b 680572 /[aio] (deleted)
7ff6ba801000-7ff6ba802000 rw-p 00000000 00:00 0
7ff6ba802000-7ff6ba803000 r--p 00021000 fd:00 1152710 /usr/lib64/ld-2.17.so
7ff6ba803000-7ff6ba804000 rw-p 00022000 fd:00 1152710 /usr/lib64/ld-2.17.so
7ff6ba804000-7ff6ba805000 rw-p 00000000 00:00 0
7ffe7c793000-7ffe7c7b4000 rw-p 00000000 00:00 0 [stack]
7ffe7c7f5000-7ffe7c7f7000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
05:55:54 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=2
max_threads=500
thread_count=2
connection_count=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 206879 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.Thread pointer: 0x7ff660000ae0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7ff6902fae68 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xf4e815]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x4a4)[0x7d1be4]
/lib64/libpthread.so.0(+0xf5d0)[0x7ff6ba3d45d0]
/lib64/libc.so.6(gsignal+0x37)[0x7ff6b8dbe207]
/lib64/libc.so.6(abort+0x148)[0x7ff6b8dbf8f8]
/lib64/libc.so.6(+0x78d27)[0x7ff6b8e00d27]
/lib64/libc.so.6(+0x82c96)[0x7ff6b8e0ac96]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x7ff6b8e0d1ac]
/lib64/libstdc++.so.6(_Znwm+0x1d)[0x7ff6b96cbecd]
/lib64/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x7ff6b972aa19]
/lib64/libstdc++.so.6(_ZNSs12_S_constructIPKcEEPcT_S3_RKSaIcESt20forward_iterator_tag+0x21)[0x7ff6b972c2a1]
/lib64/libstdc++.so.6(_ZNSsC2EPKcRKSaIcE+0x38)[0x7ff6b972c6d8]
/usr/local/mysql/lib/plugin/group_replication.so(_Z24get_ipv4_local_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x86b)[0x7ff67c37c3cb]
/usr/local/mysql/lib/plugin/group_replication.so(_Z32get_ipv4_local_private_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0x5c)[0x7ff67c37e27c]
/usr/local/mysql/lib/plugin/group_replication.so(_Z21fix_parameters_syntaxR24Gcs_interface_parameters+0x565)[0x7ff67c3837c5]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN18Gcs_xcom_interface10initializeERK24Gcs_interface_parameters+0x270)[0x7ff67c391960]
/usr/local/mysql/lib/plugin/group_replication.so(_ZN14Gcs_operations9configureERK24Gcs_interface_parameters+0x98)[0x7ff67c3a9e48]
/usr/local/mysql/lib/plugin/group_replication.so(_Z29configure_group_communicationP23st_server_ssl_variables+0xb3a)[0x7ff67c3b942a]
/usr/local/mysql/lib/plugin/group_replication.so(_Z26initialize_plugin_and_join25enum_plugin_con_isolationP29Delayed_initialization_thread+0x1f2)[0x7ff67c3ba702]
/usr/local/mysql/lib/plugin/group_replication.so(_Z30plugin_group_replication_startv+0x5a3)[0x7ff67c3bae43]
/usr/local/mysql/bin/mysqld(_Z23group_replication_startv+0x8f)[0xe8871f]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x2acc)[0xd1956c]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x40d)[0xd1bded]
/usr/local/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x11a5)[0xd1d015]
/usr/local/mysql/bin/mysqld(_Z10do_commandP3THD+0x194)[0xd1dec4]
/usr/local/mysql/bin/mysqld(handle_connection+0x29c)[0xdef52c]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x174)[0xf74ee4]
/lib64/libpthread.so.0(+0x7dd5)[0x7ff6ba3ccdd5]
/lib64/libc.so.6(clone+0x6d)[0x7ff6b8e85ead]Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7ff660005640): start group_replication
Connection ID (thread ID): 253
Status: NOT_KILLEDThe manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.