bluefs _flush_range allocated: osd用空间但是显示ceph_bluefs_db_used_bytes is 100%

news/2024/10/21 22:52:29/

1、查看日志

tail -f  /var/log/ceph/ceph-osd.2.log

   -6> 2024-05-23 09:29:20.734328 7f837d847d00  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1716452960734313, "job": 1, "event": "recovery_started", "log_files": [1064066, 1064069]}-5> 2024-05-23 09:29:20.734340 7f837d847d00  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:482] Recovering log #1064066 mode 0-4> 2024-05-23 09:29:52.503818 7f837d847d00  5 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:815] [default] [WriteLevel0TableForRecovery] Level-0 table #1064067: started-3> 2024-05-23 09:30:08.239889 7f837d847d00  1 bluefs _allocate failed to allocate 0x47d00000 on bdev 1, free 0x1b300000; fallback to bdev 2-2> 2024-05-23 09:30:08.239910 7f837d847d00 -1 bluefs _allocate failed to allocate 0x47d00000 on bdev 2, dne-1> 2024-05-23 09:30:08.239912 7f837d847d00 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x47c548200> 2024-05-23 09:30:08.246076 7f837d847d00 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f837d847d00 time 2024-05-23 09:30:08.239921
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueFS.cc: 1525: FAILED assert(0 == "bluefs enospc")ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f837e2f6850]2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1b1b) [0x7f837e27501b]3: (BlueFS::_flush(BlueFS::FileWriter*, bool)+0x188) [0x7f837e275588]4: (BlueRocksWritableFile::Flush()+0x3c) [0x7f837e29113c]5: (rocksdb::WritableFileWriter::Flush()+0xaf) [0x7f837e662a6f]6: (rocksdb::WritableFileWriter::Sync(bool)+0x33) [0x7f837e662c83]7: (rocksdb::BuildTable(std::string const&, rocksdb::Env*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::EnvOptions const&, rocksdb::TableCache*, rocksdb::InternalIterator*, std::unique_ptr<rocksdb::InternalIterator, std::default_delete<rocksdb::InternalIterator> >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::string const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::CompressionType, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int)+0x1004) [0x7f837e679d54]8: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0x90c) [0x7f837e5c621c]9: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool)+0x1430) [0x7f837e5c7e90]10: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool)+0x7e6) [0x7f837e5c92e6]11: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::string const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0xed3) [0x7f837e5ca5a3]12: (rocksdb::DB::Open(rocksdb::Options const&, std::string const&, rocksdb::DB**)+0x186) [0x7f837e5cb7e6]13: (RocksDBStore::do_open(std::ostream&, bool)+0x8e8) [0x7f837e212368]14: (BlueStore::_open_db(bool)+0xdb3) [0x7f837e19d313]15: (BlueStore::_mount(bool)+0x40e) [0x7f837e1cf06e]16: (OSD::init()+0x3bd) [0x7f837dd8434d]17: (main()+0x2d07) [0x7f837dc89947]18: (__libc_start_main()+0xf5) [0x7f837a0c6b35]19: (()+0x4b9003) [0x7f837dd28003]NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this

使用ceph-bluestore-tool
ceph-bluestore-tool 是一个对 BlueStore 实例执行低级管理操作的实用工具。

使用前，关闭对应 OSD 服务

[root@node-1 ceph-objectstore-tool-test]# systemctl stop ceph-osd@0

2. 示例

元数据、数据校验（–deep true，开启数据校验）
这实际是调用bluestore中的fsck()函数，有deep、repair等选项，支持不同程度的检查工作。具体工作有：更新kvdb中super前缀信息（ondisk_format、min_alloc_size等，具体可查阅BlueStore::_upgrade_super()函数）、重放deferred_transaction、删除失效blob、删除错误pextent、更新bluefs文件系统信息等。
所以fsck主要针对bluestore中的元数据信息的修复和检查。

ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-0/ fsck [--deep true][root@localhost build]# ./bin/ceph-bluestore-tool --log-level 10 --log-file /root/bluestore-tool.log --path /var/lib/ceph/osd/ceph-2/ fsck --deep true
fsck success

修复
调用bluestore中的fsck()函数实现功能。

ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-0/ [repair|quick-fix] [--deep true][root@localhost build]# ./bin/ceph-bluestore-tool --log-level 10 --log-file /root/bluestore-tool.log --path /var/lib/ceph/osd/ceph-2/ repair --deep true
repair success

http://www.ppmy.cn/news/1532760.html

相关文章

在系统开发中提升 Excel 数据导出一致性与可维护性的统一规范与最佳实践

在系统开发中提升 Excel 数据导出一致性与可维护性的统一规范与最佳实践

背景&#xff1a; 在系统开发过程中&#xff0c;数据导出为 Excel 格式是一个常见的需求。然而&#xff0c;由于各个开发人员的编码习惯和实现方式不同&#xff0c;导致导出代码风格不一。有的人使用第三方库&#xff0c;有的人则自定义实现。这种多样化不仅影响了代码的一致性…
阅读更多...
C++之 string（中）

C++之 string（中）

C之 string string类对象的容量操作 resize 将有效字符的个数该成n个&#xff0c;多出的空间用字符c填充 虽然在string里用的不多&#xff0c;但是在vector里面常见 这里有三种情况&#xff1a; 1&#xff09;resize小于当前的size 2)resize大于当前的size,小于capacity …
阅读更多...
XML 编码

XML 编码

XML 编码 XML&#xff08;可扩展标记语言&#xff09;是一种用于存储和传输数据的标记语言。它由万维网联盟&#xff08;W3C&#xff09;开发&#xff0c;旨在提供一种标准的方式来结构化、存储和传输数据。XML的设计目标是既易于人类阅读&#xff0c;也易于机器解析。 XML的…
阅读更多...
Java企业面试题5

Java企业面试题5

1. Java 的泛型是什么&#xff1f;有什么好处和优点&#xff1f;JDK 不同版本的泛型有什么区别&#xff1f;&#xff08;软*动力&#xff09; 泛型是 Java 语言中引入的一种特性&#xff0c;允许在定义类、接口和方法时使用类型参数&#xff08;Type Parameters&#xff09;。…
阅读更多...
策略（政策）模式

策略（政策）模式

简介 策略模式&#xff08;Strategy Pattern&#xff09;又叫作政策模式&#xff08;Policy Pattern&#xff09;&#xff0c;它将定义的算法家族分别封装起来&#xff0c;让它们之间可以互相替换&#xff0c;从而让算法的变化不会影响到使用算法的用户&#xff0c;属于行为型…
阅读更多...
22.4k star，好用、强大的链路监控软件，skywalking

22.4k star，好用、强大的链路监控软件，skywalking

skywalking其实用的比较少&#xff0c;但其实目前持续更新下&#xff0c;功能还是蛮强大的&#xff0c;目前支持的agent也是蛮多的&#xff0c;如下是一些常规的支持agent&#xff0c;如果在选择链路监控的话&#xff0c;可以看下这款。 当谈到应用性能监控&#xff08;Applica…
阅读更多...
Docker精讲：基本安装，简单命令及核心概念

Docker精讲：基本安装，简单命令及核心概念

docker服务部署 docker是一个容器管理工具&#xff0c;其内部容器才是具体服务&#xff0c;所以我们在安装docker时不需要有太多定制内容&#xff0c;只需要通过yum安装即可 1. 更新系统包 #更新现有依赖包&#xff0c;防止现有依赖包版本过低影响docker安装 yum update2. 安…
阅读更多...
docker简介、安装、基础知识

docker简介、安装、基础知识

基础知识 Docker简介&#xff1a; 1.Docker是一种用于构建、发布及运行应用程序的开源项目&#xff0c;他通过容器化技术简化了应用程序的部署和管理 2.Docker是一个开源的应用容器引擎&#xff0c;基于go语言开发&#xff0c;为应用打包、部署平台&#xff0c;而非单纯的虚…
阅读更多...
最新文章

Copyright @ 2022~2023