Уровень 0 · материалов: 7
В кластер входят документы, посвященные техническим особенностям, настройке и решению проблем работы файловых систем Linux при хранении данных на диске, и не входят документы о переносе всей ОС в оперативную память.
Общие признаки: файловые системы Linux, производительность ввода-вывода, работа с огромным количеством файлов, специфика Btrfs
Группа выше: Файловые системы
Смысл: The main idea is to share professional experience and technical 'hacks' to overcome the performance and structural limitations of Linux filesystems when dealing with an extreme quantity of small files.
A technical guide offering solutions for inode exhaustion, directory collisions, and I/O performance issues when managing millions of small files on Linux.
Смысл: The main idea is to provide a toolkit and methodology for Linux users to diagnose disk write overhead and to warn against using the BTRFS file system on flash-based storage when frequent small writes occur due to severe write amplification.
A technical guide on using tools like btrace, iotop, and fatrace to identify disk-heavy Linux processes, concluding that BTRFS causes excessive write amplification on SSDs/SD cards.
Смысл: The main idea is to demonstrate how to implement high-scale file synchronization for billions of files on AWS EC2 using Lsyncd and inotify, while highlighting the importance of kernel tuning and custom configuration when dealing with extreme scale.
The author achieves high-speed synchronization of billions of files across EC2 nodes by combining Lsyncd and inotify, while bypassing default startup syncs and tuning kernel limits.
Смысл: The main idea is to determine which Linux tools are most effective for deleting millions of files in a single directory without crashing the system or consuming all available memory. The author demonstrates that standard tools like 'rm -r' are efficient, while others that load the entire directory list into RAM (like 'find' or shell wildcards) are unsuitable for this scale.
Testing various Linux methods for deleting millions of files reveals that 'rm -r' is the most efficient practical tool, while 'find' and shell wildcards fail due to excessive memory consumption.
Смысл: The main idea is to demystify the BTRFS filesystem for Linux users by explaining its unique Copy-on-Write philosophy, providing a practical manual for its administrative tools, and warning about its specific technical quirks like fragmentation.
A detailed technical introduction to the BTRFS filesystem, covering its architecture, subvolumes, snapshots, CoW mechanism, and practical administration for Linux users.
Смысл: The main idea is that while CoW file systems (like Btrfs and XFS) offer immense performance and efficiency benefits through reflinks, these features are hindered by the default settings of standard Linux tools like `cp`, requiring manual intervention from users to be fully realized.
Standard Linux `cp` doesn't use Copy-on-Write (reflink) by default, sacrificing significant speed and disk space efficiency for legacy robustness reasons.
Смысл: The text explains how to resolve a specific server failure where MySQL reports 'No space left on device' despite available disk capacity, caused by an inode exhaustion due to millions of accumulated PHP session files. The author demonstrates why standard shell commands fail in this scenario and provides a custom C program using the 'getdents' system call to safely delete files in small batches.
The author solves a 'disk full' error caused by inode exhaustion from millions of PHP session files by writing a custom C program using 'getdents' to delete files incrementally.