System Overview

• Component
  • client
    • exposing a near-POSIX file system interface to a host or process
  • a cluster of OSDs
    • collectively stores all data and metadata
  • a metadata server cluster
    • manages the namespace (file names and directories) while coordinating security, consistency and coherence
• Goal
  • Scalability(to hundreds of petabytes and beyond)
    • overall storage capacity and throughput of the system, and performance in terms of individual clients, directories, or files
    • Target Workload
      • scientific applications ~ tens or hundreds of thousands of hosts concurrently reading from or writing to the same file or creating files in the same directory.
  • Performance, Reliability

• Three fundamental design features
  • decoupled data and metadata
    • 아 이 의미가 metadata operation은 무조건 사용자 입장에서 필요한 건데 data operation을 하는 경우에도 기존에는 metadata operation을 무조건 필요로 하는 구조이다 보니까 이걸 data operation엔 metadata operation 필요 없이 data operation만 해도 되게했다는 측면에서 decoupled, separation이라는 표현을 쓴 듯
    • in contrast to existing object-based file systems [4,7,8,32] which replace long per-file block lists with shorter object lists, Ceph eliminates allocation lists entirely.
      • Instead, file data is striped onto predictably named objects, while a special-purpose data distribution function called CRUSH [29] assigns objects to storage devices.
      • This allows any party to calculate (rather than look up) the name and location of objects comprising a file's contents, eliminating the need to maintain and distribute object lists, simplifying the design of the system, and reducing the metadata cluster workload.
  • dynamic distributed metadata management
    • Ceph utilizes a novel metadata cluster architecture based on Dynamic Subtree Partitioning [30] that adaptively and intelligently distributes responsibility for managing the file system directory hierarchy among tens or even hundreds of MDSs.
      • 쪼개나봄 dynamic하게 여러 MDS에
    • A (dynamic) hierarchical partition preserves locality in each MDS's workload, facilitating efficient updates and aggressive prefetching to improve performance for common workloads.
    • Significantly, the workload distribution among metadata servers is based entirely on current access patterns, allowing Ceph to effectively utilize available MDS resources under any workload and achieve near-linear scaling in the number of MDSs.
  • reliable autonomic distributed object storage
    • distribution of data evolve to effectively utilize available resources and maintain the desired level of data replication
    • Ceph delegates responsibility for data migration, replication, failure detection, and failure recovery to the cluster of OSDs that store the data, while at a high level, OSDs collectively provide a single logical object store to clients and metadata servers.
    • This approach allows Ceph to more effectively leverage the intelligence (CPU and memory) present on each OSD to achieve reliable, highly available object storage with linear scaling.

t