This week in our reading group at LinkedIn we read and discussed this paper: Finding a needle in Haystack: Facebook's photo storage. As the name suggests the Haystack system is designed and built by Facebook guys to store and serve photos. The basic goals of Haystack system are as follows: first, high throughput and low latency, store all metadata in main memory, and save on multiple disk I/Os. They are able to achieve 4x speedup than the earlier system by doing this. Second, fault-tolerance. Replicate photos across multiple data centers. Third, cost-effectiveness. They claim to be 28% cheaper than the earlier system.In the earlier NAS (Network Attached Storage) photo storage system, the main bottleneck was 3 disk accesses for fetching each photo: one for fetching the location of corresponding inode, second for reading inode, and third for fetching the file data.The main idea of the Haystack system is the design of Haystack store that lead to fetching of a photo in a single disk access. In the Haystack store, they keep multiple photos in a single large file. They need only the offset of the photo in the file and size of the photo to access a photo. This way they are able to reduce the amount of metadata needed to access a photo, and keep all metadata for photos in memory. Hence, no disk access is needed to fetch the metadata for a photo and the photo can be fetched from the disk in a single disk access.Overall, impressed by the scale of Haystack photo storage system that stores more than 20 Petabytes in form of 260 billions of photos and serves millions of photos per second at the peak.Reference:
- D. Beaver, S. Kumar, H. C. Li, J. Sobel, and P. Vajgel. Finding a needle in Haystack: Facebook’s photo storage. In OSDI ’10
