Netflix In-Memory Data Management Tools
Netflix handles billions of daily requests that require access to video metadata. Given the huge number of stored videos, data is stored on the traffic-handling server’s RAM to reduce latency. Data is packaged and shipped using two applications:
Video Metadata Platform (VMS) packages video’s synopses, titles, artwork and streams.
Netflix-Graph stores data as a directed graph of the relationship between entities like video characters and tags. It stores any type of data that can be “phrased” as directed graph.
Zeno is a highly optimized framework that helps manage and update the giga-bytes of constantly changing data that are replicated in thousands of servers. It provides:
Duplicate dataset removal.
Efficient data deserialization.
Powerful debug tools.
Improved development-team agility providing a clear line of separation between a data model and operations.
The ability to maintain minimal changes required to keep data up to date.
Netflix Platform Libraries Tools
Archaius is a configuration management library focused on dynamic properties from multiple configuration stores. It’s a Java library that provides APIs for dynamic access to properties. It provides:
Dynamic properties of specific values.
A polling framework for dynamic configuration sources.
Ready-made configuration sources.
High throughput and thread-safe configurations.
Denominator is a portable Java library for processing DNS clouds. It has pluggable backends for:
AWS Route 53
Rackspace Cloud DNS
Feign is a Java-to-HTTP client binder that allows binding with minimal code and overhead. It provides:
Creation of customized components.
Production of multiple interfaces.
Quick integration with other open source projects.
Karyon is a framework and library that serves as a blueprint for implementing cloud-ready web services and that can clone fine-grained web services and applications. Karyon’s blueprint includes:
Bootstrapping, Libraries, and Lifecycle Management.
Runtime Insights and Diagnostics.
Pluggable Web Resources.
Karyon is based on two main entities:
Component is a standalone entity that’s required for the application to work, but is not a direct dependency of the application.
Application is a class inside the service. There can only be one application for each service and all components are initialized before the application is initialized.
Ribbon is a library that supports inter-process communication in the cloud. Features include:
Pluggable load balancing rules.
Service discovery integration.
Built-in failure resiliency.
Load balancer integration with clients.
It currently has 3 sub-projects:
Servo is an application-monitoring library that generates monitors and metrics. A monitor has a configuration that is a set of key-value pairs. A metric is the value for a given monitor at a specific time.
There are three types of monitors:
Gauges are simple monitors that return current values, such as queue size and thread number. Servo gauge implementations include BasicGauge, MinGauge, MaxGauge and NumberGauge.
Counters come in three flavors: Basic, Resettable and PeakRate.
Timers indicate time taken for a particular event..
Blitz4j is a logging framework built on top of log4j. It allows high-scalable logging while maintaining high performance. Features include:
Business intelligence reporting.
Blitz4j mitigates data loss and provides a mechanism to summarize data during log storms.
Governator is a library of extensions and utilities that extend Google Guice capabilities. Guice is a dependency injection container open sourced by Google.