Netflix Data Management & Platform Libraries Tools |Glossary Part 3

This is Part 3 of our series on Netflix open source tools. You can find Part 1 here and Part 2 here.

Netflix In-Memory Data Management Tools

Netflix-Graph

Netflix handles billions of daily requests that require access to video metadata. Given the huge number of stored videos, data is stored on the traffic-handling server’s RAM to reduce latency. Data is packaged and shipped using two applications:

• Video Metadata Platform (VMS) packages video’s synopses, titles, artwork and streams.

• Netflix-Graph stores data as a directed graph of the relationship between entities like video characters and tags. It stores any type of data that can be “phrased” as directed graph.

Zeno

Zeno is a highly optimized framework that helps manage and update the giga-bytes of constantly changing data that are replicated in thousands of servers. It provides:

• Duplicate dataset removal.

• Efficient data deserialization.

• Powerful debug tools.

• Improved development-team agility providing a clear line of separation between a data model and operations.

• The ability to maintain minimal changes required to keep data up to date.

Netflix Platform Libraries Tools

Archaius

Archaius is a configuration management library focused on dynamic properties from multiple configuration stores. It’s a Java library that provides APIs for dynamic access to properties. It provides:

• Dynamic properties of specific values.

• A polling framework for dynamic configuration sources.

• Ready-made configuration sources.

• Context management.

• High throughput and thread-safe configurations.

• Scala wrappers.

Denominator

Denominator is a portable Java library for processing DNS clouds. It has pluggable backends for:

• AWS Route 53

• Neustar Ultra

• DynECT

• Rackspace Cloud DNS

• Openstack Designate

Feign

Feign is a Java-to-HTTP client binder that allows binding with minimal code and overhead. It provides:

• Creation of customized components.

• Production of multiple interfaces.

• Quick integration with other open source projects.

Karyon

Karyon is a framework and library that serves as a blueprint for implementing cloud-ready web services and that can clone fine-grained web services and applications. Karyon’s blueprint includes:

• Bootstrapping, Libraries, and Lifecycle Management.

• Runtime Insights and Diagnostics.

• Pluggable Web Resources.

• Cloud readiness.

Karyon is based on two main entities:

• Component is a standalone entity that’s required for the application to work, but is not a direct dependency of the application.

• Application is a class inside the service. There can only be one application for each service and all components are initialized before the application is initialized.

Ribbon

Ribbon is a library that supports inter-process communication in the cloud. Features include:

• Pluggable load balancing rules.

• Service discovery integration.

• Built-in failure resiliency.

• Load balancer integration with clients.

It currently has 3 sub-projects:

• ribbon-core

• ribbon-eureka

• ribbon-httpclient

Servo

Servo is an application-monitoring library that generates monitors and metrics. A monitor has a configuration that is a set of key-value pairs. A metric is the value for a given monitor at a specific time.

There are three types of monitors:

• Gauges are simple monitors that return current values, such as queue size and thread number. Servo gauge implementations include BasicGauge, MinGauge, MaxGauge and NumberGauge.

• Counters come in three flavors: Basic, Resettable and PeakRate.

• Timers indicate time taken for a particular event..

Blitz4j

Blitz4j is a logging framework built on top of log4j. It allows high-scalable logging while maintaining high performance. Features include:

• Event monitoring.

• Business intelligence reporting.

• Debugging.

Blitz4j mitigates data loss and provides a mechanism to summarize data during log storms.

Governator

Governator is a library of extensions and utilities that extend Google Guice capabilities. Guice is a dependency injection container open sourced by Google.