Flux7 Blog
Netflix Data Management & Platform Libraries Tools |Glossary Part 3
This is Part 3 of our series on Netflix open-source tools. You can find Part 1 here and Part 2 here.
Netflix In-Memory Data Management Tools
Netflix-Graph
Netflix handles billions of daily requests that require access to video metadata. Given the huge number of stored videos, data is stored on the traffic-handling server’s RAM to reduce latency. Data is packaged and shipped using two applications:
-
Video Metadata Platform (VMS) packages video’s synopses, titles, artwork, and streams.
-
Netflix-Graph stores data as a directed graph of the relationship between entities like video characters and tags. It stores any type of data that can be “phrased” as directed graph.
Zeno
Zeno is a highly optimized framework that helps manage and update the giga-bytes of constantly changing data that are replicated in thousands of servers. It provides:
-
Duplicate dataset removal.
-
Efficient data deserialization.
-
Powerful debug tools.
-
Improved development-team agility providing a clear line of separation between a data model and operations.
-
The ability to maintain minimal changes required to keep data up to date.
Netflix Platform Libraries Tools
Archaius
Archaius is a configuration management library focused on dynamic properties from multiple configuration stores. It’s a Java library that provides APIs for dynamic access to properties. It provides:
-
Dynamic properties of specific values.
-
A polling framework for dynamic configuration sources.
-
Ready-made configuration sources.
-
Context management.
-
High throughput and thread-safe configurations.
-
Scala wrappers.
Denominator
Denominator is a portable Java library for processing DNS clouds. It has pluggable backends for:
-
AWS Route 53
-
Neustar Ultra
-
DynECT
-
Rackspace Cloud DNS
-
Openstack Designate
Feign
Feign is a Java-to-HTTP client binder that allows binding with minimal code and overhead. It provides:
-
Creation of customized components.
-
Production of multiple interfaces.
-
Quick integration with other open-source projects.
Karyon
Karyon is a framework and library that serves as a blueprint for implementing cloud-ready web services and that can clone fine-grained web services and applications. Karyon’s blueprint includes:
-
Bootstrapping, Libraries, and Lifecycle Management.
-
Runtime Insights and Diagnostics.
-
Pluggable Web Resources.
-
Cloud readiness.
Karyon is based on two main entities:
-
Component is a standalone entity that’s required for the application to work, but is not a direct dependency of the application.
-
Application is a class inside the service. There can only be one application for each service and all components are initialized before the application is initialized.
Ribbon
Ribbon is a library that supports inter-process communication in the cloud. Features include:
-
Pluggable load balancing rules.
-
Service discovery integration.
-
Built-in failure resiliency.
-
Load balancer integration with clients.
It currently has 3 sub-projects:
-
ribbon-core
-
ribbon-eureka
-
ribbon-httpclient
Servo
Servo is an application-monitoring library that generates monitors and metrics. A monitor has a configuration that is a set of key-value pairs. A metric is the value for a given monitor at a specific time.
There are three types of monitors:
-
Gauges are simple monitors that return current values, such as queue size and thread number. Servo gauge implementations include BasicGauge, MinGauge, MaxGauge and NumberGauge.
-
Counters come in three flavors: Basic, Resettable and PeakRate.
-
Timers indicate time taken for a particular event..
Blitz4j
Blitz4j is a logging framework built on top of log4j. It allows high-scalable logging while maintaining high performance. Features include:
-
Event monitoring.
-
Business intelligence reporting.
-
Debugging.
Blitz4j mitigates data loss and provides a mechanism to summarize data during log storms.
Governator
Governator is a library of extensions and utilities that extend Google Guice capabilities. Guice is a dependency injection container open-sourced by Google.
Related Articles
IT Modernization and DevOps News Week in Review 1.18.2021
re:Invent Round-Up of AWS DevOps Announcements
Recent Posts
IT Modernization and DevOps News Week in Review 1.18.2021
re:Invent Round-Up of AWS DevOps Announcements
How Will SASE Change Networking in 2021?
AWS re:Invent Machine Learning Round-Up
How to Publish Managed Images to the Azure Marketplace
AWS re:Invent News Round-Up
Shave Days off Azure Marketplace Publishing with Automated Testing
IT Modernization and DevOps News Week in Review 11.30.2020
How To: Multi-Cluster Monitoring in Amazon EKS
IT Modernization and DevOps News Week in Review 11.16.2020