This is Part 2 of our series on Netflix opensource tools. You can find Part 1 here.
Netflix Persistence Systems Tools
Astyanax is a Java Cassandra client similar to Hector that includes several features for performance improvement and ease of use. It provides lower latency, reduced latency variance and efficient error handling. It comprises three parts:
Connection pool provides complete abstraction from the Client API layer. Key features include node auto discovery, token-aware accesses, a client-latency score calculator, run-time performance tweaking and failure-tolerance rate limiting.
Cassandra Thrift API allows the caller to customize queries using set defines interfaces.
Recipes and utilities for common use cases such as CSV Importer, JSON exporter, parallel reverse index search and key unique constraint validation.
CassJMeter is a JMeter plugin for Cassandra. It’s used to load test functional behavior and to measure Cassandra cluster performance. It provides graphical analysis of performance and servlets/files testing in a heavy load scenario. It can be used to simulate and perform tests on varying loads.
Curator, now moved to Apache, is a set of libraries that makes Zookeeper easier to use. Its components include:
Framework is a high level API that manages complex ZooKeeper connections and simplifies Zookeeper use.
Recipes implementation for common operations on Zookeeper, including elections, locks, barriers, counters, caches, nodes and queues using the Curator framework.
Utilities that are helpful when using Zookeeper, including a test server, a test cluster, ZKPaths and EnsuePaths.
Client replaces the bundled Zookeeper class and manages basic housekeeping.
Error handling takes care of connection guarantees, notifications and recipes involved in error handling.
EVCache is a caching solution based on memcached and spymemcached. It’s typically used for caching frequently used data in AWS EC2 instances. It’s an abbreviated form of Ephemeral Volatile Cache which indicates that data stored in the in-memory key-value store is for a short duration and can disappear at any time. EVCache’s key features include:
Distributed Store that allows multiple instances.
AWS Zone-Aware that allows multi-zone replication.
Automatic Discovery of nodes and services in conjunction with Eureka.
Exhibitor is a Java-based tool that supervises Zookeeper. Key features include:
Ensuring that a Zookeeper instance is running.
Periodic log directory cleaning.
A GUI explorer for viewing Zookeeper nodes.
A rich REST API.
Priam runs as a co-process with Cassandra on every node. Functions include:
Performing backup and recovery.
Automating token assignments.
Providing REST API to Cassandra metrics.
Staash, which stands for STorage As A Service over Http, is a language- and storage-agnostic web interface that stores data in persistent systems. The metadata layer allows abstraction of storage details. It also provides pattern-automation APIs to automate common data access patterns. Its purpose is to free the developer from having to understand implementation details such as key-value store or column-oriented DB. Staash currently supports Cassandra and MySQL (asa proof of concept).
Netflix Build and Deploy Tools
Aminator is a tool that helps to create Amazon Machine Images (AMIs). It creates an AMI in five steps:
Aminator is written in Python and uses several opensource Python libraries. It aids AWS workflows, especially those that require autoscaling groups. It prioritizes agility and manageability over pool-management . For a discussion of various autoscaling strategies, please refer to our post on the best method for deploying a Ruby app to AWS at <INSERT LINK HERE>
Glisten, built on top of Groovy, is an Amazon Simple Workflow Service library. It was formerly part of Asgard, but is now a separate project.
Frigga, a Java library, is the logic behind Netflix’s Asgard project that parses and generates AWS object names.