OPEN CONNECT APPLIANCES
Appliances
Overview
Open Connect Appliances (OCAs) have the following characteristics:
Are provided free of charge to qualifying partners
Our appliances are provided free of charge for ISP partners who meet our basic requirements, but they are not for sale to other parties.
Are designed for high availability
OCAs include redundant system drives, power supplies, and network interface ports.
If a hard drive fails, it is automatically disabled and the system will continue to function normally. If enough hard drives fail, Netflix will replace the appliance by shipping a new one.
See the sample architectures for more information about how you can configure your system for optimal resiliency.
Have no user-serviceable components, with the exception of power supplies and optics
Netflix will ship replacement power supplies or optics in case of failure at no cost to the ISP.
Are continuously monitored for health and performance by Netflix
Netflix will monitor the health and performance of each OCA as soon as it is reachable from our Network Operations Center (NOC). OCAs report health values and get their configuration from the Open Connect supporting services.
Health and system load information feedback is automatically incorporated into Netflix content routing decisions.
The Hardware
Netflix continues to evolve our appliance hardware capabilities to help ISPs most efficiently deliver high-quality Netflix traffic with a focus on localization. We tailor deployment and hardware architectures for each ISP that we work with. Our Open Connect Appliances are based on commodity PC components, assembled in custom cases by our suppliers.
We have several types of appliances, designed to meet our global deployment requirements. High level specifications for the most recent versions of our appliances are provided here. If you need detailed specifications for a particular revision of an Open Connect Appliance, contact your Partner Engagement Manager.
Storage Appliances
Storage appliances are 2U servers that are focused on reliable dense storage and cost effective throughput. This appliance is used to hold the Netflix catalog in many IX locations around the world and embedded at our larger ISP partner locations.
Storage appliance focus areas
- Large storage capacity
- 2U for rack efficiency (no deeper than 29 inches)
- Enough low cost NAND to reach 10GB/s of throughput (<0.3 DWPD)
- Network flexibility to connect at 6x10G LAG or up to 2x100GE
- 2 and 4 post racking
- AC or DC power
- Single processor
Storage appliance high-level specifications
Option | Vendors |
---|---|
Chassis | Sanmina |
Motherboard | Tyan |
Processor | AMD |
Memory | Micron |
Hard Drive | Seagate |
Solid State Drive | Western Digital |
Network Controller | Mellanox |
Power draw operational (peak) | ~650W |
Power Supply Unit | Redundant Hot Swap AC/DC |
Operational throughput | ~96Gbps |
Raw storage capacity | Up to 360 TB |
Global Appliances
Global appliances are 2U lower cost appliances that we use for smaller ISP partners and emerging markets. This appliance is designed for low cost 10GE attached content delivery.
Global appliance focus areas
- Lower absolute cost
- Ease of installation
- Only field-replaceable optics
- 4-6 year no touch reliability
Global appliance high-level specifications
Option | Vendors |
---|---|
Chassis | Supermicro |
Motherboard | Tyan |
Processor | Intel |
Memory | Micron |
Hard Drive | Seagate |
Solid State Drive | Micron |
Network Controller | Intel |
Power draw operational (peak) | ~270W |
Power Supply Unit | Redundant Hot Swap AC/DC |
Operational throughput | ~18 Gbps |
Raw storage capacity | Up to 120 TB |
Flash Appliances
For large deployments, we might also include flash-based appliances. These 2U appliances help scale network delivery for large sites up to Terabits per second as required.
Flash appliance focus areas
- 2U for rack efficiency (no deeper than 29 inches)
- Enough low cost NAND to reach 24GB/s of throughput (<0.3 DWPD)
- Connect at up to 2X100G LAG
- 2 and 4 post racking
- AC or DC power
- Single processor
Flash appliance high-level specifications
Option | Vendors |
---|---|
Chassis | Supermicro |
Motherboard | Tyan |
Processor | AMD |
Memory | Micron |
Solid State Drive | SanDisk |
Network Controller | Nvidia (Mellanox) |
Power draw operational (peak) | ~400W |
Power Supply Unit | Redundant Hot Swap AC/DC |
Operational throughput | ~190 Gbps |
Raw storage capacity | 24 TB |
Acknowledgements
In building these systems we collaborate with a wide range of suppliers who we would like to thank for their assistance: The teams at Sanmina, MBX, and Intequus, our system integrators. Storage guidance and troubleshooting from Western Digital, Seagate, Broadcom, and Micron. Network card and driver assistance from Chelsio and Mellanox. Compute assistance from Intel and AMD.
Software
Netflix delivers streaming content using a combination of intelligent clients, a central control system, and a network of Open Connect appliances.
When designing the Open Connect Appliance Software, we focus on these fundamental design goals:
- Use of open source software
- Ability to efficiently read from disk and write to network sockets
- High-performance HTTP delivery
- Ability to gather routing information via BGP
Component | Description | Comments |
---|---|---|
Operating System | FreeBSD -CURRENT (HEAD) | FreeBSD was selected for its balance of stability and features, a strong development community and staff expertise. All code improvements, feature additions, and bug fixes are contributed directly back to the open source community via the FreeBSD committers on our team. We also strive to stay at the front of the FreeBSD development process, allowing us to have a tight feedback loop with other community and partner developers. The result has been a positive open source ecosystem that lowers our development costs and multiplies the effectiveness of our efforts. |
Web Server | NGINX | NGINX was chosen for its proven scalability and performance. The audio and video components that comprise each Netflix streaming title are served directly to the customer client software via HTTP. |
Routing Intelligence Proxy | BIRD internet routing daemon | BIRD is used to enable the collection and sharing of network topology from ISP networks to the Netflix control system in AWS that directs clients to sources of content. |
IP support | IPv4 and IPv6 are fully supported. | |
Other | The remaining software on the system manages content and communicates system health and other statistics to Netflix Open Connect supporting services. |
Security
Source Code Provenance
Open Connect Appliance (OCA) software includes the FreeBSD operating system and the NGINX web server, licensed by BSD. Both of these products have active security teams. In addition, the commercial body nginx.com provides us with pre-announcements of security issues and patches to fix any vulnerabilities. As FreeBSD committers and Security Officers with extensive background in third-party packaging, the Netflix OCA development team is on trusted mailing lists and pre-announcement groups for security and take a proactive role in security protection and assurance.
For third party software packages, the team receives notices of vulnerabilities in advance by monitoring various pre-announcement lists, including oss-security and other trusted vendor-based sources.
In practice, security vulnerabilities are usually identified and fixed prior to being made public. We release firmware updates approximately every 5 weeks - however, if we need to fix a serious security bug, we can roll out a new firmware version within an hour.
Access to Open Connect Appliances (OCAs)
There is no ISP or third-party login access to the Netflix OCAs.
Content
Various intrusion detection methods are used, including a lightweight Static Intrusion Detection System that runs regularly on the OCAs to identify abnormal activity in the file systems and report it to the control plane.
Supporting MANRS
Mutually Agreed Norms for Routing Security (MANRS) is a global initiative, supported by the Internet Society, that provides crucial fixes to reduce the most common routing threats. We believe it is in the best interest of Netflix to be a good internet citizen and join the internet industry to address routing security issues.
A secure routing framework is essential to maintaining the ongoing health and stability of the global Internet, and MANRS provides the resources to develop, foster, and promote this framework.