This section describes how Open Connect Appliances are typically configured in a network. If you are an Open Connect ISP partner, Netflix works closely with you to determine the optimal configuration for your particular needs.
For more information, see the FAQs
OCAs are directed cache appliances, meaning that the manner in which traffic is directed to the appliance is determined explicitly by you and by Netflix, not by the appliance itself.
An OCA only serves clients at IP addresses that you advertise to the OCA via a BGP session. In other words, traffic is only delivered from your embedded OCAs to the customer prefixes that you explicitly announce to them, as described in the following sections. Therefore, you as the ISP partner have full control over the networks that the appliances will serve. BGP sessions are established between appliance(s) and the closest connected router.
If content is requested that is not contained on an embedded OCA, the client request is directed to the closest Netflix content site via peering (if present) or via transit.
Reconfiguring the IP address of an OCA
Each appliance comes fully configured based on the IP address details that you provided to Netflix in your site survey before it was shipped.
For all appliances, the IP address can be updated via a keyboard and monitor. Interfaces are on the front of the chassis, but might be hidden behind a panel. The updated IP address will only take effect after a reboot, so it is import to drain the appliance two hours before the change by shutting down its BGP session (or sessions) to ensure that there is no traffic being served by the appliance.
Router interface configuration
When you are connecting the appliances to your router, follow the guidelines in this section. See also Example router configurations.
- Each OCA requires one usable IPV4 address, and it is highly recommended to also assign one IPV6 address.
- You can assign an address to the appliance from an IPv4 subnet of /31 and larger, or an IPv6 subnet of /126 and larger.
- It is acceptable to assign an address to the appliance from a larger subnet (for example, a /24). However, because only one IPv4 address is required per appliance, a smaller subnet is typically used.
- The router interfaces must be configured for Link Aggregation Group (LAG) with LACP.
- A standard maximum transmission unit (MTU) must be configured on each router interface. Jumbo frames are not supported.
- If there are multiple routers available that can provide redundancy in a site, it is recommended to stagger appliances between routers. Appliances on the same router should be in the same subnet to optimize filling. Appliances on separate routers should be in separate subnets. Appliances are not designed to be connected to two separate routers.
- Each OCA is hardened against network attack and is designed to be directly connected to the internet. Filtering inbound or outbound traffic can cause operational issues, so we strongly recommend that you allow all traffic on all ports, do not use ACLs, and ensure that your router has a default route or full routing table. If you absolutely must filter, the current list of inbound and outbound usage follows. Please note that these can change at any time without prior notification.
- Traffic from OCA: Allow all destination addresses and ports.
- Traffic to OCA: Allow TCP 22, 53, 80, 179, 443, UDP 53 and 123 (source and destination), ICMP types 0, 3, 8, 11, and all ICMPv6 from any public IP/port. Allow all return traffic from any appliance-initiated connection (TCP established).
- Each network interface must be receiving between 0 dBm and -10 dBm of light to ensure good data throughput. The LCD panel on the front of the appliance displays the current light levels for each interface. If your appliance does not have an LCD panel, access the console with a keyboard and mouse, then follow the console instructions to check light levels. If light levels are out of the acceptable range, clean the optics. If cleaning the optics does not bring them into the acceptable range, contact Netflix to have new optics shipped to you.
Routing and content steering via BGP
We steer clients to our OCAs based on an ISP’s BGP advertisements, coupled with the routing and steering algorithms in the Open Connect control plane. ISP partners can control some aspects of content steering via the BGP routes that are announced to each OCA.
The control plane steers requests from end user clients to the best available appliance based on multiple factors. Assuming that the appliance has the requested title and available serving capacity, the control plane provides clients with a ranked list of appliances (typically 3 or more reliable sources) to stream from.
Appliance selection criteria
The following appliance selection criteria are considered, in order, by the Open Connect control plane services. If there is a tie for a given criterion, then the next criterion is considered. If there is a tie on all criteria, traffic is balanced between appliances.
- The appliance that receives the most-specific route to the client’s prefix.
- The appliance that receives the route to the client’s netblock with the shortest AS path.
- The appliance that receives the route to the client’s netblock with the lowest multi-exit discriminator (MED). (See the notes on MEDs below).
- The geographically closest appliance. We geolocate based on client IPs, whose location is then compared to the latitude and longitude of nearby OCAs to determine the closest available system.
- Route announcements for Open Connect embedded appliances:
- IPv4 prefixes between /8 and /31 (inclusive) are accepted.
- IPv6 prefixes between /19 and /64 (inclusive) are accepted.
- Route announcements for Open Connect peering sessions:
- IPv4 prefixes between /8 and /24 (inclusive) are accepted.
- IPv6 prefixes between /19 and /48 (inclusive) are accepted.
- As an implicit requirement, all appliances must have a BGP session configured in order to correctly participate in Netflix content steering and delivery.
- Advertised routes that are received by an OCA are synchronized with Open Connect control plane services approximately every five minutes.
- To localize traffic, the best practice is to advertise the most specific routes to the appliance. For example, if you are announcing a /22 to the OCA, but a /24 is received from the same block over settlement-free interconnection (SFI) peering or transit, the /24 will be preferred, delivering content traffic from the remote source instead of the OCA.
- If you are deploying only one OCA in your network, you should advertise the most specific (shortest) prefix for that OCA over the peering session that you want the OCA to use for nightly filling purposes.
- If you are deploying multiple OCAs in your network:
- To enable efficient nightly fill: If you have separate clusters that are located in two different sites, ensure that the appliances within one cluster can hear the subnets for the appliances in the other cluster via the BGP connection that is established with your router. See the Fill and updates information for more details.
- See the additional information about clustering architectures.
- MEDs can be injected and will be respected. You can use MEDs to de-preference routes between appliances or otherwise distribute traffic to meet your requirements. See additional notes below.
- Netflix does not use any BGP community information that is advertised by partners to OCAs or via Open Connect peering.
Additional notes on MEDs
- Important: Marking MEDs on already installed and working Open Connect Appliances can be hazardous, because it must be done on all BGP sessions for all appliances at the same time.
- There is no cap on the maximum MED value.
- A missing MED is treated the same as a MED of 0, and indicates that the appliance should receive all servable traffic for the associated prefixes (also often referred to as MED-missing-as-best). Remember, if multiple appliances receive the same prefix with the same metric, traffic is load-balanced across those appliances. Because a missing MED will be equivalent to 0, it is preferred over any >0 MED on other appliances.
- For information about MEDs with respect to peering only, see Peering Locations.
Two or more OCAs that are intended to serve the same set of customers can be configured by the Open Connect operations team as a manifest cluster. OCAs in a manifest cluster share content storage and function together as one logical server/storage unit. The Netflix team collaborates with you to determine whether clustering is warranted and how to set up optimal clusters, depending on your particular site and network configurations.
Although partners do not need to configure manifest clusters, it is important to understand some basic clustering concepts. In particular, there are implications to consider when OCAs in a cluster are taken down for maintenance or moved to a different site.
Clustering has the following potential benefits:
Greater offload for unique content
In a typical two-OCA cluster, both appliances will use approximately 40% of their storage for the same popular content. This popular content typically represents roughly 60% of the OCA’s total offload. The remaining 60% of storage space on each OCA is used to store a unique set of less-frequently-accessed content. Because we do not store the same exact set of content on each single OCA in a cluster, a cluster of OCAs provides greater total offload than an unclustered group of OCAs. This strategy helps the OCAs in a cluster function more efficiently.
For more information about storage strategies, see this tech blog post.
Redundancy is generally acceptable in a two-OCA cluster. In the event of a single OCA failure, the healthy appliance will take over the majority of the traffic that the failed unit was serving. See the failover scenarios in the sample architectures.
Notes for partners:
- OCAs intended to serve the same set of customers can be clustered if they are located at the same site, or if they are in close geographical or network proximity.
- OCAs cannot be clustered if they are not intended to serve the same set of customers.
- After a set of OCAs has been installed in a site and grouped together as a cluster by the Open Connect team, they should be thought of as one big server. Therefore, any changes you make to a single OCA in a site has the potential to negatively impact the serving efficiency and behavior of the group.
- If you need to make changes to the OCAs in an established site - for example, if you intend to relocate an OCA from one site to another or disable one or more OCAs for a significant period of time - it is important to notify the Open Connect team so that they can make the necessary changes to the cluster configuration. Failing to do so can cause undesired consequences. For example, you may see traffic being steered to the wrong site, fill patterns may become suboptimal, and hot spots might develop.
- To enable optimal and balanced traffic patterns, OCAs in a site and within a cluster must receive the exact same BGP route advertisements. Therefore, if you relocate an OCA you must revisit your BGP route announcements to ensure that traffic continues to be steered appropriately.
Embedded OCAs combined with peering sessions
The ideal Open Connect implementation is a mixture of both SFI peering and deployed embedded OCAs. Netflix uses two separate autonomous systems for peering:
- AS2906 is the AS number that Netflix uses for peering at its PoPs
- AS40027 is the AS number that embedded OCAs use to peer with ISP networks
See BGP notes for prefix announcements that are accepted on peering sessions.
The same prefix announced both to a private or public peering session (using AS2906) and to an OCA (using AS40027) will always be preferred on the appliance over peering, because the Open Connect control plane will have two BGP entries for that prefix:
- one with an AS PATH LENGTH of 1 (<AS_NUMBER>) from the appliance itself
- one with an AS PATH LENGTH of 2 (2906 <AS_NUMBER>) from an IX location
When OCAs and Open Connect SFI peering is combined, OCAs are nominal and peering is used primarily for backup, for filling, and for serving long-tail titles.
If you are an ISP with very large amounts of Netflix traffic, we will likely include flash-based appliances in your OCA deployment architecture. Flash-based appliances are 1U flash memory-based servers that are deployed when you reach a threshold number of OCAs, to augment the delivery capability of the main (storage) appliances.
Flash-based appliances have different fill window requirements, see the fill information for more details.
Flash-based appliances are not clustered unless they are in the same site, and they should not be set up in the same manifest cluster as the main storage appliances.
Rail kit instructions for flash-based appliances can be accessed online:
Note:Flash-based appliances, unlike storage appliances, have their ports on the back of the device facing the hot isle.
Back to top
© 2016 - 2018 Netflix, Inc. All rights reserved.