Path: EDN Asia >> Design Centre >> Communications/Network >> Evaluating 10GE PHY technologies (Part 1)
Communications/Network Share print

Evaluating 10GE PHY technologies (Part 1)

04 Sep 2013  | AQuantia

Share this page with your friends

An important element of a structured cabling topology, such as that enabled by 10GBASE-T, is a zone distribution model created utilising Horizontal, Zone, Intermediate and Main Distribution Areas similar to an EoR, MoR, methodology. The size of the zone and associated distribution area can be a dedicated network row, or a zone dedicated to several rows. The idea is to limit the number of switches purchased to the number of servers in the zone that require switch connections. Top of Rack designs can work alongside distribution areas at the end of row or in a central location for uplink ports, again either at the end of a row or supporting several rows. Each zone can be repeated throughout the entire data centre white space, so that each zone may have the same configuration of server cabinet rows, network racks, and cabling routes. As the distance within a zone or between zones can extend well beyond 10m and up to 100m, the only option for connectivity is 10GBASE-T or SFP+ Fibre. SFP+ Fibre can cost as much as 4 to 7 times that of 10GBASE-T, greatly favouring the BASE-T solution over all others, as was the case already in Gigabit Ethernet deployments.

The zone distribution model utilising centralized switching is widely adopted as a cabling deployment scheme in data centres, especially in large and hyper-scale data centres. The zone distribution model in conjunction with 10GBASE-T cabling can provide excellent modularity, scalability, cost efficiency and consistency in the data centre while minimising ports purchased that cannot be used.

Comparative review of 10GE media options
Reach: Multivendor studies show the median length for network connections in data centres for ToR, EoR, and zone applications is in the range of 30m to 40m, with a substantial number of connections having greater than 50m length.

10GE SFP+ DAC is typically limited to 5m to 7m reach (respectively for passive and active cables), limiting it to ToR applications, and thus does not support the typical zone distribution model.

The 10GBASE-SR media option utilising OM3/OM4 multi-mode fibre cabling can support the longer range, but is not as cost effective as category 6A or 7A cabling, as discussed above.

Like all BASE-T implementations, 10GBASE-T supports lengths up to 100m, giving data centre managers a far greater level of flexibility in connecting devices in the data centre. 10GBASE-T is ideally suited for a zone distribution cabling architecture.

In patch cord type connectivity, where cable lengths are shorter than 10m, power consumption of the 10GBASE-T PHY is greatly reduced to about a couple of Watts. Some vendors define such mode of operation as "Data Centre" mode. The power savings, as much as 50% compared to a full 100m mode of operation, come from turning off a large part of the transceiver signal processing engine. When compared to SFP+ DAC, whether active or passive, which have both distance limitations to around 5 and 7m, the difference in power with 10GBASE-T is only about 1W or so. On a typical 1U server with two 10GE ports, and roughly a total power of 350W, that difference in power consumption equates to less than 1% which is offset by greater port utilisation than SFP+ DAC.

In brief, 10GBASE-T provides a unified solution for both the zone distribution model for scalability and consistency where reach extends to 100m, and the ToR model for modularity and efficient, low-cost cable management.

Compatibility with Legacy Networks: A key benefit of 10GBASE-T is backward compatibility. Using the standard RJ45 connector, 10GBASE-T ports can connect to 1 Gigabit and even 100 Megabit interfaces, allowing end-users to transparently upgrade to 10GE from their Fast Ethernet and Gigabit Ethernet networks. In comparison, 10GBASE-SFP+ DAC products typically have limited or no backward compatibility with previous speeds. Further, switch and server ports utilising 10GBASE-T also support 1GE in the same equipment. This allows network managers to determine which ports on a switch operate at which speeds. This can lighten bandwidth loads on the backplanes of switches and allow phased upgrades as needed without additional hardware spends.

ToR Switch Port Utilisation: Switch port densities also need to be factored in when designing and specifying data centre network architecture. Customers typically purchase aggregation switches based on both the density of the server systems per rack, number of racks deployed, and the bandwidth (number and speed of ports) required to interconnect these racks at wire-speed.

The number of servers deployed per rack is limited by power consumption (on average, 5.5kW to 6kW) in a passive environment, weight supported by the cabinet and floor space. Smaller rack configurations require ToR switches that offer 24 to 36 ports, while larger rack configurations require switches that range up to 48 ports.

In the future, as server processing and Ethernet-based storage continue to grow and the per-port power consumption continues to decrease, the number of 10GE port connections supported will likely expand beyond 48 ports within a rack.

Consistent with data centre high-availability requirements, ToR switches are deployed in dual-redundant configuration with each switch supporting redundant connection to servers. This provides resiliency and full network functionality in the event one of the switches fail. Incorporating dual power supplies for the switches can also help alleviate failure scenarios.

With ToR deployments using 10GBASE-SFP+ DAC, it is very difficult to use all the ToR switch ports due to the limited reach of the cables and due to other factors that limit the number of in-rack servers such as power, cooling and weight. Specifically, the reach limitation of 10GBASE-SFP+ DAC limits its use to intra-rack applications; any ToR switch ports that are not used for intra-rack connections to the servers are left unused. The unused DAC switch ports carry an initial cost outlay and require power (even in idle mode) and maintenance costs, making them expensive on an ongoing basis. Further the switch itself requires dual power connections and further limits the number of servers that can be installed in the cabinet.

Switches with 10GBASE-T ports offer greater design flexibility when compared to SFP+ cabling alternatives. This flexibility is based on greater distance capabilities, lower cabling and port costs, and simpler moves, adds, and changes.

10GBASE-T support of up to 100m lengths and patch panels/structured cabling gives network engineers a higher level of flexibility and greater level of utilisation of the switch ports (including ToR switches supporting servers across multiple racks, if necessary), as well as through supporting a greater number of deployment options, including ToR, EoR, MoR, and inter-cabinet connectivity.

About the author
This article is contributed by AQuantia.

To download the PDF version of this article, click here.

 First Page Previous Page 1 • 2 • 3

Want to more of this to be delivered to you for FREE?

Subscribe to EDN Asia alerts and receive the latest design ideas and product news in your inbox.

Got to make sure you're not a robot. Please enter the code displayed on the right.

Time to activate your subscription - it's easy!

We have sent an activate request to your registerd e-email. Simply click on the link to activate your subscription.

We're doing this to protect your privacy and ensure you successfully receive your e-mail alerts.

Add New Comment
Visitor (To avoid code verification, simply login or register with us. It is fast and free!)
*Verify code:
Tech Impact

Regional Roundup
Control this smart glass with the blink of an eye
K-Glass 2 detects users' eye movements to point the cursor to recognise computer icons or objects in the Internet, and uses winks for commands. The researchers call this interface the "i-Mouse."

GlobalFoundries extends grants to Singapore students
ARM, Tencent Games team up to improve mobile gaming

News | Products | Design Features | Regional Roundup | Tech Impact