«Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture Oracle White Paper—Oracle's Sun SPARC ...»
An Oracle White Paper
Oracle's Sun SPARC Enterprise
and Oracle's Sun SPARC Enterprise
T5140/T5240 Server Architecture
Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture
The Evolution of Chip Multithreading
Rule-Changing Chip Multithreading Technology
Oracle's Sun SPARC Enterprise T5120/T5220 and
Oracle's Sun SPARC Enterprise T5140/T5240 Servers
UltraSPARC T2 and T2 Plus Processors
The World's First Massively Threaded Systems-on-a-Chip
Taking Chip Multithreaded Design to the Next Level
UltraSPARC T2 and UltraSPARC T2 Plus Processor Architecture.................19 Server Architecture
Chassis Design Innovations
Oracle's Sun SPARC Enterprise T5120 Server Overview
Oracle's Sun SPARC Enterprise T5220 Server Overview
Oracle's Sun SPARC Enterprise T5140 Server Overview
Oracle's Sun SPARC Enterprise T5240 Server Overview
PCI Express Expansion Unit
Enterprise-Class Management and Software
System Management Technology
Scalability and Support for CoolThreads Technology
Fault Management and Predictive Self-Healing
Cool Tools: Performance and Rapid Time to Market
Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture Introduction Serving the dynamic and growing data center IT services space is challenging for data center operations. Services need to be able to scale rapidly, often doubling capacity in a short period even as they remain highly available. Infrastructure must keep up with these enormous scalability demands, without generating additional administrative burden.
Unfortunately, most data centers are already severely constrained by both real estate and power—and energy costs are rising. There is also a new appreciation for the role that the data center plays in reducing energy consumption and pollution. Virtualization has emerged as an extremely important tool as organizations seek to consolidate redundant infrastructure, simplify administration, and leverage underutilized systems. Security too has never been more important, with the increasing price of data loss and corruption. In addressing these challenges, organizations can ill afford proprietary infrastructure that imposes arbitrary limitations.
Employing Oracleʼs UltraSPARC T2 and T2 Plus processors—the industryʼs ﬁrst massively threaded systems-on-a-chip (SoC)— Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 servers offer breakthrough performance and energy efﬁciency to drive data center infrastructure and address other demanding data center challenges. Third-generation CoolThreads chip multithreading (CMT) technology supports up to 128 threads in as little as one rack unit (1RU)—providing increased computational density while staying within variously constrained envelopes of power and cooling. Very high levels of integration help reduce latency, lower costs, and improve security and reliability. Optimized system design provides support for a wide range of IT services application types. Uniformity of management interfaces and adoption of standards help reduce administrative costs, while an innovative chassis design shared across Oracleʼs volume servers provides density, efﬁciency, and economy for modern data centers. With both the processor and Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture Oracle Solaris available under open source licensing, organizations are free to innovate with a worldwide technical community.
Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture The Evolution of Chip Multithreading Oracle’s UltraSPARC processors have led the industry for years—first, with the introduction of multithreaded, multicore chip design in the UltraSPARC T1 processor and now with the thirdgeneration UltraSPARC T2 Plus processor. By any measure, these first-generation CMT processors were an unprecedented success. Delivering up to five times the throughput in a quarter of the space and power, systems using these processors have rapidly been welcomed and accepted. Now third-generation CMT technology is evolving rapidly to meet the constantly changing demands of a wide range of enterprise data center applications.
Business Challenges for Enterprise Applications
Organizations across many industries hope to address larger markets, reduce costs, and gain better insights into their customers. At the same time, an increasingly broad array of wired and wireless client devices are bringing network computing into the everyday lives of millions of people. This strong demand has a “pull-through” effect on the IT services that must be satisfied in the data center. These trends are redefining data center scalability and capacity requirements, even as they collide with fundamental real estate, power, and cooling constraints.
Driving data center virtualization and ecoefficiency. Coincident with the need to scale services, many data centers are recognizing the advantages of deploying fewer standard platforms to run a mixture of commercial and technical workloads. This process involves consolidating underused and often sprawling server infrastructures with effective virtualization solutions that serve to enhance business agility, improve disaster recovery, and reduce operating costs. This focus can help reduce energy costs and break through data center capacity constraints by improving the amount of realized performance for each watt of power the data center consumes.
Ecoefficiency provides tangible benefits, improving ecology by reducing the carbon footprint to meet legislative and corporate social responsibility goals, even as it improves the economy of the organization paying the electric bill. As systems are consolidated onto more dense and capable computing infrastructure, demand for data center real estate is also reduced. With careful planning, this approach can also improve service uptime and reliability by reducing hardware failures resulting from excess heat load. Servers with high levels of standard reliability, availability, and serviceability (RAS) are now considered a requirement.
Building out for Web-scale applications. Web-scale applications engender a new pace and urgency to infrastructure deployment. Organizations must accelerate time to market and time to service, while delivering scalable high-quality and high-performance applications and services. Many need to be able to start small with the ability to scale very quickly, with new customers and innovative new Web services often implying a doubling of capacity in months rather than years.
Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture At the same time, organizations must reduce their environmental impact by working within the power, cooling, and space available in their current data centers. Operational costs too are receiving new scrutiny, along with system administrative costs that can account for up to 40 percent of an IT budget. Simplicity and speed are paramount, giving organizations the ability to respond quickly to dynamic business conditions. Organizations are also striving to eliminate vendor lock-in as they look to preserve previous, current, and future investments. Open platforms built around open standards help provide maximum flexibility while reducing costs of both entry and exit.
Securing the enterprise at speed. Organizations are increasingly interested in securing all communications with their customers and partners. Given the risks, end-to-end encryption is essential to inspire confidence in security and confidentiality. Encryption is also increasingly important for storage, helping to secure stored and archived data even as it provides a mechanism to detect tampering and data corruption.
Unfortunately, the computational costs of increased encryption can increase the burden on already overtaxed computational resources. Security also needs to take place at line speed, without introducing bottlenecks that can impact the customer experience or slow transactions.
Solutions must help to ensure security and privacy for clients and bring business compliance for the organization, all without impacting performance or increasing costs.
Rule-Changing Chip Multithreading Technology Addressing these challenges has outstripped the capabilities of traditional processors and systems, and required a fundamentally new approach.
Mooreʼs Law and the Diminishing Returns of Traditional Processor Design The oft-quoted tenant of Moore’s law states that the number of transistors that will fit in a square inch of integrated circuitry will approximately double every two years. For more than three decades the pace of Moore’s law has held, driving processor performance to new heights.
Processor manufacturers have long exploited these gains in chip real estate to build increasingly complex processors, with instruction-level parallelism (ILP) as a goal. Today, these traditional processors employ very high frequencies along with a variety of sophisticated tactics to accelerate a single instruction pipeline, including Large caches Superscalar designs Out-of-order execution Very high clock rates Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture Deep pipelines Speculative prefetches Although these techniques have produced faster processors with impressive-sounding multiplegigahertz frequencies, they have largely resulted in complex, hot, and power-hungry processors that are not well-suited to the types of workloads often found in modern data centers. In fact, many data center workloads are simply unable to take advantage of the hard-won ILP provided by these processors. Applications with high shared memory and high simultaneous user or transaction counts are typically more focused on processing a large number of simultaneous threads (thread-level parallelism, or TLP) rather than running a single thread as quickly as possible (ILP).
Making matters worse, the majority of ILP in existing applications has already been extracted, and further gains promise to be small. In addition, microprocessor frequency scaling itself has leveled off because of microprocessor power issues. With higher clock speeds, each successive processor generation has seemingly demanded more power than the last, and microprocessor frequency scaling has leveled off in the 2 GHz to 3 GHz range as a result. Deploying pipelined superscalar processors requires more power, limiting this approach by the fundamental ability to cool the processors.
Chip Multiprocessing with Multicore Processors
To address these issues, many in the microprocessor industry have used the transistor budget provided by Moore’s law to group two or more conventional processor cores on a single physical die—creating multicore processors, or chip multiprocessors (CMPs). The individual processor cores introduced by many CMP designs have no greater performance than previous singleprocessor chips, and in fact, have been observed to run single-threaded applications more slowly than single-core processor versions. However, the aggregate chip performance increases since multiple programs (or multiple threads) can be accommodated in parallel (TLP).
Unfortunately, most currently available chip multiprocessors simply replicate cores from existing (single-threaded) processor designs. This approach typically yields only slight improvements in aggregate performance since it ignores key performance issues such as memory speed and hardware thread context switching. As a result, although these designs provide some additional throughput and scalability, they can consume considerable power and generate significant heat— without a commensurate increase in overall performance.
Chip Multithreading with CoolThreads Technology
Oracle engineers were early to recognize the disparity between processor speeds and memory access rates. While processor speeds continue to double every two years, memory speeds have typically doubled only every six years. As a result, memory latency now dominates much application performance, erasing even very impressive gains in clock rates. This growing Oracle White Paper—Oracle's Sun SPARC Enterprise T5120/T5220 and Oracle's Sun SPARC Enterprise T5140/T5240 Server Architecture disconnect is the result of memory suppliers focusing on density and cost as their design center, rather than speed.
Unfortunately, this relative gap between processor and memory speeds leaves ultrafast processors idle as much as 85 percent of the time, waiting for memory transactions to complete. Ironically, as traditional processor execution pipelines get faster and more complex, the effect of memory latency grows—fast, expensive processors spend more cycles doing nothing. Worse still, idle processors often continue to draw power and generate heat. It is easy to see that frequency (gigahertz) is truly a misleading indicator of real performance.
First introduced with the UltraSPARC T1 processor, CMT takes advantage of CMP advances, but adds a critical capability—the ability to scale with threads rather than frequency. Unlike traditional single-threaded processors and even most current multicore processors, hardware multithreaded processor cores allow rapid switching between active threads as other threads stall for memory. Figure 1 illustrates the difference between CMP, fine-grained hardware multithreading (FG-MT), and CMT. The key to this approach is that each core in a CMT processor is designed to switch between multiple threads on each clock cycle. As a result, the processor’s execution pipeline remains active doing real useful work, even as memory operations for stalled threads continue in parallel.
Figure 1. CMT combines CMP and ﬁne-grained hardware multithreading.