$250,000 1992 IBM Processor Tear Down

[IoTPanic]

Saw this pass video pass through my subscription feed, while watching it I thought to myself "this would make linus wet" and therefore the community might like it too. Not sure how much crossover there is between LTT viewers and the EEVBlog, if you are interested in real electronics you should give Dave Jones a watch.

 

EEVblog #1341 - AMAZING $250,000 IBM Processor Teardown!

 

The construction of the "heat spreader" is amazing and cools 121 silicon dies on a single ceramic substraight with 2772 pins. This is a real processor, they would have many in a mainframe for enterprise compute. Can anyone beat that in CPU size alone?

 

(Edit: I can spell)

[Radium_Angel]

31 minutes ago, IoTPanic said:

This is a real processor

"Made in USA"

You don't see *that* anymore....

 

Back when systems were for the ultra-hardcore only. What an awesome system.

 

[whm1974]

So that is what Mainframe/Minicomputer Processors look like....

[basedsean]

My first thought when I watched this teardown was "Linus would love this"

[Guest]

Gold lovers dream chips.

 

http://ibmcollectable.com/gallery/album22

 

Now imagine 121 Core I9's or 121 3950X processors (Or even TR's) jammed into a huge cooling plate....

 

I'm drooling.... lol

[Helpful Tech Witch]

1 minute ago, ShrimpBrime said:

Gold lovers dream chips.

 

http://ibmcollectable.com/gallery/album22

 

Now imagine 121 Core I9's or 121 3950X processors (Or even TR's) jammed into a huge cooling plate....

 

I'm drooling.... lol

with a healthy load of ability to change out huge cooling plate to waterblock of choice.

[Guest]

Just now, HelpfulTechWizard said:

with a healthy load of ability to change out huge cooling plate to waterblock of choice.

*Drops pump into bucket of Ice water*

[whm1974]

4 minutes ago, ShrimpBrime said:

Gold lovers dream chips.

 

http://ibmcollectable.com/gallery/album22

 

Now imagine 121 Core I9's or 121 3950X processors (Or even TR's) jammed into a huge cooling plate....

 

I'm drooling.... lol

And what will do you do with such a thing? How big would the Computer be?

[Guest]

2 minutes ago, whm1974 said:

And what will do you do with such a thing? How big would the Computer be?

On a serious note?

 

Well I'd rent out gaming servers and try to make a buck. And I wouldn't choose core I9 for that task either.

[S w a t s o n]

This is actually insane. I knew IBM had the first MCM CPUs but the ones I saw had less than 10 die I'm pretty sure. This is basically wafer scale computing but with all the complexities of packaging many many dies. The insane cooling for 10W per die seems super overkill. Modern CPUS can use up to 10W PER CORE often, 7nm zen chiplets are warm for a reason.

It really makes me wonder what is possible today. The modern version of this is the Z15 CPU but I think that's a monolithic CPU (cant actually find any pics of the CPU but the block digaram is definitely a monolithic design) with "only" 12 cores. I wonder if IBM wanted to, what could they do now. Also makes Intel and AMD look like the chumps they kind of are. IBM's processor tech has always been much more advanced than Intel even, they just recently have been eclipsed by many core x86 cpus.

Even POWER10 which IBM just announced wont be anything as crazy as this thing.

 

Spoiler

POWER5

 

 

[Guest]

Pretty neat. 

 

121 10w chips at 600w collectively. 

That would explain the cooling apparatus. 

 

For the sake of this thread and how awesome it is, I will put the technical data here for you guys/gals.

Copy pasted the PDF contents in case it's ever removed later. 

Spoiler

The IBM Enterprise System/9000 Type 9121 air-cooled processor by S. F. Hajek The IBM Enterprise System/9000" Type 9121 air-cooled processor achieves, with the same or reduced physical floor space and power levels, a performance level equal to or greater than those of previous IBM processors. This performance level was attained by a combination of design innovations: a new aircooled thermal conduction module (TCM), integration of bipolar and CMOS technology in this TCM, the design and implementation of a differential current switch bipolar circuit family, integrated programmable memory subsystem design, and extensive use of VLSl technology. The result of these innovations was a 15-nscycle air-cooled machine. The salient features and an overview of the machine design are presented. Introduction Three different types of processors are used in the new IBM Enterprise System/9000TM (ES/9000TM) family: watercooled processors (Type 9021), air-cooled processors (Type 9121), and an air-cooled rack processor (Type 9221). The Type 9121 processors use a thermal conduction module (TCM) [1] which is air-cooled. This radical departure from previous TCMs required extensive redesign of the internal and external TCM cooling system, TCM substrate, substrate wiring, and top-surface distribution wiring; new chip logic circuitry; and the use of bipolar and CMOS chips in the same TCM. In addition, new ancillary components such as system logic and air cooling were designed to meet the enhanced performance, space. and power-consumption specifications. This issue of the ZBM Journal of Research and Development describes the design innovations embodied in the ES/9000 Type 9121 air-cooled processor. Design advances The following design improvements have been realized in the ES/9000 Type 9121 processor: A new differential current switch (DCS) bipolar circuit family has been used to obtain the circuit density and powedspeed performance required for an air-cooled TCM design. High speed and low power are achieved through reduced signal swing and an active rising-signal transition. Wopyright 1991 by International Business Machmes Corporation. Copying in printed form for private use is permitted without payment of royalty provided that (1) each reproduction is done without alteration and (2) the Jorrmol reference and IBM copyright notice are included on the first page. The title and abstract. but no other portions. of this paper may be copied or distributed royalty free without further permission by computer-based and other information-service systems. Permission to replrblish any other ponion of this paper must be obtained from the Editor. IBM J. RES. DEVELOP. VOL. 35 NO. 3 MAY 1991 S. F. HAJEK 307 An air-cooled TCM has been developed which uses a “thin-film’’ copper wire layer for redistribution, 63 wiring layers, spring-loaded copper pistons, and a finned heat sink to cool up to 121 logic chips which generate up to 10 W individually and 600 W collectively [2]. A new, high-speed CMOS SRAM meets the power and speed requirements for the internal arrays of the processor. A 10-ns-access, 128Kb chip design has been achieved [3]. CMOS, ECL bipolar, and DCS bipolar technologies have been integrated within a single air-cooled TCM, and a high-power ECL-compatible CMOS driver has been developed [3] to obtain a 15-11s processor cycle time. “Cycle-stealing” design techniques are used to compensate for the slower CMOS SRAM arrays [4]. This constitutes the first use of CMOS technology within a TCM in any machine design. Central and expanded storage have been integrated into a single physical storage. Advanced design techniques such as single-level buffers have been implemented to achieve the performance levels of independent storages [5]. A programmable design technique has also been introduced which allows the user to dynamically configure the storages at initial program load (IPL). A vector facility has been designed for the air-cooled processor [6]. VLSIlCMOS technology is used extensively in the design of the I/O channels. A single CMOS multichip module (MCM) implements either the parallel original equipment manufacture interface (OEMI) or the fiber-optic-based Enterprise Systems Connection ArchitectureTM (ESCON T‘) serial channel. The common design of these new channels has allowed the 9121 processors to offer channel subsystem mixtures of both channel types within a single frame. As a result of these design improvements, a six-TCM dyadic processor achieves a performance level more than four times that of previous air-cooled frame designs such as the IBM ES/438ITM Model 92E. The extensive use of VLSI technology provides as many as 28 channels and 1 GB of storage within a single frame. (A frame is the structure which contains the essential elements of the system.) Figure 1 is a logical view of the system. The channel control element (CCE) TCM controls the channel subsystem. The channel subsystem comprises the CCE TCM, the inputloutput processor (IOP) contained within the CCE TCM, the channel adapter cards, the parallel OEMI channels, the ESCON serial channels, and the associated cabling. A circuit board can contain either 12 parallel OEMI and four ESCON serial channels, or 16 ESCON serial channels. The base frame can contain up to 308 two channel circuit boards, resulting in 32 channels. An S. F. HAJEK additional 16 channels on two channel circuit boards can be contained in an expansion frame, giving the user a total of 48 channels. The channel adapter card converts the common parallel OEMI and ESCON serial channel interface to that of the CCE TCM interface. Individual I/O buffers are provided for each attached channel. The ESCON serial channel utilizes high-speed fiber-optic technology to increase distances, provide higher data rates, and reduce cable bulk to the processor-attached I/O control units and devices. ESCON serial channels support speeds of 10 MB/s as compared to the 4.5 MB/s of parallel channels. The system control element (SCE) TCM controls the flow of data among the central storage, the expanded storage, the processors, and the channel subsystem. This TCM also controls the central and expanded storage configurations of the single physical processor storage. Configuration parameters are selected at power-on reset. Storage protection keys are maintained within this TCM. The buffer control element (BCE) TCM contains either a 64KB or a 128KB high-speed buffer. The high-speed buffer data array is composed of SRAM CMOS 128Kb chips. The BCE is organized with an eight-double-word line size and four-way set associativity. The BCE utilizes a byte errorcorrecting algorithm (ECC) complemented by a line-delete algorithm. No single failure point can result in more than one bit error in any given ECC word. The central processor element (CPE) TCM executes all instructions defined by the ESA/390TM architecture. The Processor Resource/Systems ManagerTM (PR/SMTM) [7] function within the processor allows several independent logical processors to share the physical processor. System/370” operating systems are supported under one or more partitions of the PNSM facility. The internal cycle time is 15 ns for all models. The internal performance level of the processor is determined by the size of the highspeed buffer contained within the BCE TCM and other design parameters. feature which executes 171 vector instructions and contains the vector registers. Vector instructions significantly improve numerically intensive applications in that the same instruction operates on multiple sets of data. In contrast, scalar operations require a set of instructions for each set of data. The 16 vector registers which together form the vector array are each 256 elements deep and 32 bits wide [6]. A single storage configuration contains both central and expanded storage [5]. A total of 1024 MB of storage can be contained on one storage circuit board. Up to eight storage array cards of varying sizes are used to form various storage configuration sizes, up to the maximum of 1024 MB. The storage is logically configured by programmable means into central and expanded storage at power-on The vector control element (VCE) TCM is an optional IBM J. RES. DEVELOP. VOL. 35 NO. 3 MAY 1991 CCE SCE , TCM Cenaal and expanded *TcM storage I I ES/9000 air-cooled frame processor: logical view. reset. A processor can have up to 256 MB of central storage. Either 1Mb or 4Mb storage chips are used for the physical storage. Error-correcting code bits are stored with the data. The ES/9000 Type 9121 air-cooled processor is packaged within two frames, a base frame and an optional expansion frame. The 9121 base frame accommodates the following: 1024MB central and expanded storage and SCE and CCE TCM subsystem. Two processor (CPE/BCE TCM) subsystems without VCE TCM, or one processor subsystem with VCE TCM. Up to 32 parallel andlor ESCON channels. An inboard service processor. A powerlthermal subsystem. The expansion frame, which measures approximately 2/3 the length of the base frame, accommodates VCE TCMs for the dyadic processor and additional channels to a maximum of 48 channels per system. Figure 2 provides a floor-space comparison of the ES/3090TM [8] and ES/4381 processors and the E39000 Type 9121 air-cooled processor. Figure 3 shows the location of the six TCMs of the dyadic processor within the base frame. Below the TCMs the channels are located on two channel circuit boards with the I/O panel adjacent. To the left of the TCMs is the operator panel, which is accessible when the covers are in place. Located below the operator panel is the processor storage, which is packaged on a single board. Located across the top of the frame are power supplies and blowers used for cooling the components. 309 IBM J. RES. DEVELOP. VOL. 35 NO. 3 MAY 1991 S. F. HAJEK I 1 ES13090 Model 180E I I 7.0 m' 4415 kg 29.1 kVA P/CDU I n I Processor svc and disks pm 2.6 mz U ES/4381 Model 14 1.34 m2 770-910 kg 4.7-7.2 kVA ES/9121 air-cooled frame processor I I 1.34 m2 748-639 kg 6.5-10.9 kVA ES/9000 air-cooled frame processor: floor-space comparison. Summary Reliability, availability, and serviceability (RAS) targets were achieved in the design of the ES/9000 Type 9121 processor. Increased circuit densities have greatly reduced the number of external interconnections, a prime source of reliability failures. The machine design has also benefited from the reduced intrinsic failure rate of the base technologies. Machine design has complemented these technology improvements with extensive error-correcting design techniques throughout the design. The high-speed buffer array, for example, uses an error-correcting code on each byte of stored data. Other error-correcting techniques are used on the other data arrays contained within the processor complex. Error-recovery designs are also implemented throughout the processor. An inboard processor controller contained within the base frame provides recovery and support services to the machine. An outboard PS/2@-based system is utilized as an 31 0 external support facility. All necessary machine functions are executed by the inboard processor. In fact, the PS/2 can be disconnected without disrupting the operation of the machine. Many advances were made in designing the power thermal subsystem for this air-cooled machine. In contrast to the IBM 3090TM processors [8], a motor-generator set is not required. Three-phase power of either 50 or 60 Hz is converted into various regulated dc voltages required by the logic technologies. A maximum of 11 kVA is required for a fully configured dyadic processor. The thermal system utilizes forced convection cooling technology to provide the appropriate cooling for all of the logic technologies. Advances in semiconductor technology played a key role in achieving success. A differential current switch circuit family was developed which made it possible to meet the powedspeed requirements of a 15-ns machine design point. This circuit technology is supported by advanced physical design, circuit design, and chip design S. F. HAJEK IBM J. RES. DEVELOP. VOL. 35 NO. 3 MAY 1991 31 1 IBM I. RES. DEVELOP. VOL. 35 NO. 3 MAY 1991 S. F. HAJEK practices. Corresponding advances in TCM technology were required to support increased chip densities and cooling requirements. Advances in logic design verification and manufacturing test verification were also required to make this machine a reality. The Engineering Verification Engine (EVE) [9] was heavily utilized in verifying the logic design and contributed significantly to the reduction of the development cycle for this product. EVE allowed the design engineers to run the System Architectural Kernel (SAK) program before releasing the hardware design. Successful execution of the SAK program on the test floor was an announcement criterion. Self-test techniques were used in the TCM design [lo] to satisfy manufacturing requirements. The Enterprise System/9000 Type 9121 processor has advanced the state of the art in air-cooled mainframe processors. A performance level four times greater than those of previous processors was achieved within the same volumetric and power characteristics. For instance, the IBM 3090 System Model S, a water-cooled mainframe, required 36 TCMs to achieve the 9121 performance level. Bipolar and CMOS technology were integrated within a TCM for the first time. Three different circuit families, CMOS, ECL bipolar, and DCS bipolar, were utilized to meet the performance and power requirements to achieve a 15-11s machine cycle design point. Appendix: Glossary ESl9000 Type 9121 air-cooled processor One of three mainframe types in the ES/9000 family of processors announced by IBM on September 5, 1991. The Type 9121 processors incorporate air-cooled TCMs to implement the design point. ESA1390 The system architecture announced by IBM on September 5, 1991. All hardware and software elements of the ES/9000 Type 9121 processors support this architecture. System1370 architecture is supported as a subset. Differential current switch (DCS) A bipolar circuit design technology which uses differential circuit design techniques to achieve the desired density, power, and performance. Emitter-coupled logic (ECL) A bipolar circuit design technology which uses emitter-coupled circuit design techniques to achieve high performance. Acknowledgments It is impossible to acknowledge individually all of the engineers responsible for the design of this machine. 31 2 However, I would like to acknowledge William Spraul, S. F. HAJEK engineering design manager, and his talented team of engineers and managers who made this machine a reality. I would also like to acknowledge Columbo Dalmas and Paul Simoneau for their tireless efforts in performing the verification and simulation efforts for this machine. Virgil McIntosh and his physical design team spent countless hours performing the physical design of the chips, TCMs, boards, and cards for this machine. Enterprise Systeml9000, ES19000, Enterprise Systems Connection Architecture, ESCON, ES14381, ESAi390, Processor Resource/Systems Manager, PRISM, Systeml370, ESi3090, and 3090 are trademarks, and PSI2 is a registered trademark, of International Business Machines Corporation. References I. A. J. Blodgett and D. R. Barbour, “Thermal Conduction Module: A High-Performance Multilayer Ceramic Package,” IBM J. Res. Develop. 26, 30-36 (1982). 2. V. L. Gani, M. C. Graf, R. F. Rizzolo, and W. F. Washburn, “IBM Enterprise System/9000 Type 9121 Model 320 Air-Cooled Processor Technology,” IBM J. Res. Develop. 35, 342-351 (1991, this issue). 3. J. L. Chu, H. R. Torabi, and F. J. Towler. “A 128Kb Static Random-Access Memory,” IBM J. Res. Develop. 35, 321-329 (1991, this issue). 4. P. R. Turgeon, A. R. Steel, and M. R. Charlebois, “Two Approaches to Array Fault Tolerance in the IBM Enterprise System19000 Type 9121 Processor,” IBM J. Res. Develop. 35, 382-389 (1991, this issue). System/9000 Type 9121 System Controller and Memory Subsystem Design,” IBM J. Res. Develop. 35, 357-366 (1991, this issue). 6. T. J. Slegel and R. J. Veracca, “Design and Performance of the IBM Enterprise System19000 Type 9121 Vector Facility,” IBM J. Res. Develop. 35, 367-381 (1991, this issue). Order No. GA24-3416-00; available through IBM branch offices. 5. B. W. Curran and M. H. Walz, “IBM Enterprise 7. ES13090 Model S Processor Complex PRISM Feature, 8. S. G. Tucker, “The IBM 3090 System: An Overview,” 9. D. K. Beece, G. Deiberg, G. Papp, and F. Villante, “IBM Engineering Verification Vehicle,” Proceedings of the IEEE 25th ACMIIEEE Design Aufomation Conference, 1988, pp. 218-224. Systems Applied to the IBM Enterprise System/9000 Type 9121 Processor,” IBM J. Res. Develop. 35, 390-399 (1991, this issue). IBM Sy~t. J. 25, 4-19 (1986). 10. S. Sarma, “Enhanced Self-Test Techniques for VLSI Received October IO, 1990 Steven F. Hajek IBM Datu Systems Division, Neighborhood Roud, Kingston, New York 12401. Mr. Hajek is the Engineering Program Manager for the ES/9000 Type 9121 air-cooled processor. He joined the Data Systems Division in Kingston in 1968 and has since worked in various technical and managerial assignments both within DSD and other IBM divisions. Mr. Hajek received a B.S. and an M.S. in electrical engineering from Purdue University in 1966 and 1968. He is a member of the Institute of Electrical and Electronics Engineers. IBM J. KES. DEVELOP. VOL. 35 NO. 3 MAY 1991

https://pdfs.semanticscholar.org/fd3f/cf77924dc5c007859b356729b396b0f58d54.pdf

 

 

[S w a t s o n]

12 minutes ago, ShrimpBrime said:

121 10w chips at 600w collectively. 

That would explain the cooling apparatus.

I'm still skeptical. Given the massive surface area to spread and dissipate heat I think 600W is pretty manageable. If the chips were soldered like they are now I'm sure it would have been sufficient. Chips are only getting smaller, the heat density here is actually pretty low compared to a Zen2 chip, The total of 600w isnt the problem because of the huge package.

[Guest]

1 hour ago, S w a t s o n said:

 the heat density here is actually pretty low compared to a Zen2 chip, The total of 600w isnt the problem because of the huge package.

Or is it? 

 

121 logical chips. Measured in Millimeters and perhaps in the thousands of transistors and likely running between 2 and 5v. (Guessing numbers)

 

Zen 2 under 1.3v running Billions of transistors at 7nm.

 

That's one hell of a comparison. 

[S w a t s o n]

9 minutes ago, ShrimpBrime said:

Or is it? 

 

121 logical chips. Measured in Millimeters and perhaps in the thousands of transistors and likely running between 2 and 5v. (Guessing numbers)

 

Zen 2 under 1.3v running Billions of transistors at 7nm.

 

That's one hell of a comparison. 

Each one of those die are probably the size of a Zen2 chiplet from eyeballing it. Each Zen2 8C chiplet can do a 95W TDP. Look at the size of the desktop CPU he compares it with in the video, that can handle over 100W on the intel side.

 

If you OC them you can take them easily over 200W. 600W in this form factor is not a lot, but maybe they couldnt solder these to an IHS

 

Spoiler

 

[Guest]

4 minutes ago, S w a t s o n said:

Each one of those die are probably the size of a Zen2 chiplet or larger from eyeballing it. Each Zen2 8C chiplet can do a 95W TDP. Look at the size of the desktop CPU he compares it with in the video, that can handle over 100W on the intel side.

 

If you OC them you can take them easily over 200W. 600W in this form factor is not a lot, but maybe they couldnt solder these to an IHS

They did make an air cooled version of this processor though. I think the liquid cooled video just added to it being a cool video lol.

 

But to your comment of 200w OCed modern processor, compared to 600w chips of 1991... The cooling hasn't really changed much. You can air or liquid cool obviously old processors and new processors. The main point is only to keep within an operating temp. 

 

I'd like to figure the IBM chips from back then did not have a 95c throttle temp.... But more realistically designed to run optimally under 60c if not possibly lower expectations.

 

So many variables though. I think the IBM chip does have the advantage of surface area like you're saying. 

[S w a t s o n]

7 minutes ago, ShrimpBrime said:

So many variables though. I think the IBM chip does have the advantage of surface area like you're saying. 

The mass ratio though is hilarious, especially with it being oil filled, I would expect this to literally run at near (+10) ambient temp at 600W.

[Guest]

4 minutes ago, S w a t s o n said:

The mass ratio though is hilarious, especially with it being oil filled, I would expect this to literally run at near (+10) ambient temp at 600W.

Very well could be designed to run that cool. 

But from what I see in that picture, there's 3 of these IBM processors.

600w is roughly 2050 BTU/hr of heat to dissipate.

 

For comparison's sake, I'd be willing to bet that a single Zen2 core would be faster than 4 of these entire racks combined lol. 

 

 

 

 

 

[IoTPanic]

15 hours ago, ShrimpBrime said:

For comparison's sake, I'd be willing to bet that a single Zen2 core would be faster than 4 of these entire racks combined lol.

Lol, certainly, earlier IBM mainframes can be emulated with all their sister processors and banks of memory on a $5 Pi Zero.