Thunderbolt 3 Technology

[Kyklas]

Hi,

I am caching up on technology and Thunderbolt 3 cought my interest. I have read Wikipedia, Intel and Thunderbolt Technology side but I feel information is missing on the pure technological specification.

 

I write this topic after watching LG 5K Display for Mac - A PC User's Perspective from LTT. The explanation of displayport lanes into the thunderbolt is not satisfying for me. I think thunderbolt 3 can use more details.

 

Referring to the above video, how does Thunderbolt manage to channel "two displayport" and USB 3 data to the monitor ? My guess would be an encapsulation of the different protocol across the link.

 

My first investigation with Thunderbolt 3 is with USB :

I wonder how the Thunderbolt 3 was providing USB connectivity. I thought that the Thunderbolt controller could use one USB port from my chipset however it is not the case. The Thunderbolt 3 controller exposes a new PCIe USB XHCI Controller when a USB cable is connected. See screenshot. 

 

I would be curious to know what the system reports when other devices are connected like the dongles LTT reviewed.

 

Regards,
Stan

[Mira Yurizaki]

Thunderbolt, if I understand it correctly myself, is communication protocol to allow DisplayPort and PCIe data to share a single physical channel.

 

Anything not DisplayPort related on the Thunderbolt chain likely acts like any other PCIe based device. So if you plugged in a PCIe based USB port card, it'd be the same thing.

[Glenwing]

7 hours ago, Kyklas said:

Hi,

I am caching up on technology and Thunderbolt 3 cought my interest. I have read Wikipedia, Intel and Thunderbolt Technology side but I feel information is missing on the pure technological specification.

 

I write this topic after watching LG 5K Display for Mac - A PC User's Perspective from LTT. The explanation of displayport lanes into the thunderbolt is not satisfying for me. I think thunderbolt 3 can use more details.

 

Referring to the above video, how does Thunderbolt manage to channel "two displayport" and USB 3 data to the monitor ? My guess would be an encapsulation of the different protocol across the link.

 

My first investigation with Thunderbolt 3 is with USB :

I wonder how the Thunderbolt 3 was providing USB connectivity. I thought that the Thunderbolt controller could use one USB port from my chipset however it is not the case. The Thunderbolt 3 controller exposes a new PCIe USB XHCI Controller when a USB cable is connected. See screenshot. 

 

I would be curious to know what the system reports when other devices are connected like the dongles LTT reviewed.

 

Regards,
Stan

Thunderbolt 3 contains a USB 3.1 controller itself, it doesn't take in a USB connection provided by another controller or from the chipset. Attaching a Thunderbolt controller is essentially like attaching any other 3rd party USB controller to add additional ports via PCIe expansion lanes provided by the chipset or CPU, except it can also do a lot of additional things besides provide USB ports.

 

Thunderbolt 3 is connected to the system by four PCIe 3.0 lanes, giving it a maximum of 4 GB/s (32 Gbit/s) in each direction of general purpose data transfer to the system, using the Thunderbolt protocol, USB protocol, or simply exposing the PCIe lanes to the end device directly. It also has 8 lanes of DisplayPort connected to it separately, which themselves can go up to 34.56 Gbit/s combined. The DisplayPort and PCIe connection are not fully used at the same time, as the maximum datarate that the Thunderbolt controller can physically signal across the cabling system is 40 Gbit/s. This is allocated depending on what devices are attached; if all 8 lanes of DisplayPort (34.56 Gbit/s) are required, i.e. for a 5K 60 Hz monitor, then the remaining 5.5 Gbit/s is available to transfer data to the system through the PCIe lanes, whether that's USB 3.0 (aka USB 3.1 Gen 1), Thunderbolt, or PCI Express itself. If only four lanes of DisplayPort are required (17.28 Gbit/s), such as with a single 4K 60 Hz monitor, then 23 Gbit/s are left over for data transfer through the PCIe lanes.

 

https://thunderbolttechnology.net/sites/default/files/Thunderbolt3_TechBrief_FINAL.pdf

 

Quote

How Thunderbolt 3 Works

 

Fundamentally, Thunderbolt is a tunneling architecture designed to take a few underlying protocols, and combine them onto a single interface, so that the total speed and performance of the link can be shared between the underlying usages of these protocols – whether they are data, display, or something else.

 

At the physical interface level, Intel’s Thunderbolt 3 silicon builds in a few important features:

• A physical interface (PHY) layer that can dynamically switch it’s operating mode to drive either:
  • USB 2.0, 3.0, and 3.1
  • DisplayPort 1.1 and 1.2a
  • Thunderbolt at 20 and 40 Gbps
• In the Thunderbolt mode, Thunderbolt 3 port has the ability to support at least one or two (4 lane) DisplayPort interface(s), and up to 4 lanes of PCI Express Gen 3

 

Different Connection Modes

 

With Thunderbolt 3 having dynamic detection of the capabilities of the cables and devices that are plugged in, there are several modes that can be detected and activated, in a way generally transparent to the consumer.

 

USB Only Mode

 

If a USB device is plugged in, a USB host controller inside the Thunderbolt 3 enabled system is activated, and the Thunderbolt 3 silicon PHY drives USB (2.0, 3.0, or 3.1) signals to the USB-C port. In this mode, a Thunderbolt 3 port behaves exactly like a typical USB-C 3.1 enabled connector.

 

DisplayPort Only Mode

 

If a DisplayPort display or adapter is plugged in, the Thunderbolt 3 enabled system will detect this, and switch the pins driving the USB-C connector to the DisplayPort alternate mode. Thunderbolt 3 silicon will then act as a router to send raw DisplayPort traffic from the graphics engine within the system out over the USB-C connector pins and pass that DisplayPort link directly to the display or adapter.

 

In this mode, a Thunderbolt 3 enabled USB-C port will support a single four lane (4 x 5.4 Gbps, or HBR2) link of DisplayPort. These four links run across the two pairs of high speed wires in the USB-C connector and cable. This kind of DisplayPort link can support a single, uncompressed display, at 4K resolution at 60 Hz.

 

DisplayPort and USB Multi-Function Mode

 

In this alternate mode of operation, one of the high speed connector pin pairs of signals will be dedicated to DisplayPort (now 2 lanes at 5.4 Gbps) and one to USB 3.1 This allows for a basic connectivity for data and display devices such as docking stations or data and display dongles. With two lanes of DisplayPort 1.2a, resolutions Quad HD (QHD) can be achieved, or 2560 x1600 at 60 Hz.

 

Thunderbolt 3 Mode

 

If a cable and device supporting Thunderbolt are plugged in, the Thunderbolt silicon activates its highest capability mode and configures four high-speed links at either 10 Gbps or 20 Gbps (depending on cable and device support) to support the Thunderbolt transport. This provides bidirectional data rates of 20 or 40 Gbps (Figure 4 illustrates the 40 Gbps case).

 

Additionally, to fill this Thunderbolt link, the silicon extracts and routes up to 4 lanes of PCI Express Gen 3 (4 x 8 Gbps) and up to two full (4 lane) links of DisplayPort out over the Thunderbolt cable and connector to the device(s) attached downstream from the host system.

 

The first device in the Thunderbolt link has a few options of how it can use this underlying PCI Express and DisplayPort traffic:

• Consume the data from PCI Express by connecting it to PCI Express enabled devices such as networking, storage, cameras, DSP or FPGA adapters, or perhaps some new custom product
• Consume the DisplayPort links by exposing them to a display panel, or a display connector(s)
• Pass the unused PCI Express or DisplayPort data capabilities down to another Thunderbolt device(s) via a second, daisy chained Thunderbolt 3 port
• Some or all of the above

 

[...]

 

What Gets Priority?

 

Note that there are many cases where the underlying PCI Express and DisplayPort traffic will not consume the full 40 Gbps Thunderbolt 3 interface, but you will also note that in some of the configurations described, the amount of data and display together from these underlying protocols will consume more than 40 Gbps.

 

As a reminder, four lanes of PCI Express Gen 3 operate at (4 x 8 Gbps) 32 Gbps roughly. Thunderbolt 3 uses PCIe x4 gen 3 data rate with 128kB header sizes. For a single Thunderbolt chip with two ports, the x4 PCIe interface data rate is shared across the ports. Two links of (4 lane) DisplayPort 1.2 consume 2x (4 x 5.4 Gbps) or 43.2 Gbps. For both of these numbers, the underlying protocol uses some data to provide encoding overhead which is not carried over the Thunderbolt 3 link reducing the consumed bandwidth by roughly 20 percent (DisplayPort) or 1.5 percent (PCI Express Gen 3). But regardless, adding both together gets you above 40 Gbps.

 

Because a Thunderbolt 3 chip can support either one or two connectors, there is the need to provide more capability than can be used on a single connector. Many Thunderbolt 3 usages are around single connector consolidation, but there are also many consumers who want huge expansion with different devices on each port.

 

So if a consumer stresses a specific Thunderbolt port, and attempts to use dual simultaneous displays at high resolution and additional PCI Express data, the Thunderbolt 3 silicon will prioritize the display traffic first and throttle the (PCI Express) data traffic. When each DisplayPort 1.2 link is establish (via either a downstream Thunderbolt display, or via a downstream dongle to DisplayPort), the Thunderbolt silicon checks the maximum data rate that the link can demand, and ensures there is enough Thunderbolt link bandwidth available.

 

For DisplayPort 1.2, the maximum bandwidth on the Thunderbolt interface is about 17 Gbps of data, so for a 20 Gbps Thunderbolt link, a single DisplayPort 1.2 interface can be used, and for a 40 Gbps Thunderbolt link, two DisplayPort interfaces can be connected (see Figure 7).

 

After the link is established, Thunderbolt only transmits the display traffic as demanded, so even though about 17 Gbps of available data bandwidth is needed to setup a DisplayPort 1.2 connection, if the screen resolution is set to 1080p, only ~4 Gbps of the Thunderbolt link is used for display. If it is a 4K display, perhaps 14 Gbps will be consumed depending on the exact pixel count, color depth, and refresh rate.

 

If a display is plugged in exceeding the maximum number of displays available from the Thunderbolt silicon, the display is lit up, and the first display in the chain is deactivated.

 

It is important to note that the PCI Express traffic on the Thunderbolt interface is then allowed to consume the entire remainder of the link. Unlike display traffic in which a fixed resolution and color depth is equal to a fixed bandwidth, PCI Express and data traffic in general are more variable in nature. So PCI Express devices will continue to function with a variable rate of bandwidth, but certain performance levels may not be achieved if two high resolution displays are being used on the same port.

 

However, it is key to understand that the Thunderbolt interface is bidirectional, while display traffic is mostly outbound from the host system. So if the Thunderbolt link is using a PCI Express device to route traffic to the host system, that PCI Express bandwidth is mostly unaffected by outbound display traffic (outside of some flow control impacts).

 

[Kyklas]

LTT has release a new review of the Razer Core.

 

It demonstrates that the Thunderbolt 3 PCIe suffers from the path it takes thru the Chipset.

 

Could it be possible to test the Razer Core on a desktop computer with a Thunderbolt 3 PCIe card in the CPU dedicated PCIe lanes vs the Chipset PCIe lanes ?

Can the Razer Core give more performance then ?

 

Some comments on the video about using Mini-PCIe to External Graphics card are very amusing since Mini-PCIe is most likely from the Chipset PCIe lanes and that Mini-PCIe is only 1x vs 4x for the external box tested in the video.

 

Regards,
Stan