21 Jan

UCS and Cluster-on-Die

So, I have had a couple of conversations recently around the Intel v3 Haswell improvements, specifically in relation to UCS.  One thing that has come up is around Cluster-on-Die (COD).  Seems in some environments its caused a little confusion, particularly with regards to what it is, and which operating systems currently support it.  This was probably highlighted when it was noticed in the UCS release notes for 2.2(3c) below, that QPI snoop mode was supported via Service Profiles:

There are a number of good articles out there which explain what COD is, how it works etc., so I wont explain that here – but have a look at the below:

http://www.anandtech.com/show/8423/intel-xeon-e5-version-3-up-to-18-haswell-ep-cores-/4

http://techreport.com/review/27018/intel-xeon-e5-2687w-v3-processor-reviewed/2

In essence, its a mode supported on the v3 10-core processors and above, which divides the socket into two, and presents two affinity domains up to the operating system (if the OS can take advantage of that).  So for any applications which are NUMA aware, it will ensure access to local cache and therefore keep the latency down.

Now, for any operating system which doesnt currently support it, it can cause some issues.  For instance, as of now, VMware ESX (5.1/5.5) doesnt have COD support, so it can potentially mean that VMware will see additional CPU’s.  See VMware’s KB article here.

With UCS 2.2(3c), you are now able to disable this as part of the Service Profile configuration.  Im assuming we all understand the benefits of Service Profiles here, so I wont explain that.  However, the nice thing about this is that if you have a mixed workload estate across the UCS domain, that BIOS policy configuration will follow the Service Profile wherever it ends up… so you dont have to manually configure the BIOS across all the domain, then manually change the BIOS settings back when you now need to deploy an operating system which can now take advantage of the benefits of Intel’s COD.

In UCSM, you can find the configuration as part of the BIOS policy, under the QPI tab, or QPI configuration step if creating a BIOS policy from scratch:

Hope that helps.

MH

 

About

Mark is a Consulting Systems Engineer @ Cisco, focusing on all things Data Centre. All views are personal opinion, not those of Cisco.

One thought on “UCS and Cluster-on-Die

  1. Pingback: VMware don’t play nice with CoD. For now. Me neither … | vKraL2

Comments are closed.