原创文章，转载请注明： 转载自Erlang非业余研究

本文链接地址: ECUG2010分享：C1000K高性能服务器构架技术

点击这里下载 ppt
视频这里
照片这里

这次ECUG2010的讲师来自不同的国家，有点国际化的味道，也说明了这个世界越来越扁平，技术越越来越同步，我们追随技术的压力也越来越大。

原创文章，转载请注明： 转载自Erlang非业余研究

本文链接地址: CPU拓扑结构的调查

在做多核程序的时候(比如Erlang程序)，我们需要了解cpu的拓扑结构, 了解logic CPU和物理的CPU的映射关系，以及了解CPU的内部的硬件参数，比如说
L1，L2 cache的大小等信息。

Linux下的/proc/cpuinfo提供了相应的信息，但是比较不全面。 /sys/devices/system/cpu/也提供了topology结构但是比较难解读。

很多时候我们需要更专业的工具了。intel提供了这样的救助。参见： http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/

下载下来编译执行就好。

[admin@my174 cpu-topology]$ ./cpu_topology64.out

Advisory to Users on system topology enumeration

This utility is for demonstration purpose only. It assumes the hardware topology
configuration within a coherent domain does not change during the life of an OS
session. If an OS support advanced features that can change hardware topology
configurations, more sophisticated adaptation may be necessary to account for
the hardware configuration change that might have added and reduced the number
of logical processors being managed by the OS.

User should also`be aware that the system topology enumeration algorithm is
based on the assumption that CPUID instruction will return raw data reflecting
the native hardware configuration. When an application runs inside a virtual
machine hosted by a Virtual Machine Monitor (VMM), any CPUID instructions
issued by an app (or a guest OS) are trapped by the VMM and it is the VMM’s
responsibility and decision to emulate/supply CPUID return data to the virtual
machines. When deploying topology enumeration code based on querying CPUID
inside a VM environment, the user must consult with the VMM vendor on how an VMM
will emulate CPUID instruction relating to topology enumeration.

Software visible enumeration in the system:
Number of logical processors visible to the OS: 16
Number of logical processors visible to this process: 16
Number of processor cores visible to this process: 8
Number of physical packages visible to this process: 2

Hierarchical counts by levels of processor topology:
# of cores in package 0 visible to this process: 4 .
# of logical processors in Core 0 visible to this process: 2 .
# of logical processors in Core 1 visible to this process: 2 .
# of logical processors in Core 2 visible to this process: 2 .
# of logical processors in Core 3 visible to this process: 2 .
# of cores in package 1 visible to this process: 4 .
# of logical processors in Core 0 visible to this process: 2 .
# of logical processors in Core 1 visible to this process: 2 .
# of logical processors in Core 2 visible to this process: 2 .
# of logical processors in Core 3 visible to this process: 2 .

Affinity masks per SMT thread, per core, per package:
Individual:
P:0, C:0, T:0 –> 1
P:0, C:0, T:1 –> 100

Core-aggregated:
P:0, C:0 –> 101
Individual:
P:0, C:1, T:0 –> 4
P:0, C:1, T:1 –> 400

Core-aggregated:
P:0, C:1 –> 404
Individual:
P:0, C:2, T:0 –> 10
P:0, C:2, T:1 –> 1z3

Core-aggregated:
P:0, C:2 –> 1010
Individual:
P:0, C:3, T:0 –> 40
P:0, C:3, T:1 –> 4z3

Core-aggregated:
P:0, C:3 –> 4040

Pkg-aggregated:
P:0 –> 5555
Individual:
P:1, C:0, T:0 –> 2
P:1, C:0, T:1 –> 200

Core-aggregated:
P:1, C:0 –> 202
Individual:
P:1, C:1, T:0 –> 8
P:1, C:1, T:1 –> 800

Core-aggregated:
P:1, C:1 –> 808
Individual:
P:1, C:2, T:0 –> 20
P:1, C:2, T:1 –> 2z3

Core-aggregated:
P:1, C:2 –> 2020
Individual:
P:1, C:3, T:0 –> 80
P:1, C:3, T:1 –> 8z3

Core-aggregated:
P:1, C:3 –> 8080

Pkg-aggregated:
P:1 –> aaaa

APIC ID listings from affinity masks
OS cpu 0, Affinity mask 000001 – apic id 10
OS cpu 1, Affinity mask 000002 – apic id 0
OS cpu 2, Affinity mask 000004 – apic id 12
OS cpu 3, Affinity mask 000008 – apic id 2
OS cpu 4, Affinity mask 000010 – apic id 14
OS cpu 5, Affinity mask 000020 – apic id 4
OS cpu 6, Affinity mask 000040 – apic id 16
OS cpu 7, Affinity mask 000080 – apic id 6
OS cpu 8, Affinity mask 000100 – apic id 11
OS cpu 9, Affinity mask 000200 – apic id 1
OS cpu 10, Affinity mask 000400 – apic id 13
OS cpu 11, Affinity mask 000800 – apic id 3
OS cpu 12, Affinity mask 001000 – apic id 15
OS cpu 13, Affinity mask 002000 – apic id 5
OS cpu 14, Affinity mask 004000 – apic id 17
OS cpu 15, Affinity mask 008000 – apic id 7

Package 0 Cache and Thread details

Box Description:
Cache is cache level designator
Size is cache size
OScpu# is cpu # as seen by OS
Core is core#[_thread# if > 1 thread/core] inside socket
AffMsk is AffinityMask(extended hex) for core and thread
CmbMsk is Combined AffinityMask(extended hex) for hw threads sharing cache
CmbMsk will differ from AffMsk if > 1 hw_thread/cache
Extended Hex replaces trailing zeroes with ‘z#’
where # is number of zeroes (so ’8z5′ is ’0×800000′)
L1D is Level 1 Data cache, size(KBytes)= 32, Cores/cache= 2, Caches/package= 4
L1I is Level 1 Instruction cache, size(KBytes)= 32, Cores/cache= 2, Caches/package= 4
L2 is Level 2 Unified cache, size(KBytes)= 256, Cores/cache= 2, Caches/package= 4
L3 is Level 3 Unified cache, size(KBytes)= 8192, Cores/cache= 8, Caches/package= 1
+———–+———–+———–+———–+
Cache | L1D | L1D | L1D | L1D |
Size | 32K | 32K | 32K | 32K |
OScpu#| 0 8| 2 10| 4 12| 6 14|
Core |c0_t0 c0_t1|c1_t0 c1_t1|c2_t0 c2_t1|c3_t0 c3_t1|
AffMsk| 1 100| 4 400| 10 1z3| 40 4z3|
CmbMsk| 101 | 404 | 1010 | 4040 |
+———–+———–+———–+———–+

Cache | L1I | L1I | L1I | L1I |
Size | 32K | 32K | 32K | 32K |
+———–+———–+———–+———–+

Cache | L2 | L2 | L2 | L2 |
Size | 256K | 256K | 256K | 256K |
+———–+———–+———–+———–+

Cache | L3 |
Size | 8M |
CmbMsk| 5555 |
+———————————————–+

Combined socket AffinityMask= 0×5555

Package 1 Cache and Thread details

Box Description:
Cache is cache level designator
Size is cache size
OScpu# is cpu # as seen by OS
Core is core#[_thread# if > 1 thread/core] inside socket
AffMsk is AffinityMask(extended hex) for core and thread
CmbMsk is Combined AffinityMask(extended hex) for hw threads sharing cache
CmbMsk will differ from AffMsk if > 1 hw_thread/cache
Extended Hex replaces trailing zeroes with ‘z#’
where # is number of zeroes (so ’8z5′ is ’0×800000′)
+———–+———–+———–+———–+
Cache | L1D | L1D | L1D | L1D |
Size | 32K | 32K | 32K | 32K |
OScpu#| 1 9| 3 11| 5 13| 7 15|
Core |c0_t0 c0_t1|c1_t0 c1_t1|c2_t0 c2_t1|c3_t0 c3_t1|
AffMsk| 2 200| 8 800| 20 2z3| 80 8z3|
CmbMsk| 202 | 808 | 2020 | 8080 |
+———–+———–+———–+———–+

Cache | L1I | L1I | L1I | L1I |
Size | 32K | 32K | 32K | 32K |
+———–+———–+———–+———–+

Cache | L2 | L2 | L2 | L2 |
Size | 256K | 256K | 256K | 256K |
+———–+———–+———–+———–+

Cache | L3 |
Size | 8M |
CmbMsk| aaaa |
+———————————————–+

我们可以很清楚的看到我们CPU的信息，L1,L2,L3, cacheline的大小等，这些信息我们在做程序的时候经常需要的。
玩的开心！

参考文献：

1 . https://kevinclosson.wordpress.com/2009/04/22/linux-thinks-its-a-cpu-but-what-is-it-really-mapping-xeon-5500-nehalem-processor-threads-to-linux-os-cpus/

2. http://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-system-cpu

3. http://chemnitzer.linux-tage.de/2010/vortraege/shortpaper/470-slides.pdf

4. http://software.intel.com/sites/oss/pdfs/mclinux.pdf

原创文章，转载请注明： 转载自Erlang非业余研究

本文链接地址: sysbench(系统性能基准)介绍

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.
Current features allow to test the following system parameters:
* cpu性能
* file I/O performance
* scheduler performance
* mutex的性能
* memory allocation and transfer speed, 支持hugepage
* POSIX threads implementation performance
* database server performance (OLTP benchmark) 支持mysql，pgsql, oracle

项目地址是：http://sysbench.sourceforge.net/

从源码来看这个项目做的非常的模块化。 sysbench提供了诸如读取配置， 创建线程， 日志， 计时和模块化框架，支持的测试模式都是通过插件方式加入到框架去的。

用户很容易扩展相应的模块， 通常模块只需要关注自己要实现的测试功能，其他的事情由框架来做，很大的方便用户自己编写特定的测试模块。其他的如多线程什么的都无需自己去考虑。

在数据库的驱动方面，目前提供了mysql，pgsql, oracle的驱动。在数据库抽象方面也是模块的，用户自己也能容易加入自己的数据库支持。

作为一个轻量级别的bench工具，在系统的系统测量方面，可以了解到对系统运行产生很大影响的性能，如内存，cpu，磁盘，锁，线程调度，数据库等方面的信息，是一个得心应手的工具。

在ubuntu下可以用apt-get install sysbench来安装，具体的使用参看 man sysbench。

玩的开心！