这个课程最主要focus在数据的局部性原理，从硬件到操作系统到应用程序这样的顺序过来的，对于我们提高核心系统软件的性能非常有启发意义.

课件下载点击这里

以下是教程的介绍:
课程简介
________________________________________
　　在计算机系统中，数据的存储、访问和传输已成为阻碍系统性能的主要瓶颈，访问数据的速度远远低于计算的速度，而对数据访问的需求已经成为计算机运行的主要部分。 数据在计算机系统中的存储层次也变得越来越深：从硬件快速寄存器(Registers)到快速存储器(Cache)再到主存的DRAM，通过数据总线，系统可以与各种磁盘，如半导体的固态存储器（Solid State Device）或机械的硬磁盘等。通过互联网，计算机之间还可以通过网络进行大规模的远程存储、访问和通讯。
　　计算机系统中的数据访问的速度和效率是由两个基本要素决定的：（1）点与点之间的传输的速度（Bandwidth）；（2）在某点读或写数据的时间延迟（Latency）。在过去的20多年里，在计算机的各个不同的联结点之间数据的传输速度，有了很大的提高，但数据访问的延迟却没能有效地改善。为了解决这一对越来越突出的矛盾，在计算机系统设计和实现中，三种最有效的硬件或软件的方法是：缓存（Caching），备份（Replication）和预取（Prefetching）。
课程大纲
________________________________________
　　这门课以上面提到的三个基本技术为核心，全面介绍计算机各个存储层面上的最有代表性和最先进的数据管理方法。课程安排如下：

（一） 硬件快速存储器（Cache）的设计与改进
（1） 基本结构
（2） 命中率和访问延迟的取舍
（3） 高命中率和低访问延迟Cache的设计
（4） 多核处理器(Multicore)中的Cache管理
（二） 主存DRAM内局部性（Locality）的挖掘
（1） DRAM缓存区(row buffer)的结构
（2） 在DRAM内局部性丢失的原因
（3） 一种保留局部性的内存地址空间的映射方法
（4） 嵌入Cache 的DRAM的设计与系统应用
（三） 内存和磁盘管理中的核心技术：替换算法
（1） LRU算法优点以及难以解决的问题
（2） LIRS算法是如何解决LRU问题的
（3） Clock-pro: LIRS是如何实现在操作系统内核的
（4） BP-wrapper:消除替换算法在系统实现中的同步竟争
（四） 提高操作系统对磁盘的管理功能和效率
（1） 操作系统对磁盘管理的局限性
（2） 扩大操作系统的视野去获得关键的磁盘数据存储地址信息
（3） DULO-Caching和DULO-Prefetchin:感知磁盘数据分布的缓存(Caching)和预取(Prefetching)方法以及系统实现
（五） 固态闪存系统(Solid State Device Flash Memory)
（1） SSD Flash Memory的结构和性能
（2） SmartSaver: 利用闪存来降低磁盘能耗
（3） SSD的读、写，以及缓存的高效管理
（六） 在互联网和无线网上的数据管理和有效传输
（1） Stretched Exponential Distribution: 为什么P2P在互联网上是传输多媒体的一种最有效的方法？
（2） 数据缓存在无线网上的作用

sshpass -noninteractive ssh password provider.

在ubuntu下可以用 apt-get install sshpass来安装。

它支持密码从命令行，文件， 环境变量中读取。

有需要的同学实验下吧。 特别是erlang支持自定义的rsh, 参数是erl -rsh xxxxx

OneCached supports the set, add, replace, get, incr, decr, delete, flush_all and quit commands. It doesn’t handle expiration time.

You call pull the source code from the public repository at: https://git.process-one.net/onecached

git clone git://git.process-one.net/onecached/mainline.git

To compile, just run make, and to start, just type:
bin/onecachedctl start

这个项目是非常简单的一个memcached的模仿， 亮点在于底层用ets, dest, mnesia这样的erlang特有的存储。这个项目的意义在于演示了如何编写一个严肃的Erlang工程，以及需要掌握的技能。对于初学者来讲，这是很好的一个切入点。 你可以在上面修修改改， 打造出你自己的版本。

玩的开心！

为了解决这个问题，Erlang的团队推出了halfword-emulator, 通过修改Etrem指针的长度， 来达到节省内存，加速的目的。
我们可以在编译的时候通过 configure –enable-halfword-emulator 来启用这个VM.

half word 64-bit Erlang VM有以下特性
– 4 Gbytes process heaps (in total)
–max size of Erlang term 4 Gbytes
– ets tables and binaries in separate space can utilize the full 64 bit address space

但是需要注意的是，这个patch推出的时间不长，可能存在风险。 在提高速度的同时，也有不稳定的风险。 用户需要自己去评估。

项目地址是：http://sysbench.sourceforge.net/

从源码来看这个项目做的非常的模块化。 sysbench提供了诸如读取配置， 创建线程， 日志， 计时和模块化框架，支持的测试模式都是通过插件方式加入到框架去的。

用户很容易扩展相应的模块， 通常模块只需要关注自己要实现的测试功能，其他的事情由框架来做，很大的方便用户自己编写特定的测试模块。其他的如多线程什么的都无需自己去考虑。

在数据库的驱动方面，目前提供了mysql，pgsql, oracle的驱动。在数据库抽象方面也是模块的，用户自己也能容易加入自己的数据库支持。

作为一个轻量级别的bench工具，在系统的系统测量方面，可以了解到对系统运行产生很大影响的性能，如内存，cpu，磁盘，锁，线程调度，数据库等方面的信息，是一个得心应手的工具。

在ubuntu下可以用apt-get install sysbench来安装，具体的使用参看 man sysbench。

玩的开心！

IO架构图：

内存管理图：

很容易误解的socket buffer：

需要进一步了解Tsung的同学 点这里看官方的文档!
想了解如何使用的还可以到Erlang china这里看看!

— HIGHLIGHTS ———————————————————-

OTP-8217 == erts stdlib compiler ==

The module binary from EEP31 (and EEP9) is implemented.

OTP-8485 == common_test ==

It is now possible for the user to provide specific callback
modules that handle test configuration data, so that data on
arbitray form can be accessed (e.g. by reading files or by
communicating with a configuration server process). Two
default callback modules have been introduced in Common Test:
ct_config_plain and ct_config_xml. The former is used to
handle the traditional Common Test configuration files (with
terms on key-value tuple form) and the latter to handle
configuration data on XML representation.

OTP-8555 == erts ==

New NIF features:

Send messages from a NIF, or from thread created by NIF, to
any local process (enif_send)

Store terms between NIF calls (enif_alloc_env,
enif_make_copy)

Create binary terms with user defined memory management
(enif_make_resource_binary)

And some incompatible changes made to the API. For more
information see the warning text in erl_nif(3).

OTP-8623 == compiler erts hipe stdlib ==

Receive statements that can only read out a newly created
reference are now specially optimized so that it will execute
in constant time regardless of the number of messages in the
receive queue for the process. That optimization will benefit
calls to gen_server:call(). (See gen:do_call/4 for an example
of a receive statement that will be optimized.)

OTP-8650 == common_test ==

The run_test script has been replaced by a program (with the
same name) which can be executed without explicit
installation. The start flags are the same as for the legacy
start script.

OTP-8706 == erts hipe tools ==

eprof has been reimplemented with support in the Erlang
virtual machine and is now both faster (i.e. slows down the
code being measured less) and scales much better. In
measurements we saw speed-ups compared to the old eprof
ranging from 6 times (for sequential code that only uses one
scheduler/core) up to 84 times (for parallel code that uses 8
cores).

Note: The API for the eprof has been cleaned up and extended.
See the documentation.

期待R14B有大的动作！

;;(keyboard-translate ?\C-h ?\C-?)
(defalias 'yes-or-no-p 'y-or-n-p)

(setq time-stamp-active t)
(setq time-stamp-warn-inactive t)
(setq time-stamp-format "%:y-%02m-%02d %3a %02H:%02M:%02S")
(add-hook 'write-file-hooks 'time-stamp)

(global-set-key [f2] 'rgrep)
(global-set-key [(f5)] 'eshell)
(global-set-key [C-f5] 'shell)
(global-set-key (kbd "M-3") 'capitalize-word)

(global-set-key [f6] 'find-file)
(global-set-key [C-Z] nil)
(global-set-key [C-f4] 'kill-this-buffer)

(global-set-key [(f4)] 'ibuffer)
(global-set-key [(f8)] 'other-window)

(require 'color-theme)
;;(color-theme-initialize)
(color-theme-dark-blue2)
(setq font-lock-verbose t)
;;(set-default-font "Bitstream Vera Sans Mono-12")
(global-font-lock-mode t)
(tool-bar-mode -1)
(setq column-number-mode t)
(setq display-battery-mode t)
(setq size-indication-mode t)
(setq default-major-mode 'outline-mode)
(add-hook 'outline-mode-hook 'turn-off-auto-fill)

(setq transient-mark-mode t)
(setq-default shell-cd-regexp nil)
(setq-default shell-pushd-regexp nil)
(setq-default shell-popd-regexp nil)
(setq inhibit-startup-message t)
(setq visible-bell t)
(setq mouse-yank-at-point t)
(setq kill-ring-max 100)
(setq default-fill-column 60)

(global-set-key (kbd "M-g") 'goto-line)
(global-set-key (kbd "M-o") 'ff-find-related-file)

(setq require-final-newline t)

(setq track-eol t)

(setq gnus-inhibit-startup-message t)

(setq enable-recursive-minibuffers t)
(setq auto-save-mode nil)
(setq Man-notify-method 'pushy)

(display-time-mode 1)
(setq display-time-24hr-format t)
(setq display-time-day-and-date t)
(setq display-time-use-mail-icon t)
(setq display-time-interval 10)

(tool-bar-mode -1)
(customize-set-variable 'scroll-bar-mode 'right)

(setq speedbar-show-unknown-files t)
(setq speedbar-update-flag nil)
(setq speedbar-use-images nil)
(setq speedbar-verbosity-level 0)
(global-set-key [f9] 'speedbar)

;; set paren mode
(show-paren-mode t)
(setq show-paren-style 'parentheses)
(mouse-avoidance-mode 'animate)

;; set the default username and email
(setq user-full-name "yufeng")
(setq user-mail-address "mryufeng@gmail.com")

;; set the title name
(setq frame-title-format "Emacs - %f")

;;-------------------------Shell 使用 ansi color-------------
(autoload 'ansi-color-for-comint-mode-on "ansi-color" nil t)
(add-hook 'shell-mode-hook 'ansi-color-for-comint-mode-on)

(require 'uniquify)
(setq
  uniquify-buffer-name-style 'post-forward
  uniquify-separator ":")

;;
(require 'ibuffer)
(global-set-key (kbd "C-x C-b") 'ibuffer)
(require 'ido)
(ido-mode t)

;;-------------------------让Emacs拥有tabs-------------------
(require 'tabbar)
(setq tabbar-buffer-groups-function
          (lambda ()
            (list "All")))

(tabbar-mode)
(global-set-key [(control shift tab)] 'tabbar-backward)
(global-set-key [(control tab)] 'tabbar-forward)
(global-set-key (kbd "M-1") 'tabbar-backward)
(global-set-key (kbd "M-2") 'tabbar-forward) 

;;(require 'session)
;;(add-hook 'after-init-hook 'session-initialize)
;;(load "desktop")
;;(desktop-load-default)
;;(desktop-read)
;;(setq desktop-save-mode t)

(require 'browse-kill-ring)
(global-set-key [(control c)(k)] 'browse-kill-ring)
(browse-kill-ring-default-keybindings)

;;erlang
;;(require 'erlang-eunit)
;;(load "erlang_appwiz" t nil)
(global-set-key [f3] 'erlang-next-error)
(global-set-key [C-f7] 'erlang-compile)
(global-set-key [f7] 'compile)
(global-set-key [f10] 'linum-mode)

;;distel
(add-to-list 'load-path "/usr/local/share/distel/elisp")
(require 'distel)
(distel-setup)
;; Some Erlang customizations
(add-hook 'erlang-mode-hook
	    (lambda ()
	          ;; when starting an Erlang shell in Emacs, default in the node name
`	          (setq inferior-erlang-machine-options '("-sname" "emacs"))
		      ;; add Erlang functions to an imenu menu
		      (imenu-add-to-menubar "imenu")))

;; A number of the erlang-extended-mode key bindings are useful in the shell too
(defconst distel-shell-keys
  '(("\C-\M-i"   erl-complete)
    ("\M-?"      erl-complete)
    ("\M-."      erl-find-source-under-point)
    ("\M-,"      erl-find-source-unwind)
    ("\M-*"      erl-find-source-unwind)
    )
  "Additional keys to bind when in Erlang shell.")

(add-hook 'erlang-shell-mode-hook
	    (lambda ()
	          ;; add some Distel bindings to the Erlang shell
	          (dolist (spec distel-shell-keys)
		          (define-key erlang-shell-mode-map (car spec) (cadr spec)))))

;wrangler
(add-to-list 'load-path "/usr/local/share/wrangler/elisp")
(require 'wrangler) 

;;ecb
(require 'xcscope)
(require 'cedet) ;;load cedet
(require 'ecb) ;;load ecb

(setq ecb-tip-of-the-day nil
inhibit-startup-message t
ecb-auto-compatibility-check nil
ecb-version-check nil)

(global-set-key [f12] 'ecb-activate)
(global-set-key [C-f12] 'ecb-deactivate)
(global-set-key [f11] 'delete-other-windows)
(global-set-key [(meta return)] 'semantic-ia-complete-symbol-menu)
(global-set-key (kbd "C-c e") 'ecb-goto-window-edit-last)
(global-set-key (kbd "C-c s") 'ecb-goto-window-sources)
(global-set-key (kbd "C-c d") 'ecb-goto-window-directories)
(global-set-key (kbd "C-c m") 'ecb-goto-window-methods)
(global-set-key (kbd "C-c h") 'ecb-goto-window-history)
(global-set-key (kbd "C-c r") 'ecb-redraw-layout)
(global-set-key (kbd "C-.") 'cscope-find-global-definition-no-prompting)
(global-set-key (kbd "C-,") 'cscope-pop-mark)
(custom-set-variables
  ;; custom-set-variables was added by Custom.
  ;; If you edit it by hand, you could mess it up, so be careful.
  ;; Your init file should contain only one such instance.
  ;; If there is more than one, they won't work right.
 '(ecb-options-version "2.32")
 '(safe-local-variable-values (quote ((erlang-indent-level . 2)))))
(custom-set-faces
  ;; custom-set-faces was added by Custom.
  ;; If you edit it by hand, you could mess it up, so be careful.
  ;; Your init file should contain only one such instance.
  ;; If there is more than one, they won't work right.
 )

;;msf abbrev
(add-to-list 'load-path "~/emacs")
;; ensure abbrev mode is always on
(setq-default abbrev-mode t)
;; do not bug me about saving my abbreviations
(setq save-abbrevs nil)
;; load up modes I use
(require 'cc-mode)
;;(require 'perl-mode)
;;(require 'cperl-mode)
;;(require 'sh-script)
;;(require 'shell)
;;(require 'lua-mode)
;;(require 'erlang)
;; load up abbrevs for these modes
(require 'msf-abbrev)
(setq msf-abbrev-verbose t) ;; optional
(setq msf-abbrev-root "~/emacs/mode-abbrevs")
(global-set-key (kbd "C-c l") 'msf-abbrev-goto-root)
(global-set-key (kbd "C-c a") 'msf-abbrev-define-new-abbrev-this-mode)
(msf-abbrev-load)

;;go mode
(require 'go-mode-load)
(require 'tramp)

;;gud
(add-hook 'gdb-mode-hook '(lambda ()
                            (define-key c-mode-base-map [(f5)] 'gud-go)
                            (define-key c-mode-base-map [(f7)] 'gud-step)
                            (define-key c-mode-base-map [(f8)] 'gud-next)))

(defvar no-easy-keys-minor-mode-map (make-keymap)
  "no-easy-keys-minor-mode keymap.")
(let ((f (lambda (m)
           `(lambda () (interactive)
              (message (concat "No! use " ,m " instead."))))))
  (dolist (l '(("<left>" . "C-b") ("<right>" . "C-f") ("<up>" . "C-p")
               ("<down>" . "C-n")
               ("<C-left>" . "M-f") ("<C-right>" . "M-b") ("<C-up>" . "M-{")
               ("<C-down>" . "M-}")
               ("<M-left>" . "M-f") ("<M-right>" . "M-b") ("<M-up>" . "M-{")
               ("<M-down>" . "M-}")
               ("<delete>" . "C-d") ("<C-delete>" . "M-d")
               ("<M-delete>" . "M-d") ("<next>" . "C-v") ("<C-next>" . "M-x <")
               ("<prior>" . "M-v") ("<C-prior>" . "M-x >")
               ("<home>" . "C-a") ("<C-home>" . "M->")
               ("<C-home>" . "M-<") ("<end>" . "C-e") ("<C-end>" . "M->")))
    (define-key no-easy-keys-minor-mode-map
      (read-kbd-macro (car l)) (funcall f (cdr l)))))
(define-minor-mode no-easy-keys-minor-mode
  "A minor mode that disables the arrow-keys, pg-up/down, delete
  and backspace."  t " no-easy-keys"
  'no-easy-keys-minor-mode-map :global t)
(no-easy-keys-minor-mode 1)

(require 'smex)
(smex-initialize)

(global-set-key (kbd "M-x") 'smex)
(global-set-key (kbd "M-X") 'smex-major-mode-commands)

(require 'org-install)
(add-to-list 'auto-mode-alist '("\\.org$" . org-mode))
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map "\C-ca" 'org-agenda)
(setq org-log-done t)

(setq transient-mark-mode t) ; highlight text selection
(setq delete-selection-mode t) ; delete seleted text when typing

; highlight current line
(require 'hl-line)
(global-hl-line-mode t)

我们在源码里面发现了以下注释：
/*
* The native implementation functions for the module binary.
* Searching is implemented using aither Boyer-More or Aho-Corasick
* depending on number of searchstrings (BM if one, AC if more than one).
* Native implementation is mostly for efficiency, nothing
* (except binary:referenced_byte_size) really *needs* to be implemented
* in native code.
*/

这个模块兼顾了效率和方便性，使用起来就大大简化了代码的复杂度，有福气了。