Home > Erlang探索 > 最快的Erlang http hello world 服务器调优指南 (20Khttp短链接请求/S每桌面CPU)

最快的Erlang http hello world 服务器调优指南 (20Khttp短链接请求/S每桌面CPU)

November 4th, 2009

erl的虚拟机有2种方式 plain版本的和smp版本的。 smp版本由于锁的开销相比要比plain版本的慢很多。而32位机器由于内存访问比64位的少,也会快出很多。所有我选择在32位的linux系统下调优这个httpd服务器。这个服务器就是实现个简单的功能,在browser下返回hello world。以下我们会先编译我们的优化版本的虚拟机,然后再分别测试R13B02的标准版本的和我们优化版的性能:

root@nd-desktop:/build_opt_plain# uname -a
Linux nd-desktop 2.6.31-14-generic #3 SMP Sun Nov 1 23:03:10 CST 2009 i686 GNU/Linux

#准备开发环境
root@nd-desktop:/# apt-get build-dep erlang

#下载otp R13B02-1源码包
root@nd-desktop:/# wget http://www.erlang.org/download/otp_src_R13B02-1.tar.gz

#解开patch包
root@nd-desktop:/# tar xzvf build_opt_plain.tar.gz

#解开源码包
root@nd-desktop:/# tar xzf otp_src_R13B02-1.tar.gz

#打补丁
root@nd-desktop:/# cd otp_src_R13B02-1
root@nd-desktop:/otp_src_R13B02-1# patch -p1 <../build_opt_plain/otp_src_R13B02-1_patch_by_yufeng
patching file erts/emulator/beam/erl_binary.h
patching file erts/emulator/beam/erl_process.c
patching file erts/emulator/beam/sys.h
patching file erts/emulator/drivers/common/inet_drv.c
patching file erts/preloaded/src/Makefile
patching file erts/preloaded/src/prim_inet.erl
patching file lib/asn1/src/Makefile
patching file lib/hipe/Makefile
patching file lib/parsetools/src/Makefile
root@nd-desktop:/otp_src_R13B02-1# ../build_opt_plain/build.plain
。。。

如果编译都没有任何错误的话, 就大功告成了。

好 现在我们开始性能比较:

#先加大文件句柄数

root@nd-desktop:/otp_src_R13B02-1# cd ../build_opt_plain
root@nd-desktop:/build_opt_plain# ulimit -n 99999

#标准发布版本
root@nd-desktop:/build_opt_plain# erlc ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 erl +K true +h 99999  +P 99999 -smp enable +S 2:1 -s ehttpd
Erlang R13B03 (erts-5.7.4) [source][/source] [smp:2:1] [rq:2] [async-threads:0] [hipe] [kernel-poll:true]

ehttpd ready with 2 schedulers on port 8888
Eshell V5.7.4  (abort with ^G)
1>

#在另外的一台机器上发动ab测试

[root@localhost src]# ab -c 60 -n 100000 http://192.168.235.147:8888/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.235.147 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:       
Server Hostname:        192.168.235.147
Server Port:            8888

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      60
Time taken for tests:   8.925945 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      5100051 bytes
HTML transferred:       1200012 bytes
Requests per second:    11203.29 [#/sec] (mean)
Time per request:       5.356 [ms] (mean)
Time per request:       0.089 [ms] (mean, across all concurrent requests)
Transfer rate:          557.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  65.7      0    3001
Processing:     0    3   1.5      4       7
Waiting:        0    2   1.8      4       6
Total:          0    4  65.8      4    3007
WARNING: The median and mean for the waiting time are not within a normal deviation
        These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      5
  95%      5
  98%      5
  99%      5
100%   3007 (longest request)

标准smp版本1个CPU的结果是: 11203.29 [#/sec] (mean)

#启用hipe的标准版本

root@nd-desktop:/build_opt_plain# erlc +native +"{hipe, [o3]}" ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 erl +K true +h 99999  +P 99999 -smp enable +S 2:1 -s ehttpd
Erlang R13B03 (erts-5.7.4) [source][/source] [smp:2:1] [rq:2] [async-threads:0] [hipe] [kernel-poll:true]

ehttpd ready with 2 schedulers on port 8888
Eshell V5.7.4  (abort with ^G)
1>

标准smp hipe版本1个CPU结果是: 12390.32 [#/sec] (mean)

#我们的优化版本

root@nd-desktop:/build_opt_plain#  ../otp_src_R13B02-1/bin/erlc  ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1   ../otp_src_R13B02-1/bin/erl +K true +h 99999  +P 99999   -s ehttpd
Erlang R13B02 (erts-5.7.3) [source][/source] [rq:1] [hipe] [kernel-poll:true]

ehttpd ready with 1 schedulers on port 8888
Eshell V5.7.3  (abort with ^G)
1>

优化版本单个cpu: 19662.37 [#/sec] (mean)

#启用hipe的优化版本

root@nd-desktop:/build_opt_plain#  ../otp_src_R13B02-1/bin/erlc +native +"{hipe, [o3]}"  ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1   ../otp_src_R13B02-1/bin/erl +K true +h 99999  +P 99999   -s ehttpd
Erlang R13B02 (erts-5.7.3) [source][/source] [rq:1] [hipe] [kernel-poll:true]

ehttpd ready with 1 schedulers on port 8888
Eshell V5.7.3  (abort with ^G)
1>

优化版本启用hipe单个cpu:20090.83 [#/sec] (mean)

附上我们的最小的高性能的http echo 服务器:

root@nd-desktop:/build_opt_plain# cat ehttpd.erl
-module(ehttpd).
-compile(export_all).


start() ->
    start(8888).
start(Port) ->
    N = erlang:system_info(schedulers),
    listen(Port, N),
    io:format("ehttpd ready with ~b schedulers on port ~b~n", [N, Port]),

    register(?MODULE, self()),
    receive Any -> io:format("~p~n", [Any]) end.  %% to stop: ehttpd!stop.

listen(Port, N) ->
    Opts = [{active, false},
            binary,
            {backlog, 256},
            {packet, http_bin},
            {raw,6,9,<<1:32/native>>}, %defer accept
            %%{delay_send,true},
            %%{nodelay,true},
            {reuseaddr, true}],

    {ok, S} = gen_tcp:listen(Port, Opts),
    Spawn = fun(I) ->    
                    register(list_to_atom("acceptor_" ++ integer_to_list(I)),
                             spawn_opt(?MODULE, accept, [S, I], [link, {scheduler, I}]))
            end,
    lists:foreach(Spawn, lists:seq(1, N)).

accept(S, I) ->
    case gen_tcp:accept(S) of
        {ok, Socket} -> spawn_opt(?MODULE, loop, [Socket], [{scheduler, I}]);
        Error    -> erlang:error(Error)
    end,
    accept(S, I).

loop(S) ->
    case gen_tcp:recv(S, 0) of
        {ok, http_eoh} ->
            Response = <<"HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\nhello world!">>,
            gen_tcp:send(S, Response),
            gen_tcp:close(S),
            ok;

        {ok, _Data} ->
            loop(S);

        Error ->
            Error
    end.

这个服务器是最小的,单是在多处理器和单cpu上都有非常好的性能。

root@nd-desktop:/build_opt_plain# cat /proc/cpuinfo
model name      : Pentium(R) Dual-Core  CPU      E5200  @ 2.50GHz

注:这个http服务器基本上是在c的程序跑,erlang的代码执行的很少, 所以hipe的提升效果不是很明显。对于复杂的业务,应该是有很大的帮助的。

附件里是用到的脚本和补丁。

我们可以得出结论:
hipe启用要比不启用快。
优化版本的和标准版本的 20090:11203, 性能提高了将近80% 还是非常可观的。

点击下载附件

Post Footer automatically generated by wp-posturl plugin for wordpress.

Categories: Erlang探索 Tags: , , , ,
  1. November 4th, 2009 at 22:50 | #1

    Hi Yufeng, great article about benchmarking Erlang. Where can I download your Erlang patch? What kind of changes did you make exactly? I’m very interested to learn more about it.

  2. November 25th, 2009 at 17:26 | #2

    如果把 ehttpd的http报文分析去掉。

    也就是说 {packet,http}改成{packet,raw} 还的快些。

  3. shellxu
    September 7th, 2010 at 21:54 | #3

    我试验了一下这个ehttpd,在真实环境下,不到1000次请求/秒就挂掉了
    ping服务器出现timeout,应该是出现网络阻塞了。不知具体原因出在哪里?

    Yu Feng Reply:

    你是如何配置你的环境的。换句话说把你的环境贴上来。。。

  4. dzeda
    September 14th, 2010 at 20:18 | #4

    请问这个ehttpd为什么不能在Linux后台运行呢?
    试过了detatch,noshell,noinput 参数都不可以。

    Yu Feng Reply:

    erl -detached 注意拼写!

  5. dzeda
    September 15th, 2010 at 09:07 | #5

    当然是 -detached,请问你自己试过没有呢?

  6. alex
    January 21st, 2011 at 13:53 | #6

    请问本文提供的patch在 Erlang 14B 下有没有相应的版本?

    Yu Feng Reply:

    还没有,有兴趣可以联系我。。。

  7. July 12th, 2012 at 17:15 | #7

    这个环境下,每秒2万个短连接 并不算高啊!!!

    Yu Feng Reply:

    现在16核可以上到30W这个有点高了吧

Comments are closed.