原创文章，转载请注明： 转载自系统技术非业余研究
本文链接地址: erlang高级原理和应用PPT
公司培训用的 凑合看吧 主要讲erlang系统的特点,分布集群以及mnesia的使用, 从比较高的角度来看erlang, 让你有了大体观.
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
原文地址：http://hyperstruct.net/2007/6/26/literal-xml-in-erlang-with-parse-transform-2
One of the things I dislike about Erlang is that it severely impairs bragging opportunities. Yesterday I wrote a module that allows writing literal XML in the source and have it parsed into Erlang structures at compile time—sort of like E4X minus the manipulation goodies at runtime (at least for now).
You write:
Doc = '<greeting>Hello!</greeting>',
io:format("~p~n", [Doc]).
And it prints…
{xmlElement,greeting,
greeting,
[],
{xmlNamespace,[],[]},
[],
1,
[],
[{xmlText,[{greeting,1}],1,[],"Hello!",text}],
[],
"/tmp",
undeclared}
In most languages I’m familiar with, this would have granted the author instant Yacc-demigod status. With Erlang… it was less than 40 LOC. Hardly something you’d wear at a party.
Anyway, this code owes everything to Philip’s writings. It also uses parse_transform/2, and “programmers are strongly advised not to engage in parse transformations and no support is offered for problems encountered”. So unless you, like me, are still at the kid-in-a-candy-shop stage of Erlang experience, think twice before using this in production, ok?
The code is here http://repo.hyperstruct.net/inline_xml/
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
本文链接地址: 如何用gdb调试erlang运行期（高级）
前些天在erlang的源码里面闲逛的时候发现了 bin目录下的cerl，一看原来是个调试的高级货。
我之前写过一篇文章http://mryufeng.javaeye.com/blog/113852 如何调试erlang 但是这是土八路的方法， cerl才是现代化工业。
# This is a script to start Erlang/OTP for debugging. PATH is set to
# include this script so if slave nodes are started they will use this
# script as well.
#
# usage: cerl [ OPTIONS ] [ ARGS ]
#
# The OPTIONS are
#
# -rootdir $MYROOTDIR
# Run an installed emulator built from this source
# -debug Run debug compiled emulator
# -gdb Run the debug compiled emulator in emacs and gdb.
# You have to start beam in gdb using “run”.
# -break F Run the debug compiled emulator in emacs and gdb and set break.
# The session is started, i.e. “run” is already don for you.
# -xxgdb FIXME currently disabled
# -purify Run emulator compiled for purify
# -quantify Run emulator compiled for quantify
# -purecov Run emulator compiled for purecov
# -gcov Run emulator compiled for gcov
# -valgrind Run emulator compiled for valgrind
# -lcnt Run emulator compiled for lock counting
# -nox Unset the DISPLAY variable to disable us of X Windows
#
要使用cerl 我们最好准备个调试版本的erlang。R13B 修正了些编译错误,可以编译出debug版本的系统：./configure && make TYPE＝debug && make
这样就生成了个beam.debug的运行期。
我们要调试的时候 就可以在otp的binx目录下运行 cerl -debug -gdb -break main
这时候cerl自动会加载 emacs 启动 gud, 整个过程都是自动的。但是这个脚本有点小问题， gud模型下没有把源码和当前对应的调试对应起来。可以通过以下方式修正：
yu-fengdemacbook-2:bin yufeng$ diff cerl cerl2 284c284 < exec $EMACS --eval "(progn (gdb \"gdb $EMU\") $gdbcmd)" --- > exec $EMACS --eval "(progn (gdb \"gdb --annotate=3 $EMU\") $gdbcmd)"
具体的操作和界面可以参照这篇文章：
http://www.nabble.com/printing-of-Eterm%27s-from-gdb-td19240493.html
在调试的时候 我们会希望查看 eterm的值，但是由于eterm的格式非常复杂， gdb的print什么的无法满足我们的需求。 otp开发团队于是开发出了一套方法来减轻我们的负担：
1. erts/etc/unix/etp-commands 这是gdb的脚本 包含了几十个etp方法，而且文档非常详细。
2. 源码里面的 pp， ps等函数， 这些函数是专门为gdb调试开发的。 可以用gdb的 call pp(xxx)来调用。
有了这些工具 调试和研究erts变成了一件很快乐的事情！
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
本文链接地址: binary的常量优化
erlang的binary在这个网络程序里面占着非常重要的地位，所以otp团队采用了非常多的优化手段包括：
1. binary操作对应着 opcode
2. 根据生命期和作用，有4种类型的binary
3. hipe优化，把bs_操作直接翻译成asm指令
4. 编译器层面消除无必须的操作。
下面的例子就是演示4的特性：
yu-fengdemacbook-2:~ yufeng$ cat bin.erl
-module(bin).
-export([start/1]).
start(A)->
B1= <<12>>,
B2 = <<B1/binary, 5.0/float>>,
B3= <<B2/binary, "yes">>,
% B3的值是预先可以知道的， 无需一步步的构造
<<"abcd", 3:32,B3:128/binary,_/binary>> = <<"abcd1234",A/binary,2:32, 8773:64, "a", 5.0/float>>.
％ 2:32, 8773:64, "a", 5.0/float 这些都是预先知道的 直接翻译成二进制流。
yu-fengdemacbook-2:~ yufeng$ erlc +"'S'" bin.erl yu-fengdemacbook-2:~ yufeng$ cat bin.S
{module, bin}. %% version = 0
{exports, [{module_info,0},{module_info,1},{start,1}]}.
{attributes, []}.
{labels, 8}.
{function, start, 1, 2}.
{label,1}.
{func_info,{atom,bin},{atom,start},1}.
{label,2}.
{move,{integer,0},{x,1}}.
{gc_bif,byte_size,{f,0},2,[{x,0}],{x,2}}.
{bs_add,{f,0},[{x,1},{x,2},1],{x,1}}.
{bs_add,{f,0},[{x,1},{integer,29},1],{x,1}}.
{bs_init2,{f,0},{x,1},0,1,{field_flags,[]},{x,1}}.
{bs_put_string,8,{string,"abcd1234"}}.
{bs_put_binary,{f,0},{atom,all},8,{field_flags,[unsigned,big]},{x,0}}.
％％ 一步到位
{bs_put_string,21,
{string,[0,0,0,2,0,0,0,0,0,0,34,69,97,64,20,0,0,0,0,0,0]}}.
{test,bs_start_match2,{f,3},[{x,1},2,0,{x,0}]}.
{test,bs_match_string,{f,3},[{x,0},64,{string,[97,98,99,100,0,0,0,3]}]}.
％％ 一步到位
{test,bs_get_binary2,
{f,3},
[{x,0},
2,
{integer,128},
8,
{field_flags,[{anno,[8,{file,"./bin.erl"}]},unsigned,big]},
{x,2}]}.
{test,bs_skip_bits2,
{f,3},
[{x,0},
{atom,all},
8,
{field_flags,[{anno,[8,{file,"./bin.erl"}]},unsigned,big]}]}.
{test,is_eq_exact,
{f,3},
[{x,2},{literal,<<12,64,20,0,0,0,0,0,0,121,101,115>>}]}.
％％ 一步到位
{move,{x,1},{x,0}}.
return.
{label,3}.
{badmatch,{x,1}}.
{function, module_info, 0, 5}.
{label,4}.
{func_info,{atom,bin},{atom,module_info},0}.
{label,5}.
{move,{atom,bin},{x,0}}.
{call_ext_only,1,{extfunc,erlang,get_module_info,1}}.
{function, module_info, 1, 7}.
{label,6}.
{func_info,{atom,bin},{atom,module_info},1}.
{label,7}.
{move,{x,0},{x,1}}.
{move,{atom,bin},{x,0}}.
{call_ext_only,2,{extfunc,erlang,get_module_info,2}}.
所以我们在使用binary的时候， 尽可能的利用这个特性。
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
本文链接地址: cover的原理及其启示
我们先看下cover模块的功能：
The module cover provides a set of functions for coverage analysis of Erlang programs, counting how many times each executable line of code is executed when a program is run.
那它是如何做到的呢？
它是这样实现的： cover一个模块的时候要先编译， 这个过程中， 根据模块的abstract code 里面的行号，在每个有效的语句前面插入一个 ets:update_couter() 语句，这样编译出来的模块运行的时候，我们就可以收集到每个有效行的运行信息。
我hack了下cover.erl：
yu-fengdemacbook-2:src yufeng$ diff cover.erl cover_orig.erl
1242,1243c1242
< {ok, Module, Binary} = compile:forms(Forms, [debug_info]),
< io:format("abstract code: ~n~p~n", [get_abstract_code(Module, Binary)]),
---
> {ok, Module, Binary} = compile:forms(Forms, []),
yu-fengdemacbook-2:~ yufeng$ cat float.erl
-module(float).
-export([new/1,update/3]).
new(N) ->
hipe_bifs:bytearray(N*8,0).
update(Arr,N,Float) ->
<<A1,A2,A3,A4,A5,A6,A7,A8>> = <<Float/float>>,
Start=N*8,
hipe_bifs:bytearray_update(Arr,Start,A1),
hipe_bifs:bytearray_update(Arr,Start+1,A2),
hipe_bifs:bytearray_update(Arr,Start+2,A3),
hipe_bifs:bytearray_update(Arr,Start+3,A4),
hipe_bifs:bytearray_update(Arr,Start+4,A5),
hipe_bifs:bytearray_update(Arr,Start+5,A6),
hipe_bifs:bytearray_update(Arr,Start+6,A7),
hipe_bifs:bytearray_update(Arr,Start+7,A8).
yu-fengdemacbook-2:~ yufeng$ otp_src_R13B02-1/bin/erl
Erlang R13B02 (erts-5.7.3) [source][/source][/source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]
Eshell V5.7.3 (abort with ^G)
1> cover:start().
{ok,<0.33.0>}
2> cover:compile(float).
abstract code:
{raw_abstract_v1,
[{attribute,1,file,{"/Users/yufeng/float.erl",1}},
{attribute,1,module,float},
{attribute,3,export,[{new,1},{update,3}]},
{function,5,new,1,
[{clause,5,
[{var,5,'N'}],
[],
[{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,new},
{integer,0,1},
{integer,0,1},
{integer,0,6}]},
{integer,0,1}]},
%% 我们清楚的看到 ets:update_counter()的调用
{call,6,
{remote,6,{atom,6,hipe_bifs},{atom,6,bytearray}},
[{op,6,'*',{var,6,'N'},{integer,6,8}},{integer,6,0}]}]}]},
{function,8,update,3,
[{clause,8,
[{var,8,'Arr'},{var,8,'N'},{var,8,'Float'}],
[],
[{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,9}]},
{integer,0,1}]},
{match,9,
{bin,9,
[{bin_element,9,{var,9,'A1'},default,default},
{bin_element,9,{var,9,'A2'},default,default},
{bin_element,9,{var,9,'A3'},default,default},
{bin_element,9,{var,9,'A4'},default,default},
{bin_element,9,{var,9,'A5'},default,default},
{bin_element,9,{var,9,'A6'},default,default},
{bin_element,9,{var,9,'A7'},default,default},
{bin_element,9,{var,9,'A8'},default,default}]},
{bin,9,[{bin_element,9,{var,9,'Float'},default,[float]}]}},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,10}]},
{integer,0,1}]},
{match,10,
{var,10,'Start'},
{op,10,'*',{var,10,'N'},{integer,10,8}}},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,11}]},
{integer,0,1}]},
{call,11,
{remote,11,{atom,11,hipe_bifs},{atom,11,bytearray_update}},
[{var,11,'Arr'},{var,11,'Start'},{var,11,'A1'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,12}]},
{integer,0,1}]},
{call,12,
{remote,12,{atom,12,hipe_bifs},{atom,12,bytearray_update}},
[{var,12,'Arr'},
{op,12,'+',{var,12,'Start'},{integer,12,1}},
{var,12,'A2'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,13}]},
{integer,0,1}]},
{call,13,
{remote,13,{atom,13,hipe_bifs},{atom,13,bytearray_update}},
[{var,13,'Arr'},
{op,13,'+',{var,13,'Start'},{integer,13,2}},
{var,13,'A3'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,14}]},
{integer,0,1}]},
{call,14,
{remote,14,{atom,14,hipe_bifs},{atom,14,bytearray_update}},
[{var,14,'Arr'},
{op,14,'+',{var,14,'Start'},{integer,14,3}},
{var,14,'A4'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,15}]},
{integer,0,1}]},
{call,15,
{remote,15,{atom,15,hipe_bifs},{atom,15,bytearray_update}},
[{var,15,'Arr'},
{op,15,'+',{var,15,'Start'},{integer,15,4}},
{var,15,'A5'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,16}]},
{integer,0,1}]},
{call,16,
{remote,16,{atom,16,hipe_bifs},{atom,16,bytearray_update}},
[{var,16,'Arr'},
{op,16,'+',{var,16,'Start'},{integer,16,5}},
{var,16,'A6'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,17}]},
{integer,0,1}]},
{call,17,
{remote,17,{atom,17,hipe_bifs},{atom,17,bytearray_update}},
[{var,17,'Arr'},
{op,17,'+',{var,17,'Start'},{integer,17,6}},
{var,17,'A7'}]},
{call,0,
{remote,0,{atom,0,ets},{atom,0,update_counter}},
[{atom,0,cover_internal_data_table},
{tuple,0,
[{atom,0,bump},
{atom,0,float},
{atom,0,update},
{integer,0,3},
{integer,0,1},
{integer,0,18}]},
{integer,0,1}]},
{call,18,
{remote,18,{atom,18,hipe_bifs},{atom,18,bytearray_update}},
[{var,18,'Arr'},
{op,18,'+',{var,18,'Start'},{integer,18,7}},
{var,18,'A8'}]}]}]},
{eof,19}]}
{ok,float}
这个故事告诉我们对 erlang系统的跟踪除了trace机制以为， 我们还可以用parse transform在编译的时候加入我们想要的代码，达到跟踪， 了解系统的目的。具体的可以参考 compiler模块 parse_transform的文档。
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
本文链接地址: erlang的profile工具原理和优缺点
erlang的tools application下包含了一系列的profile工具， 包括 eprof cprof fprof， 具体的使用可以参看文档和<< erlang effective guide>>.
我这里要说的是他们的工作原理。 这些模块的核心都是根据erlang的trace机制实现的。在模块执行的时候，trace机制会通知那个函数被调用 返回。根据这些信息就可以统计出来函数调用的频度，调用栈等。
但是利用这个机制会有严重的性能损失。因为每个函数调用都要发送一条trace信息，每个trace 信息会引起上下文切换 而且要耗费2-3的时间。这个对大型的系统是不可接受的。
所以知道这些原理以后， 我们在profile大型的系统的时候，我们可以在dbg模块的帮助下， 只收集我们感兴趣的东西，而且严格限定范围，避免对系统造成大的干扰，这样收集出来的东西才有意义。
Post Footer automatically generated by wp-posturl plugin for wordpress.
原创文章，转载请注明： 转载自系统技术非业余研究
前端时间看到JVM的 64 pointer compress技术，蛮多感慨的。具体的可以 google
64位 pointer compress 来了解更多。
好不容易从32位系统中逃脱，解决了4G内存的问题，而且64位的cpu更多的寄存器，可以带来更好的性能。但是怎么又碰到问题了。64位的系统，64位的指针，意味着更多的数据访问， cpu速度是提高了，但是内存的带宽和访问速度没有大的提高。而且内存的访问速度和cpu的cycle差几个数量级别，所以对于普通的网络程序来讲，大部分是处理信息的变形，也就是说是把内存里面的数据从一个形式变成另外一个形式。 cpu性能的提高对这种程序来讲 影响不是很大。倒是因为数据的量加大了1倍， 导致整体的性能降低了百分几十。实际的硬件系统也没有那么多内存，一般都是16G以下，所以才有人去想在64位系统下，用36位的指针，减少内存的访问。
erlang的系统基本上是个网络程序，所以这个问题就非常突出。 目前没看到otp vm的这方面的打算， 我还是乖乖的用我的32位系统
Post Footer automatically generated by wp-posturl plugin for wordpress.
Recent Comments