Archive

Author Archive

简单的Map reduce用的收集函数

October 27th, 2011 3 comments

在处理大量重复任务的时候,为了加快速度,通常会用map-reduce的方式,要是能有段代码做这个事情就好了。作者luke写了底下的代码片段,用起来感觉挺爽的,推荐给大家。原文见这里

%% http://lukego.livejournal.com/6753.html – that doesn’t care about
%% the order in which results are received.
upmap(F, L) ->
Parent = self(),
Ref = make_ref(),
[receive {Ref, Result} -> Result end
|| _ <- [spawn(fun() -> Parent ! {Ref, F(X)} end) || X <- L]].

这个函数的特点是不依赖于任务完成的顺序,结构很简单优雅。

祝玩得开心!

Categories: Erlang探索 Tags: ,

看multitrace代码学习如何定制自己的dbg信息

October 27th, 2011 Comments off

multitrace是ttb应用带的一个例子,给了个例子让用户来格式化和定制自己的dbg信息。 文档在这里

The module multitrace.erl which can be found in the src directory of the Observer application implements a small tool with three possible trace settings. The trace messages are written to binary files which can be formatted with the function multitrace:format/1/2.

代码太长,我就不贴了,可以点这里查看.

我演示下如何使用:
Read more…

Categories: Erlang探索 Tags: ,

Erlang如何限制节点对集群的访问之net_kernel:allow

October 24th, 2011 6 comments

默认情况下Erlang的集群访问是全授权的,只要cookie认证过了后,新加入的节点可以访问集群里面的任何机器,这给运维带来很大风险。目前erlang有二种方法可以限制 1. IP网段限制,参看这里 2. 节点名称限制。这个是通过net_kernel:allow来实现的,参看:

allow/1
Limits access to the specified set of nodes. Any access attempts made from (or to) nodes not in Nodes will be rejected.

Returns error if any element in Nodes is not an atom.

我们假设集群有节点x,y,z, foo:
1. 有个节点叫foo, 它只允许来自x,y节点的请求,其他的节点访问被拒;
2. z只能访问x,其他拒

我们来试验下:
Read more…

Categories: Erlang探索 Tags:

Erlang epmd的角色以及使用

October 23rd, 2011 1 comment

很多同学误会了epmd的作用,认为epmd就是erlang集群的协议,我来澄清下:

Epmd是Erlang Port Mapper Daemon的缩写,在Erlang集群中的作用相当于dns的作用,提供节点名称到端口的查询服务,epmd绑定在总所周知的4369端口上。

epmd文档:这里
epmd协议:这里

Erlang的节点名称是类似这样的foo@ip的格式,当一个节点启动的时候,首先会在本机启动epmd,同时把自己的节点名称和节点监听的tcp端口登记在上面。
看代码:

// erlexec.c
... 
case 'n':
                    if (strcmp(argv[i], "-name") == 0) { /* -name NAME */
                        if (i+1 >= argc)
                            usage("-name");

                        /*                                                                                                                                              
                         * Note: Cannot use add_args() here, due to non-defined                                                                                         
                         * evaluation order.                                                                                                                            
                         */

                        add_arg(argv[i]);
                        add_arg(argv[i+1]);
                        isdistributed = 1;
                        i++;
...
case 's':     /* -sname NAME */
                    if (strcmp(argv[i], "-sname") == 0) {
                        if (i+1 >= argc)
                            usage("-sname");
                        add_arg(argv[i]);
                        add_arg(argv[i+1]);
                        isdistributed = 1;
                        i++;
                    }
...
    if (isdistributed && !no_epmd)
        start_epmd(epmd_prog);
...

我们可以透过erl -epmd EPMD_PROG来传入不同的参数。

再看下实验:

$erl -sname a
$erl -sname b
$erl -sname c
$ ps -ef|grep epmd
membase   4592     1  0 Aug25 ?        00:00:39 /usr/local/bin/epmd -daemon
...
$ netstat -an|grep 4369
tcp        0      0 0.0.0.0:4369                0.0.0.0:*                   LISTEN    
$ epmd -names
epmd: up and running on port 4369 with data:
name c at port 4096
name b at port 4097
name a at port 4098
...
$erl -sname x
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:16:16] [rq:16] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.5  (abort with ^G)
(x@my031091)1> erl_epmd:port_please(x, "127.0.0.1"). 
{port,34625,5}

(x@my031091)2> erl_epmd:names().
{ok,[{"a",4096},{"b",4097},{"c",4098},{"x",34625}]}

epmd是个标准的tcp服务器,它的协议如下:

kernel的erl_epmd模块提供epmd协议的封装,向net_kernel模块提供服务。如果net_kernel要连接其他节点的时候,就取出节点名称的ip部分,透过erl_epmd建立连接到ip:4369,通过epmd协议来查询想要的foo的端口,然后再用ip:port去连接真正的服务。

新版本的epmd提供了强行移除名称的功能,避免由于erlang虚拟机由于某种原因crash,没有注销名字,导致无法再使用这个名字。
要使用stop功能,epmd必须以 -relaxed_command_check 启动,具体参考epmd –help
演示下:
Read more…

Categories: Erlang探索 Tags: ,

trace机制新增exception_trace

October 21st, 2011 4 comments

我们在使用Erlang的时候,经常会发现exception被静悄悄得忽略掉了,这点对于诊断问题非常的不友好。R14B04新添加exception_trace帮助用户在异常的时候,得到异常得调用栈,就马上可以解决问题。 这个功能主要面对高级用户,文档里面没怎么描述这个事情,主要的实现在erts的trace模块里面,有兴趣的同学可以自己看看。
我来演示下这个功能:
Read more…

Categories: Erlang探索 Tags: ,

rebar和common_test使用实践和疑惑澄清

October 19th, 2011 8 comments

rebar是个功能非常强大的Erlang项目管理工具,参看这里 https://github.com/basho/rebar,他的定位是:

rebar is an Erlang build tool that makes it easy to compile and
test Erlang applications, port drivers and releases.

rebar is a self-contained Erlang script, so it’s easy to distribute or even
embed directly in a project. Where possible, rebar uses standard Erlang/OTP
conventions for project structures, thus minimizing the amount of build
configuration work. rebar also provides dependency management, enabling
application writers to easily re-use common libraries from a variety of
locations (git, hg, etc).

common_test是Erlang强大的黑盒测试框架 参见这里

Common Test is a portable application for automated testing. It is suitable for black-box testing of target systems of
any type (i.e. not necessarily implemented in Erlang), as well as for white-box testing of Erlang/OTP programs. Blackbox
testing is performed via standard O&M interfaces (such as SNMP, HTTP, Corba, Telnet, etc) and, if required, via
user specific interfaces (often called test ports). White-box testing of Erlang/OTP programs is easily accomplished by
calling the target API functions directly from the test case functions. Common Test also integrates usage of the OTP
cover tool for code coverage analysis of Erlang/OTP programs.
Common Test executes test suite programs automatically, without operator interaction. Test progress and results is
printed to logs on HTML format, easily browsed with a standard web browser. Common Test also sends notifications
about progress and results via an OTP event manager to event handlers plugged in to the system. This way users can
integrate their own programs for e.g. logging, database storing or supervision with Common Test.
Common Test provides libraries that contain useful support functions to fill various testing needs and requirements.
There is for example support for flexible test declarations by means of so called test specifications. There is also
support for central configuration and control of multiple independent test sessions (towards different target systems)
running in parallel.
Common Test is implemented as a framework based on the OTP Test Server application.

但是common_test由于太强大了,新手使用起来会比较麻烦,经常会碰到些问题,不好解决。

这时候rebar来救助了,它的ct功能把common_test使用的麻烦给解决掉了,让你轻松做测试。
我们先来体验下:
Read more…

Categories: Erlang探索 Tags: , ,

Erlang port巧用环境变量

October 15th, 2011 1 comment

Erlang与外面世界的交互主要通过port来进行的,特别是和外部程序的协作,通常是通过管道进行的。
基本上有2种方法可以调用外部程序: 1. os:cmd 2. erlang:open_port, 这二种方式各有利弊,先看文档:
Read more…

Categories: Erlang探索 Tags: , , ,