Archive

Posts Tagged ‘rebar’

Erlang取当前时间的瓶颈以及解决方案

November 4th, 2013 4 comments

原创文章,转载请注明: 转载自系统技术非业余研究

本文链接地址: Erlang取当前时间的瓶颈以及解决方案

高性能网络服务器通常会涉及大量和时间相关的场景和操作,比如定时器,读取事件的发生时间,日志等等。

erlang提供了二种方式来获取时间:
1. erlang:now()
2. os:timestamp()
获取取到时间后,我们通常用calendar:now_to_universal_time来格式化类似”{{2013,11,4},{8,46,20}}”这样人可读的时间。

由于时间调用非常的频繁,而且通常发生在关键路径上,所以效率和性能就非常值得深挖了。

我们先来看下这二个函数的说明:
erlang:now 参看这里

now() -> Timestamp

Types:

Timestamp = timestamp()
timestamp() =
{MegaSecs :: integer() >= 0,
Secs :: integer() >= 0,
MicroSecs :: integer() >= 0}
Returns the tuple {MegaSecs, Secs, MicroSecs} which is the elapsed time since 00:00 GMT, January 1, 1970 (zero hour) on the assumption that the underlying OS supports this. Otherwise, some other point in time is chosen. It is also guaranteed that subsequent calls to this BIF returns continuously increasing values. Hence, the return value from now() can be used to generate unique time-stamps, and if it is called in a tight loop on a fast machine the time of the node can become skewed.

It can only be used to check the local time of day if the time-zone info of the underlying operating system is properly configured.

If you do not need the return value to be unique and monotonically increasing, use os:timestamp/0 instead to avoid some overhead.

os:timestamp 参看这里

timestamp() -> Timestamp

Types:

Timestamp = erlang:timestamp()
Timestamp = {MegaSecs, Secs, MicroSecs}
Returns a tuple in the same format as erlang:now/0. The difference is that this function returns what the operating system thinks (a.k.a. the wall clock time) without any attempts at time correction. The result of two different calls to this function is not guaranteed to be different.

The most obvious use for this function is logging. The tuple can be used together with the function calendar:now_to_universal_time/1 or calendar:now_to_local_time/1 to get calendar time. Using the calendar time together with the MicroSecs part of the return tuple from this function allows you to log timestamps in high resolution and consistent with the time in the rest of the operating system.

但是事情没这么简单!

由于erlang支持时间纠正机制,简单的说在时间发生突变的时候,还能维持正常的时间逻辑,具体的实现参看这篇:服务器时间校正思考。

时间纠正机制让事情变得复杂,这个时间纠正机制如何禁止呢:

+c
Disable compensation for sudden changes of system time.

Normally, erlang:now/0 will not immediately reflect sudden changes in the system time, in order to keep timers (including receive-after) working. Instead, the time maintained by erlang:now/0 is slowly adjusted towards the new system time. (Slowly means in one percent adjustments; if the time is off by one minute, the time will be adjusted in 100 minutes.)

When the +c option is given, this slow adjustment will not take place. Instead erlang:now/0 will always reflect the current system time. Note that timers are based on erlang:now/0. If the system time jumps, timers then time out at the wrong time.

正是由于时间纠正机制的存在,所以服务器需要不时的修正时间,同一时刻可能还有很多线程在读取时间,为了维护时间的一致性,需要有个锁来保护。
我们来看下相关的代码实现:
Read more…

Post Footer automatically generated by wp-posturl plugin for wordpress.

ms()用途浅析

July 27th, 2013 1 comment

原创文章,转载请注明: 转载自系统技术非业余研究

本文链接地址: ms()用途浅析

erlang系统的application,稍微复杂一点的都会提供一个ms/0的导出函数,而且这个导出函数通常在文档里面找不到描述,很奇怪不是吗?
比如mnesia就有这样的ms, 我们来看下:

$ erl
Erlang R17A (erts-5.11) [source-b7fbc28] [64-bit] [smp:16:16] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.11  (abort with ^G)
1> mnesia:ms().
[mnesia,mnesia_backup,mnesia_bup,mnesia_checkpoint,
 mnesia_checkpoint_sup,mnesia_controller,mnesia_dumper,
 mnesia_loader,mnesia_frag,mnesia_frag_hash,
 mnesia_frag_old_hash,mnesia_index,mnesia_kernel_sup,
 mnesia_late_loader,mnesia_lib,mnesia_log,mnesia_registry,
 mnesia_schema,mnesia_snmp_hook,mnesia_snmp_sup,
 mnesia_subscr,mnesia_sup,mnesia_text,mnesia_tm,
 mnesia_recover,mnesia_locker,mnesia_monitor,mnesia_event]
2> 

看起来貌似只是返回组成mnesia的模块列表而已。
那么它的作用是什么呢?

复杂一点的程序都需要在运行期间来进行观察或者优化,比如说dbg跟踪一个函数或者模块运作的时候,是需要这个模块的名字的,如:

dbg:tp(Module,MatchSpec)

那么如果要跟踪整个application的运作,我们通常会写这样的代码:
[do_some_thing(M) || M<-myapp:ms()]. 所以这就是ms的意义所在。 现在的问题是mnesia的代码是把ms的模块硬编码的,这样会带来一个维护的问题,比如添加,改名或者减少一个模块都要记得去修改这个列表,很麻烦。 [erlang] %mnesia.erl ms() -> [ mnesia, mnesia_backup, mnesia_bup, ... ]. [/erlang] 程序员是个很懒的群体,必定不会这么挫的,于是rebar就专门花了点力气把这个事情做的漂亮。 rebar在编译application的时候,会把src/myapp.app.src的内容添加以下内容:

{modules,[mod_a, mod_b,…]}

形成ebin//myapp.app文件,这个文件是每个app必须的!

rebar处理这个事情的核心代码如下:

%%rebar_otp_app.erl
AppVars = load_app_vars(Config1) ++ [{modules, ebin_modules()}],
ebin_modules() ->
    lists:sort([rebar_utils:beam_to_mod("ebin", N) ||
                   N <- rebar_utils:beams("ebin")]).

Read more…

Post Footer automatically generated by wp-posturl plugin for wordpress.

Categories: Erlang探索, 源码分析 Tags: ,

rebar和common_test使用实践和疑惑澄清

October 19th, 2011 8 comments

原创文章,转载请注明: 转载自系统技术非业余研究

本文链接地址: rebar和common_test使用实践和疑惑澄清

rebar是个功能非常强大的Erlang项目管理工具,参看这里 https://github.com/basho/rebar,他的定位是:

rebar is an Erlang build tool that makes it easy to compile and
test Erlang applications, port drivers and releases.

rebar is a self-contained Erlang script, so it’s easy to distribute or even
embed directly in a project. Where possible, rebar uses standard Erlang/OTP
conventions for project structures, thus minimizing the amount of build
configuration work. rebar also provides dependency management, enabling
application writers to easily re-use common libraries from a variety of
locations (git, hg, etc).

common_test是Erlang强大的黑盒测试框架 参见这里

Common Test is a portable application for automated testing. It is suitable for black-box testing of target systems of
any type (i.e. not necessarily implemented in Erlang), as well as for white-box testing of Erlang/OTP programs. Blackbox
testing is performed via standard O&M interfaces (such as SNMP, HTTP, Corba, Telnet, etc) and, if required, via
user specific interfaces (often called test ports). White-box testing of Erlang/OTP programs is easily accomplished by
calling the target API functions directly from the test case functions. Common Test also integrates usage of the OTP
cover tool for code coverage analysis of Erlang/OTP programs.
Common Test executes test suite programs automatically, without operator interaction. Test progress and results is
printed to logs on HTML format, easily browsed with a standard web browser. Common Test also sends notifications
about progress and results via an OTP event manager to event handlers plugged in to the system. This way users can
integrate their own programs for e.g. logging, database storing or supervision with Common Test.
Common Test provides libraries that contain useful support functions to fill various testing needs and requirements.
There is for example support for flexible test declarations by means of so called test specifications. There is also
support for central configuration and control of multiple independent test sessions (towards different target systems)
running in parallel.
Common Test is implemented as a framework based on the OTP Test Server application.

但是common_test由于太强大了,新手使用起来会比较麻烦,经常会碰到些问题,不好解决。

这时候rebar来救助了,它的ct功能把common_test使用的麻烦给解决掉了,让你轻松做测试。
我们先来体验下:
Read more…

Post Footer automatically generated by wp-posturl plugin for wordpress.

Categories: Erlang探索 Tags: , ,