New features in Kamailio v3.1.0

The page collects the summary of the new features in Kamailio v3.1.0.

Release date: Oct 6, 2010.

Kamailio 3.1.0 comes with an enhanced configuration file, among additions:

• use of defines with values to set parameters such as db_url
• multidamoan support toggled by define WITH_MULTIDOMAIN
• config flags use defined IDs for more meaningful names
• tls configuration toggled by define WITH_TLS
• flooding (DoS) attack detection toggled by define WITH_ANTIFLOOD
• xmlrpc configuration toggled by define WITH_XMLRPC
• database aliases configuration toggled by define WITH_ALIASDB
• main route restructured:
  • reflect better the parts registrar and location services
  • reflect better the parts for within dialog routing

• send accounting records to a Radius server
  • stand-alone, does not require to recompile acc module
  • old way provided by acc re-compiled with radius support is still kept, but will be deprecated and removed over the time
• can be used in parallel with database or syslog accounting
• same configuration parameters like with old acc-module-based radius accounting
  • flag mechanism for accounting
  • function to send accounting request immediately
• http://www.kamailio.org/docs/modules/3.1.x/modules_k/acc_radius.html

Example - send a Radius accounting request:

acc_rad_request("200 Start");

• execute embedded Lua scripts
• you get access to SIP message processed at that time
  • via config variables, e.g., $rU, $fu, $var(x)
• you can set config variables from Lua programs
• several internal SIP router functions are exported to Lua as well
• http://sip-router.org/wiki/api/lua/devel
• http://www.kamailio.org/docs/modules/3.1.x/modules/app_lua.html

Example sending a SIP reply through Lua:

lua_dostring("sr.sl.send_reply(200, [[Lua says is ok]])");

Presentation at Amoocon 2010 about how to send Twitter notifications from your SIP server using Lua, slides at:

• http://www.amoocon.de/talks/101

Next is a Lua function showing how you get access to config pseudo-variables:

• print a message to a syslog via SIP server
• append a header to reply with request URI

function my_append_hdr_ruri()
    sr.err("message to syslog from Lua\n")
    sr.hdr.append_to_reply("P-RUri: <" .. sr.pv.get("$ru") .. ">\r\n");
end

• execute embedded python scripts
• you get access to SIP message processed at that time
• http://www.kamailio.org/docs/modules/3.1.x/modules/app_python.html

• functions to handle internal counters in configuration files and via RPCs
• atomic operations
• the internal counters are the base for new statistics API
  • main benefit faster (lockless) writing
  • allow grouping of counters
• http://www.kamailio.org/docs/modules/counters.html

Example - count requests for offline users:

modparam("counters", "script_counter", "location.offline requests for offline users")
...
route {
...
	if (!lookup("location"))
		cnt_inc("location.offline");
...
}

• config file debugger
• print the trace of executed actions in config for SIP messages
• step-by-step execution of each action in config (similar to gdb)
  • can be done from remote system via RPC interface (e.g., XMLRPC)
  • evaluate pseudo-variables at runtime
  • print line and action type for each step
• http://www.kamailio.org/docs/modules/3.1.x/modules/debugger.html

Example - execution trace:

 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=501 a=17 n=if
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=470 a=25 n=has_totag
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=386 a=17 n=if
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=379 a=26 n=is_method
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=386 a=25 n=t_check_trans
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=389 a=6 n=route
 9(6285) ERROR: *** cfgtrace: c=[/etc/kamailio/debugger.cfg] l=643 a=3 n=return

Example - interactive debugger:

sercmd> dbg.bp show 6402
at bkp [/etc/kamailio/debugger.cfg:369] a=6 n=route
 
sercmd> dbg.bp next 6402
exec [/etc/kamailio/debugger.cfg:369] a=6 n=route
 
sercmd> dbg.bp next 6402
exec [/etc/kamailio/debugger.cfg:462] a=17 n=if
 
sercmd> dbg.bp eval 6402 $fu
$fu : t=str v=sip:test@kamailio.org
 
sercmd> dbg.bp move 6402
200 ok

• perform lookups based on IP against a local geoip database
• resulted attributes are available in pseudo-variables
• support to perform many lookups and store the results at the same time
• IP parameter can be taken from source address, destination address, headers or other storage
• can be used to block unwanted traffic coming from untrusted countries
  • prevent spit
  • prevent scanning attacks
• http://www.kamailio.org/docs/modules/3.1.x/modules/geoip.html

Example - blocking traffic from Canada:

geoip_match("$si", "src");
if($gip(src=>cc)=="CA") {
  xlog("SIP message from Canada (ip: $si) - drop it\n");
  send_reply("403", "Forbidden");
  exit;
}

• can do arbitrary lookup operations over an array
• http://www.kamailio.org/docs/modules/3.1.x/modules/matrix.html

• message queue for inter-process communication
• first message added is first consumed
• messages are pairs of strings (key, value)
• useful for asynchronous processing in configuration file
  • add tasks in a queue from SIP workers that are handled in by special process
  • for example:
    • send notification to twitter
    • write a log message
    • write accounting record
• http://www.kamailio.org/docs/modules/3.1.x/modules/mqueue.html

Example - adding and fetching items from a queue:

modparam("mqueue", "mqueue", "name=myq;size=0;")
...
route {
  mq_add("myq", "ruri", "$ru");
  mq_fetch("myq");
  xlog("SCRIPT: mqueue value ($ru): $mqk(myq) >> $mqv(myq)\n");

• load db tables in shared memory indexed on tree based on prefix key
• fast lookup for keys like DIDs
• can manage many trees at the same time
• http://www.kamailio.org/docs/modules/3.1.x/modules/mtree.html

Example - load db table 'dids' in a tree named 'prefixes' and match against request URI username:

modparam("mtree", "mtree", "name=prefixes;dbtable=dids;type=0;")
 
route {
 
  if(!mt_match("prefixes", "$rU", "0")) {
     send_reply("404", "Not found");
     exit;
  }
 
}

• limit traffic based on different algorithms
  • e.g., if number of requests per second exceed a threshold, drop them
• definition of pipes is taken from database
• unlimited and dynamic pipe IDs
• the IDs can be string names
• spawned by reusing parts of ratelimit module
• useful for flood detection, working as well for detecting and stopping distributed denial of service attacks
  • if you detect a rate of requests far higher than in normal operation mode, you can trigger alerts
    • e.g., if number of requests is higher than 200 per second, write to syslog, send email, etc.
• http://www.kamailio.org/docs/modules/3.1.x/modules/pipelimit.html

# pipes per request type - e.g., INVITE
$var(p) = "$rm;
if (!pl_check("$var(p)")) {
  pl_drop();
  exit;
}

• presence server extension to handle audio conference notification events
  • RFC 4353: A Framework for Conferencing with SIP
  • RFC 4575: A SIP Event Package for Conference State
• notify participants in an audio conference about who is speaking and audio levels
• fruitful outcome of our first Google Summer of Code participation
• http://www.kamailio.org/docs/modules/3.1.x/modules_k/presence_conference.html

• new module to deal with rtpproxy servers
• rtpproxy functionality was removed from nathelper and moved to this new module
• support for RTP-Timeout notifications from the RTP-Proxy (requires dialog module for call-termination)
• http://www.kamailio.org/docs/modules/3.1.x/modules_k/rtpproxy.html

• new module for extra text operations
• function msg_apply_changes() moved from modules_k/texops in this module
  • apply changes to message headers or body so they are visible immediately in configuration file
• new function to change the reply status - change_reply_status(code, reason)
• http://www.kamailio.org/docs/modules/3.1.x/modules/textopsx.html

Example - change 603 replies to 404:

onreply_route {
    if (status == "603") {
        change_reply_status(404, "Not Found");
        exit;
    }
}

• XCAP server implementation
• fully integrated with internal Presence server
• controlled from same configuration file
• XCAP docs can be send via:
  • SIP (UDP, TCP, TLS, SCTP)
  • HTTP and HTTPS
• can be used as stand-alone XCAP server
• can detect XCAP requests sent through SIP channel
  • can listen for XCAP requests on many ports, e.g., 80, 443, 5060, 5061
• readme at: http://www.kamailio.org/docs/modules/3.1.x/modules_k/xcap_server.html

See a complete tutorial of using embedded XCAP server with SIP Communicator softphone:

• https://kb.asipto.com/kamailio:presence:k31-made-simple

• basic HTTP server
  • reuses core transport layers for TCP and TLS
• support for HTTP v1.0 and v1.1 chucked transfer
• handling of HTTP requests is done within event_route[xhttp:request]
• HTTP URL is available in pseudo-variable $hu
• HTTP requests can be sent through SIP channel (port 5060) - type of message is auto-detected and handled properly
• can listen on multiple ports for HTTP: 80, 442, 5060, 5061, 8080, a.s.o.
• readme at: http://www.kamailio.org/docs/modules/3.1.x/modules/xhttp.html

Next example shows how to send back a welcome message if HTTP URL starts with “/welcome”:

event_route[xhttp:request] {
  switch($hu) {
    case /"^/welcome":
      xhttp_reply("200", "OK", "text/html",
        "<html><body>Welcome! You are connected to us from [$si:$sp]</body></html>");
    break;
    default:
      xhttp_reply("404", "Not found", "text/html",
        "<html><body>Page not found</body></html>");
  }
}

• xml operations in configuration file with XPATH
• readme at: http://www.kamailio.org/docs/modules/3.1.x/modules/xmlops.html

• functions for ignoring blacklist events per message:
  • blst_set_ignore(mask): set the events in mask in the per per message blacklist ignore mask for a request (see dst_blacklist_udp_imask for possible values). The basic operation is: msg_blst_ignore_mask|=mask.
  • blst_clear_ignore(mask): like blst_set_ignore(mask), but instead of setting some events, it clears them (msg_blst_ignore_mask&=~mask).
  • blst_rpl_set_ignore(mask): like blst_set_ignore(mask), but sets the mask for possible local replies to the current message.
  • blst_rpl_clear_ignore(mask): like blst_rpl_ignore(mask), but clears instead of setting.

• new function to detect if a request belongs to an active dialog
• support for detection of SIP spirals
• new callback executed when a SIP spiral is detected

• new load balancing algorithms
  • weight based load balancing
  • call load load balancing
• dynamic configuration of valid response codes for the availability check
• parameter to control append branch removed - change of r-uri is auto-detected

• Major rewrite:
  • lcr and gw tables have been replaced by lcr_rule, lcr_rule_target, and lcr_gw tables
  • upgrade script is available as http://box.openxg.com/tmp/lcr_upgrade_from_3.0.sh.
• A rule can be specified as a “stopper” rule meaning that no shorter prefixes will be tried if “stopper” rule matches.
• A rule may be marked as “disabled”.
• Added new gateway property: URI parameters.
• Gateway IP address can be NULL, if hostname is not NULL.
• If gateway IP address is NULL, all test functions fail on that LCR instance, because for performance reasons, DNS queries are not done and test functions operate only on IP addresses.
• MI functions are not supported anymore; use RPC functions instead.

• new parameter: add_contact
  • control addition of contact header in local generated messages (default off, as per rfc3428)

• rtpproxy functionality was moved from nathelper to rtpproxy

• support for timeout notifications from the rtpproxy (using XML-RPC)

• new pv class to get access to timeval attributes (seconds and microseconds)
  • $PV(u) - microseconds (cached)
  • $PV(s) - seconds (cached)
  • $PV(un) - microseconds (not cached)
  • $PV(sn) - seconds (not cached)
  • $PV(Sn) - string with sec.usec
• new pv class to get access to next hop address attributes
  • $nh(u) - uri
  • $nh(U) - username
  • $nh(d) - domain
  • $nh(p) - port
  • $nh(P) - protocol
• new transformations
  • s.strip - strip characters from beginning of a string pseudo-variable
  • s.striptail - strip characters from ending of a string pseudo-variable
  • see: http://sip-router.org/wiki/cookbooks/transformations/devel#sstrip_len
• $branch(name) PV is writable, you can update uri, dst uri, etc. for each branch
  • $(branch(dst_uri)[2]) = “sip:10.10.10.10”;
  • assigning $null to uri attribute will drop the branch: $(branch(uri)[0]) = $null;
  • adding a new branch can be done via old ways with km_append_branch(uri) or append_branch()
• From/To attributes are writable via pseudo-variables

• new parameter for randomization of Expire header
• http://lists.sip-router.org/pipermail/sr-dev/2010-April/006920.html

• module is working with the new core and tm
• no support yet for asynchronous tcp/tls

• new parameters for advanced openssl options
• http://lists.sip-router.org/pipermail/sr-dev/2010-March/006559.html
• asynchronous TLS processing support has been added
• certificate revocation list (CRL) support.
• new TLS RPCs (tls.info, tls.options), tls.list more detailed.
• removed handshake_timeout and send_timeout module parameters / config variables. The values from tcp are used instead (tcp_connect_timeout and tcp_send_timeout).
• runtime config support
• more config options:
  • crl - certificate revocation list file path (PEM format).
  • send_close_notify - enables/disables sending close notify alerts prior to closing the corresponding TCP connection. Sending the close notify prior to tcp shutdown is “nicer” from a TLS point of view, but it has a measurable performance impact. Default: off. Can be set at runtime (tls.send_close_notify).
  • con_ct_wq_max - per connection tls maximum clear text write queue size. The TLS clear-text write queues are used when a send attempt has to be delayed due to an on-going TLS level renegotiation. Can be set at runtime (tls.con_ct_wq_max). Default: 65536 (64 Kb).
  • ct_wq_max - maximum total for all the tls clear text write queues (summed). Can be set at runtime (tls.ct_wq_max). Default: 10485760 (10 Mb).
  • ct_wq_blk_size - internal TLS pre-write (clear-text) queue minimum block size (advance tunning or debugging). Can be set at runtime (tls.ct_wq_blk_size). Default: 4096 (4 Kb).
• verbose debug messages can be enable by re-compiling with -DTLS_RD_DEBUG (for the read path) and -DTLS_WR_DEBUG (for the write path).
• new options for better tuning memory usage for modern openssl versions: ssl_release_buffers (default 1), ssl_freelist_max_len (default 0), ssl_max_send_fragment, ssl_read_ahead (default 0). For more info see modules/doc/tls/README.
• compression is now disabled by default. To enable it set tls_disable_compression to 0, but note that memory usage will increase dramatically especially for large number of connections (>1000).

• t_reply() can be used in main and tm onreply_route to change reply status code and reason
• Function next_contacts() is not anymore setting any timers. Check if contact_avp has any values left and call t_set_fr() accordingly before calling t_relay().
• Removed unused fr_timer_next module parameter.
• reason header support (RFC3326) both for CANCELs generated due to a received final reply and for hop by hop CANCELs generated because of a received CANCEL.
  • reason header added for a CANCEL generated after a 200 reply was eceived on one of the branches “Reason: SIP;cause=200”.
  • reason header support can be turned on/off using either tm module parameters or in the end to end CANCEL case also on a per transaction basis, using a script function:
    • local_cancel_reason = 0 | 1 (default 1/on) - turns on adding reason headers for CANCELs generated due to a final reply. Can be changed at runtime.
    • e2e_cancel_reason = 0 | 1 (default 1/on) - turns on copying reason headers from a received end to end CANCEL (the generated hop by hop CANCELs will have the same reason headers as the received CANCEL). Can be changed at runtime.
    • t_set_no_e2e_cancel_reason(0|1) - enable/disable cancel reason header copying on a per transaction basis (0 - enable, 1 disable).

• new functions for remote control of INVITE transaction
  • transaction can be canceled by call id
    • one useful scenario: cancel the transaction in order to redirect it to a new destination by failure_route
  • transaction can be replied by call id
    • you can intercept the transaction by call id and send reply to it
• http://sip-router.org/docbook/sip-router/branch/master/modules_k/tmx/tmx.html

Example:

• say you stored call id and cseq in database for active transaction
• from a web interface you fetch them and when you want to cancel a transactionthen you click on HTTP links having the format /cancel/callid/cseq

event_route[xhttp:request] {
  if($hu =~ "^/cancel/")
  {
    t_cancel_callid("$(hu{s.select,2,/})", "$(hu{s.select,3,/})", "0");
    xhttp_reply("200", "ok", "text/html", "<html><body>cancelled</body></html>");
  }
}

• register in behalf of users to remote SIP servers
  • the list with users is stored in databse
  • user profiles are loaded at startup
• the registrations are automatically refreshed before expiration
• config function to lookup local user based on contact address
  • when a call comes in from remote server you can discover what is the local user
  • then you can do local location lookup

• new function xlogl([level], text) - print cfg line before log message
• new function xdbgl(text) - print cfg line before log message

xlogl("this message is prefixed by config line number\n");

• various parameters from kamailio's modules are integrated with ser config framework and can be modified at runtime with sercmd
• the list of modules includes : carrierroute, registrar, siputils, ratelimit (and growing)
• a complete list of changeable parameters are in modules documentation pages

• global, per protocol blacklist ignore masks (via extended send_flags). See dst_blacklist_udp_imask a.s.o (dst_blacklist_*_imask).
• per message blacklist ignore masks

• hooks for asynchronous TLS
• integration of TCP and TLS layers for asynchronous communication
  • tcp_async=yes enables async communication on both

• new more write-performant statistics framework
• accessible in config via counters module
• old statistics were migrated transparently to the new framework

• a new module exports version allow the modules functions to take expressions as parameters
• many module functions automatically support now expressions or variables in function parameters. This applies to all the module functions declared without a fixup, with a fixup and the corresponding free_fixup function or with a compatible ser or kamailio style standard fixup (declared in sr_module.h or mod_fix.h).

 f($a, "b = " + $b);  t_set_fr($v + 2 + $x).
 t_set_fr($foo) (equivalent now with t_set_fr("$foo")).

• all the module functions can now be called with any constant expression as parameters

 f("7 *" +" 6 = " + 7 * 6);

• support to communicate via UDP raw sockets
• boost in UDP communication performances
• major performance increase on Linux multi-cpu machines that send a lot of UDP IPv4 packets (40-50% faster in stateless mode)
• to work udp4_raw must be enabled or set into auto mode in cfg and server must be started as root or with CAP_NET_RAW
• note that even if udp4_raw is off (default), if server was started with enough privileges, it can be enabled at runtime.
• the support for using raw sockets is also available on FreeBSD (compiled by default but not tested for performance yet), NetBSD, OpenBSD and Darwin (not tested and not compiled by default, it needs make cfg extra_defs=-DUSE_RAW_SOCKS)
• to check if the support is compiled, use sercmd core.udp4_raw_info.

• you can define values for IDs

#!define MYINT 123
#!define MYSTR "xyz"

• defined IDs are replaced at startup, during config parsing, e.g.,:

$var(x) = 100 + MYINT;

• is interpreted as:

$var(x) = 100 + 123;

• you can have multi-line defined IDs

#!define IDLOOP $var(i) = 0; \
                while($var(i)<5) { \
                    xlog("++++ $var(i)\n"); \
                    $var(i) = $var(i) + 1; \
                }

• then in routing block

route {
    ...
    IDLOOP
    ...
}

• new preprocessor directive

#!subst "/regexp/subst/"

• perform substitutions inside the strings of config (note that define is replacing only IDs - alphanumeric tokens not enclosed in quotes)

• #!subst offers an easy way to search and replace inside strings before cfg parsing. E.g.,:

#!subst "/DBPASSWD/xyz/"
modparam("acc", "db_url", "mysql://user:DBPASSWD@localhost/db")

• will do the substitution of db password in db_url parameter value
• number of allowed defines is now set to 256

• notes:
  • multilines defines are reduced to single line, so line counter should be fine
  • column counter goes inside the define value, but you have to omit the '\' and CR for the accurate inside-define position

• new option -A to set defined values from command line (e.g., -A WITH_TLS -A MYVALUE=1)
• new option -a to control auto-aliases (e.g., -a no/yes)

• dst_blacklist_udp_imask - global blacklist events ignore mask for udp (a blacklist event/reason set in this variable will be ignored when deciding whether or not to blacklist an udp destination). Can be set at runtime. Default: 0 (no blacklist reason is ignored).

    Possible values:  0 -disabled, 2 - send error; 4 - connect error,
                      8 - icmp (reserverd), 16 - transaction timeout,
                     32 - 503 received, 64 - administratively prohibited
                     (manually set).

• dst_blacklist_tcp_imask - like dst_blacklist_udp_imask, but for tcp.
• dst_blacklist_tls_imask - like dst_blacklist_tls_imask, but for tls.
• dst_blacklist_sctp_imask -like dst_blacklist_sctp_imask, but for sctp.
• tcp_accept_no_cl - accept messages on TCP without Content-Lenght header (assume is 0 - no body)

• support for using sercmd as control tool to interact with SIP server via RPC
• can be used to send command to Kamailio from a remote host (so far was only from same host)

• web management interface released preview of version 2.0 that works with Kamailio 3.1.0. Release expected in the next days.
• new GUI and self management tools
• support for many new modules of Kamailio
• http://siremis.asipto.com

The tool located in utils/route_graph/ prints a nice text based graph of your configuration file, with the routing blocks and relations between them.