Home page logo

nmap-dev logo Nmap Development mailing list archives

[NSE] Lua 5.2 Upgrade
From: Patrick Donnelly <batrick () batbytes com>
Date: Wed, 4 Apr 2012 01:43:38 -0400

Hello list,

Over the past few months I've been working on a branch that upgrades
NSE to use Lua 5.2. I'll start off by giving you the location of the
branch so you can try it out:

$ svn checkout --username guest --password ""
https://svn.nmap.org/nmap-exp/patrick/nse-lua52 nse-lua52
$ cd nse-lua52 && ./configure --with-liblua=included && make
$ ./nmap --datadir . --script safe localhost

If you want to play with the Lua interpreter/compiler, you can simply
compile (after making nmap) using:

$ gcc -o liblua/lua liblua/lua.c liblua/liblua.a -lm -ldl
$ gcc -o liblua/luac liblua/luac.c liblua/liblua.a -lm -ldl

This upgrade brings numerous improvements. I've added a page to the
wiki which lists the most relevant or important changes to script


I've included the bulk of that wiki page at the end of this email for
archival purposes.

Please try out the branch and look at the wiki page to get a feel for
the changes. All feedback including "it works" is welcome! I would
like feedback on socket:receive_buf in particular as the code is new
but functionally the same.

The nse_bitlib.cc code is deprecated for now (the 'bit' library).
There may be some functional differences of significance between the
old bit library and the new Lua 5.2 bit32 library. I feel this should
be sorted out separate from this branch.

== Wiki page as Plain Text ==

New features in Lua 5.2 of interest to script developers:

o New hexadecimal escape sequence for strings. For example, instead of
using decimal escape "\010" you can instead use "\x0a".

o Lexical environments with _ENV [1]. Environments are now simply
upvalues of a function and you may introduce a new environment into a
scope by creating a new local named _ENV with a lifetime that expires
at the end of the scope. For example:

local print = print
function foo ()
  a = 5
    local _ENV = {}
    print(a) --> nil
    a = 6
    print(a) --> 6
  print(a) --> 5

You may also do things like:

local print = print
function foo (_ENV)
foo({a = 5}) --> 5
foo({a = 6}) --> 6

The old Lua 5.1 environments are completely gone. setfenv/getfenv are
gone as a result.

o Coroutines may yield across most C call boundaries. One of the
simplest examples of such an error in 5.1:

$ lua51
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
F = coroutine.wrap(function(...)
  return pcall(function(...)
    return coroutine.yield(...)
  end, ...)
false   attempt to yield across metamethod/C-call boundary

This changes allows for coroutines in generic for loop iterators which
yield outside of the for loop. For example:

local function lines (socket)
  while true do coroutine.yield(socket:receive_buf("\r?\n", false)) end
for line in lines(socket) do
  --> do something

[The above example doesn't handle EOF or other errors.]

o Lua 5.2 comes with a default bit library called "bit32".

o Lua closures are now cached. The ramifications are best illustrated:

$ lua51
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
for i = 1, 2 do print(function() end) end
function: 0x2104ad0
function: 0x2104fe0

$ lua52
Lua 5.2.0  Copyright (C) 1994-2011 Lua.org, PUC-Rio
for i = 1, 2 do print(function() end) end
function: 0x1d51070
function: 0x1d51070

Notice the closures are equivalent (and we could confirm equality
using ==). Notice though:

$ lua52
Lua 5.2.0  Copyright (C) 1994-2011 Lua.org, PUC-Rio
for i = 1, 2 do print(function() return i end) end
function: 0x796070
function: 0x796d30

This produces distinct closures because the upvalues are different.
Namely, the i local is recreated on each iteration of the loop so each
closure references a different i.

It is an optimization to cache closures. This optimization lessens the
performance impact of an anonymous function used in a loop. For

-- Escape non-graphical characters
function escape (str)
  return (str:gsub("[^%w%s%p]", function (a) return
("\\x%02x"):format(a:byte()) end));

In Lua 5.1, if the above escape function is called in a loop, it would
create a closure for the call to gsub each time. These become
"garbage" because they are only used once. The way to correct this is
to pull the function out into a local above the escape function:

-- Lua 5.1 optimization for escape function
  local function helper (a)
    return ("\\x%02x"):format(a:byte())
  function escape (str)
    return (str:gsub("[^%w%s%p]", helper));

In Lua 5.2, this is largely unnecessary as the closure is cached for future use.

o Lua 5.2 now supports goto. Be sure the Raptor fences aren't out before use.

goto forever

The goto mechanism was added partially to solve the lack of a continue
statement and nested break.

for i = 1, 10 do
  if i % 2 == 0 then goto cont end

for i = 1, 10 do
  for j  = 1, 10 do
    for k = 1, 10 do
      if i + j + k == 99 then goto done end

o __pairs and __ipairs metamethods. You can now define how an object
is iterated over using the pairs and ipairs functions.

$ lua52
Lua 5.2.0  Copyright (C) 1994-2011 Lua.org, PUC-Rio
t = {1, 2, 3, a = 1}
setmetatable(t, {__ipairs = pairs}) -- make ipairs work like pairs
for i, v in ipairs(t) do print(i, v) end
1       1
2       2
3       3
a       1

[1] http://www.lua.org/manual/5.2/manual.html#2.2

- Patrick Donnelly
Sent through the nmap-dev mailing list
Archived at http://seclists.org/nmap-dev/

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]