Wireshark-dev: Re: [Wireshark-dev] Lua 5.3
Date Index · Thread Index · Other Months · All Mailing Lists
Date Prev · Date Next · Thread Prev · Thread Next
From: João Valverde <joao.valverde@xxxxxxxxxxxxxxxxxx>
Date: Sat, 20 Aug 2016 14:47:24 +0100


On 08/20/2016 02:06 PM, João Valverde wrote:



On 08/20/2016 02:03 PM, João Valverde wrote:



On 08/20/2016 01:18 PM, Peter Wu wrote:

On Fri, Aug 19, 2016 at 05:38:57PM +0100, João Valverde wrote:



On 08/19/2016 04:05 PM, João Valverde wrote:



On 08/19/2016 03:56 PM, João Valverde wrote:



On 08/19/2016 02:54 PM, Peter Wu wrote:

On Mon, Aug 08, 2016 at 09:17:35PM +0100, João Valverde wrote:



On 08/08/2016 05:58 PM, Pascal Quantin wrote:

Hi João,

2016-08-08 18:52 GMT+02:00 João Valverde
<joao.valverde@xxxxxxxxxxxxxxxxxx
<mailto:joao.valverde@xxxxxxxxxxxxxxxxxx>>:

    Is moving to Lua 5.3 something we should look into?

    The 64 bit integer support seems really promising.


Hariel explained us that Lua 5.3 was a completely different
language (a
bit like what you have between Python 2.X and 3.X). You can find
much
more info (from people knowing what they are taling about - so
not me
;)) in bug 10881.

Pascal.



Thanks for that Pascal. The only sane way to approach the issue
IMHO
is to
accept that this may and probably will break backward compatibility
(not
even think about supporting 5.1 or 5.2) and just consider whether
that break
is justified (hint: it is).


Why is it justified to break backwards compatibility and move from
5.2
to 5.3 without the ability to chose for 5.2? What is the killer
feature
of 5.3 that makes it totally worth to possibly break older
dissectors?

The disadvantage of C plugins is that it had to be recompiled for
newer
versions. With a move from 5.2 to 5.3 and also removing GRegex and
bitop
you make it quite likely to break Lua dissectors in some way.

I have once written a Lua library in C, interfacing with Libgcrypt
for
which I studied the Lua manual. The API changes with 5.3 were not
that
significant if I remember correctly (though you have to be careful
with
providing a compatibility layer), but the ABI is certainly not
compatible.

In the recent proposed patches, you seem to have no issues with
breaking
backwards compatibility. Have you developed Lua dissectors before?
Breaking things for the sake of "shiny, new, future" is not an
acceptable motivation, there must be something more appealing to
justify
such breakage. Having 64-bit integer support, but taking away the
bitop
library is a net loss without even considering the other factors.



Doesn't Lua 5.3 provide native bit operators? If so there is not net
loss of functionality. That was my reasoning at least.

The language incompatibilities between 5.2 and 5.3 are minor. The
wireshark API is exactly the same.

LPeg is more powerful and Lua-thonic than lrexlib, but there is a
learning curve for that, no doubt. For anyone relying on lrexlib,
it's a
significant break. We can keep lrexlib, that's not a problem and
it is
orthogonal to the other changes.

As far as killer features go, besides the obvious, how about better
UTF-8 support?

I don't have time for a more detailed answer right now but I'd
like to
say I think this change is entirely justified but I also completely
understand disagreeing with that opinion.


I'm referring to the upgrade to Lua 5.3 here, i.e, breaking backward
compatibility, same as any other Lua script moving from 5.1/5.2 to
5.3.


Let me also ask, Peter, you're pushing back against building Lua from
source
and also pushing back against upgrading to 5.3.


I am against pulling the Lua source tree in the tree, reasons were
already given in https://code.wireshark.org/review/#/c/17172

I am not against adding Lua 5.3 compatibility, but would like to see
compatibility for existing dissectors. Keeping support for 5.2 (or even,
argh, 5.1) would allow distros/users to build with older versions if
they desire.


I'm not seeing how that is a viable long-term option on Linux.


If 5.3 compatibility is added it should be no problem. After looking at
the actual changes (https://www.lua.org/manual/5.3/readme.html#changes
and also the linked incompatibilities), I see the following changes to
the Lua code:

 - 64-bit integer support. Nice!
 - Bitwise operators. Cool, but we would have to add a compatibility
   library to keep old dissectors working.
 - UTF-8 "support". This just adds some helper functions that can
   calculate lengths, converts between codepoints and chars and adds a
   %U format specifier. It adds no new data type so won't really affect
   the WSLUA API.

So you are right that there are not many changes in that area. On the
C library side we have also have no really window shattering changes
other than the integer stuff. Of course you would have to recompile any
C libraries, but I realize that it is probably a special case.


I think there is a disconnect here because you are seeing Lua as a
system dependency. I see it as Wireshark's own embedded language
interpreter (although developed by the Lua team under a suitable
license).

(Sorry for my brevity, I really appreciate your input).


I also meant to add that it would be better to rip-off the compatibility
band-aid, IMO, if the 5.3 features justify the upgrade.
I'm not aware of any policy that says our Lua dialect and APIs must be stable forever. That's nice, I don't like fixing working code as much as the next person, but sometimes reality intervenes. It would be difficult to provide that guarantee, seeing as Lua 5.2 is already unmaintained upstreams. 

But if change 17172 is merged then it becomes at least possible.