Wireshark · Wireshark-dev: Re: [Wireshark-dev] Lemon crashing on Solaris? (was Re: [Wireshark-commits] buildbot failure in Wire

[Guy Harris <guy@xxxxxxxxxxxx>]

(CCing the list this time.)

On Nov 24, 2013, at 5:37 PM, Bálint Réczey <balint@xxxxxxxxxxxxxxx> wrote:

> I tried to trigger a crash using GCC and -ftrapv without success while
> the clang-compiled binary crashed as expected:


	$ gcc -S -O2 test.c
	$ mv test.s test.s.noftrapv
	$ gcc -ftrapv -S -O2 test.c
	$ diff test.s.noftrapv test.s
	$ 

whereas:

	$ clang -S -O2 test.c
	$ mv test.s test.s.noftrapv
	$ clang -ftrapv -S -O2 test.c
	$ diff test.s.noftrapv test.s
	15,21c15
	<       leaq    L_.str(%rip), %rdi
	<       movl    $-2147483550, %esi      ## imm = 0xFFFFFFFF80000062
	<       xorb    %al, %al
	<       callq   _printf
	<       xorl    %eax, %eax
	<       popq    %rbp
	<       ret
	---
	>       ud2
	24,27d17
	<       .section        __TEXT,__cstring,cstring_literals
	< L_.str:                                 ## @.str
	<       .asciz   "res:%d\n"

and, if we make it so that the compiler can't do all the work at compile time:

	$ cat test.c
	#include <limits.h>
	#include <stdio.h>

	int
	foo(int i)
	{
	 int b = 100;
	#ifdef TEST
	 if ((i + b) < i) {
	   printf("overflow!\n");
	   return 1;
	 }
	#endif
	 i += b;
	 printf("res:%d\n",i);
	 return i;
	}
	$ gcc -S -O2 test.c
	$ mv test.s test.s.noftrapv
	$ gcc -ftrapv -S -O2 test.c
	$ diff test.s.noftrapv test.s
	$ clang -S -O2 test.c
	$ mv test.s test.s.noftrapv
	$ clang -ftrapv -S -O2 test.c
	$ diff test.s.noftrapv test.s
	20a21,22
	>       jo      LBB0_1
	> ## BB#2:
	29a32,33
	> LBB0_1:
	>       ud2

so GCC is apparently not actually doing overflow checks, while clang is - and "gcc" is, on my machine, actually llvm-gcc:

	$ gcc --version
	i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
	Copyright (C) 2007 Free Software Foundation, Inc.
	This is free software; see the source for copying conditions.  There is NO
	warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(you might not be surprised to hear that there's no button under my machine's trackpad - the trackpad is the one-and-only mouse button :-))

On SPARC, there's a TRAPV instruction that traps if the integer overflow condition code bit is set; on 32-bit x86, there's INTO, which traps if the overflow condition code bit is set, but it's illegal in 64-bit x86.  JO jumps if the overflow bit is set in both 32-bit and 64-bit mode, and UD2 is an instruction that's defined to trap (in 32-bit and 64-bit mode), so jumping to a UD2 if the overflow flag is set is a valid "trap if overflow" instruction sequence even on x86-64.

I don't know why GCC isn't actually implementing -ftrapv on x86 but is implementing it on SPARC, however; it's a little more work on x86 than just sticking in TRAPVs on SPARC, but not *that* much more.