Wireshark · Wireshark-dev: Re: [Wireshark-dev] GUI testing

[Roland Knall <rknall@xxxxxxxxx>]

Hi Dario

We do extensive UI testing, but this has some flaws:

Basically, as it works, X/Y positioning does not work across plattforms at all or even on Windows particularly good. ToolKits like Ranorex use the internal UI registration to select object names for each element and use that as hotspot for performing actions. The difference is, that changes in the UI (minor ones like increasing the border of a table cell by 1) don't lead to broken UI tests, but the caveat is here a major offset in developing the tests (usually those have to be "programmed" somehow as well).

I don't think headless testing will work Windows based, but I could assume, that it might work X11 or Wayland based.

CI based is 90% redundant because we already do something like that with tshark and the fuzzy tests. Doing it in a UI would need to be special just for features behaving differently when displaying with the Qt interface, and for now, I can't think of anything that would match that special case. If there is something, I'd rather investigate on how to trigger that with tshark as well.

That being said, I am all for it, but feature testing is a never ending story, and to be quite honest, I would give unittesting the Qt framework and creating a Qt unittest framework first, a higher priority. 

kind regards

Roland

Am Mo., 14. Jan. 2019 um 09:11 Uhr schrieb Dario Lombardo <lomato@xxxxxxxxx>:

Hi

I am investigating technologies for testing the GUI component of Wireshark. I have in mind 2 scenarios

1) feature testing

In this scenario I'd like to launch qt ui and have a software that acts like a user, by opening menus and clicking on buttons. The goal of it is to check that some features are working as expected.

2) regression testing

In this scenario the test is run in a CI environment. The ui is launched against some capture files with different options and we could either: (1) verify it doesn't crash, (2) check resulting output wherever possible.

Remember that usually CI machines aren't regular machines but they are headless with no graphics. Travis docs explain how to do headless testing in a web scenario

https://docs.travis-ci.com/user/gui-and-headless-browsers/

Is anyone aware of software/techniques useful in those scenarios? Any shareable experience would be useful as well.

Consider that at the moment Wireshark CIs are based on travis, gitlab-ci, appveyor.

Thanks.

Dario.

--

Naima is online.

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe