Documentation/Widget testing

Overview

Testing of both CTK widgets and application built on top of CTK could leverage the capabilities offered by the QtTesting library developed by Paraview folks.

How added the testing framework to CTK :

• Turn the variable CTK_USE_QTTESTING into ON.

If the option CTK_USE_QTTESTING is enabled, the build system will :

1. Download external project QtTesting
2. Compile the CTKQtTesting Library
   • Including the class :
     • ctkEventTranslaterPlayerWidget : Widget used by the Unit Test - Image
     • ctkQtTestingUtility : Class used by the application - inherits from pqTestUtility
     • ctkXMLEventObserver : Recover the Event and write the .xml file.
     • ctkXMLEventSource : Read the .xml file and create the events.
3. For CTKWidget - CTKVisualizationWidget - CTKCoreWidget :
   • Include the event translator and player required for custom widgets. (Located in the same folder as their associated widget sources)
4. If BUILD_TESTING is enabled: widgets EventTranslatorTests will be compiled.
   • These tests will use ctkEventTranslaterPlayerWidget and ensure that events associated with widgets can be properly recorded and played.

Note:
Option CTK_USE_QTTESTING will be automatically enabled if BUILD_TESTING is ON

Adding the testing framework into your application

Adding the testing framework will allow users to recover script, but also a easy way to create tutorial or to test all the widget.
You can see the application exemple created in CTK to have an exemple:

How

Follow these steps to add this testing framework to your application.

1. Create a variable ctkQtTestingUtiliy - can be added to your mainWindow class.
   • And initialize this variable as following :

this->TestUtility = new ctkQtTestingUtility(this);
this->TestUtility->addEventObserver("xml", new ctkXMLEventObserver(this));
this->TestUtility->addEventSource("xml", new ctkXMLEventSource(this));

Note: You can have your own EventObserver and EventSource

1. Create a button Record, and link it to the ctkQtTestUtility slot record
2. Create a button Play, and link it to the ctkQtTestingUtility slot play

QObject::connect(Ui.RecordButton, SIGNAL(clicked(bool)), this, SLOT(record()));
QObject::connect(Ui.PlayBackButton, SIGNAL(clicked(bool)), this, SLOT(play()));

Limits

Note that there are still severals limits to this testing framework :

1. The ctkVTKRenderView had to have the same size between the record and the playback.
   • To solve the issue, ctkTestingUtility record all the application property - size, state, font ... - and before a playback, if these property are differents, ask the user if he wants to load the settings.
     Note: We are working on this limit to find a better solution.
2. All the CTK Widgets are not yet tested - Below the summary table -

Issues found on QtTesting

• QSpinBox/QDoubleSpinBox :
  • A long click on the up/down arrow -> just one click even if the value change severals times.
  • Fast click -> Double click -> no effect.
  • clik on the up/down arrow just after editing the spin Box -> no effect.

Solve : Contribut to the QtTesting by modified the pqSpinBoxEventTranslator
Here the link to my Github for the QSpinBox
Here the link to my Github for the QDoublSpinBox

• In pq3DView :
  • The scroll button has no effect - Implemented for ctk.
  • Shift between the current/Expected image.

Solve : new Implementation with the ctkVTKRenderViewEventTranslator and the ctkVTKRenderViewEventPlayer :

1. Change the normalization into a normalization by the widget center.
2. Implemented the scroll action
3. Add a better rounded when we cast from double to int to not have shift issue.

• QComboBox when it is editable :
  • The hight-event "set_sting" crash if we edit the comboBox.
    • Exemple : If we have an item "foo", we are going to edit the comboBox with "f", but the item "f" doesn't exist, and the player will crash.

Solve : ...

Building

Milestones

1. Integrate QtTesting has an external projects
2. Add unit test to all the widget - Below, the summuray table -
   1. Implement ctkEventTranslaterPlayerWidget
   2. Create custom translator/player if needed
   3. Implement widgets EventTranslatorTests
3. Create an Application test - ctkQtTesting -

Player/Translators Widget Testing

Summury table, to know each widgets' state.
The state can be Done : , Done with QtTesting issue : , in progress : , or without unit test: .

CTKWidgets Widgets State Priority Notes ctkActionsWidget ctkAddRemoveComboBox ctkAxesWidget Translator/player implemented ctkBasePopupWidget NA ctkPopupWidget inherit of it. See ctkPopupWidget ctkButtonGroup ctkCheckableComboBox ctkCheckableHeaderView ctkCheckBoxPixmaps Not used in Slicer ctkCheckablePushButton Not used in Slicer ctkComboBox ctkCompleter NA ctkCollapsibleButton ctkCollapsibleGroupBox ctkColorDialog ctkColorPickerButton ctkConsole May have a problem with the command ctkConfirmExitDialog ctkCoordinatesWidget ctkCrosshairLabel ctkDateRangeWidget Problem with QDateTimer ctkDirectoryButton - TO VERIFY Value OK but no dialog ctkDoubleRangeSlider ctkDoubleSlider ctkDynamicSpacer NA ctkErrorLogStatusMessageHandler NA Not a Widget ctkErrorLogWidget ctkExpandButton ctkFileDialog ctkFittedTextBrowser NA This widget just display text. ctkFlowLayout NA Layout are not tested through the event translator/player mechanism ctkFontButton ctkIconEnginePlugin NA Plugin are not tested through the event translator/player mechanism ctkLayoutManager NA ctkMaterialPropertyPreviewLabel NA ctkMaterialPropertyWidget ctkMatrixWidget no visuel effects ctkMenuButton Use in Slicer ? ctkMenuComboBox Developer should make sure menus have a parent. See https://bugreports.qt.nokia.com/browse/QTBUG-20929 ctkModalityWidget ctkPathLineEdit Note that the first time the listview is displayed, it will show on the top left corner of the screen. ctkPixmapIconEngine NA ctkPopupWidget ctkQImageView NA ctkRangeSlider ctkRangeWidget ctkScreenshotDialog ctkSearchBox ctkSettings NA ctkSettingsDialog ctkSettingsPanel See ctkSettingsDialog ctkSignalMapper NA This object is not a widget and doesn't have to be tested through the event translator/player mechanism ctkSimpleLayoutManager NA ctkSliderWidget Translator and player if popup ctkTestApplication NA This is a utility class. Shouldn't be tested using the event translator/player mechanism ctkThumbnailListWidget ctkThumbnailWidget ctkToolTipTrapper NA This is a utility class. Shouldn't be tested using the event translator/player mechanism ctkTransferFunctionBarsItem ctkTransferFunctionControlPointsItem ctkTransferFunctionGradientItem ctkTransferFunctionItem ctkTransferFunctionScene ctkTransferFunctionView ctkTreeComboBox Note: This won't work if there multiple items with a similar name. This is not an issue in Slicer since selection is based on string ctkWidgetsUtils NA This is a utility class. Shouldn't be tested using the event translator/player mechanism ctkWorkflowAbstractPagedWidget ctkWorkflowButtonBoxWidget ctkWorkflowGroupBox ctkWorkflowStackedWidget ctkWorkflowTabWidget ctkWorkflowWidget ctkWorkflowWidgetStep

CTKVisualizationVTKWidgets Widgets State Priority Notes ctkVTKAbstractMatrixWidget See ctkVTKMatrixWidget ctkVTKAbstractView See ctkVTKRenderView ctkVTKChartView ctkVTKDataSetArrayComboBox ctkVTKDataSetModel ctkVTKMagnifyView Not used in Slicer ctkVTKMatrixWidget See ctkVTKMatrixWidget - There is no need for specific translator & player ctkVTKRenderView Translator/Player implemented What we want : - Wheel event - Better normalization - Not working if the renderView has a different size ctkVTKScalarBarWidget QSpinBox issue + key "."(pad) on the line edit ctkVTKScalarsToColorsUtils ctkVTKScalarsToColorsView ctkVTKScalarsToColorsWidget ctkVTKSliceView Since it's based on QVTKWidget - See ctkVTKRenderView translator & player ctkVTKSurfaceMaterialPropertyWidget ctkVTKTextPropertyWidget ctkVTKThresholdWidget ctkVTKThumbnailView ctkVTKVolumePropertyWidget ctkVTKWidgetsUtils NA CTKDICOMWidgets Widgets State Priority Notes ctkDICOMAppWidget ctkDICOMDatasetView ctkDICOMDirectoryListWidget ctkDICOMImage ctkDICOMImportWidget ctkDICOMItemTreeModel ctkDICOMListenerWidget ctkDICOMQueryResultsTabWidget ctkDICOMQueryRetrieveWidget ctkDICOMQueryWidget ctkDICOMServerNodeWidget ctkDICOMThumbnailGenerator ctkDICOMThumbnailListWidget qMRMLWidgets Widgets State Priority Notes qMRMLCaptureToolBar qMRMLCheckableNodeComboBox qMRMLClipNodeWidget qMRMLCollapsibleButton qMRMLColorListView qMRMLColorPickerWidget qMRMLColorTableComboBox qMRMLColorTableView qMRMLDisplayNodeWidget qMRMLEventBrokerWidget qMRMLLabelComboBox qMRMLLayoutWidget qMRMLLinearTransformSlider qMRMLListWidget qMRMLMatrixWidget qMRMLNavigationView qMRMLNodeComboBox qMRMLRangeWidget qMRMLROIWidget qMRMLScalarInvariantComboBox qMRMLSceneFactoryWidget qMRMLSceneViewMenu qMRMLScreenShotDialog qMRMLSliceControllerWidget qMRMLSliceWidget qMRMLThreeDViewControllerWidget qMRMLThreeDView qMRMLThreeDWidget qMRMLTransformSliders NA qMRMLTreeView qMRMLVolumeInfoWidget qMRMLVolumeThresholdWidget qMRMLWindowLevelWidget

Screenshots

• 
  Simple application with recording / playing Qt testing capabilities
• 
  Little exemple of the widget testing