Tests

Automated continuous integration (CI) tests (GitHub actions) are triggered by commits to the medigan repository. These CI tests can be found here.

Apart from that, to facilitate testing if medigan is setup correctly and whether all of the features in medigan work as desired, the following set of automated test cases is provided. Below, each test function is described and a command is provided to run each test.

Setup medigan for running tests

Open your command line, and clone medigan from Github with:

git clone https://github.com/RichardObi/medigan.git
cd medigan

To install dependencies and to setup and activate a virtual environment, run:

pip install pipenv
pipenv install
pipenv shell

Test 1: test_medigan_imports

This test checks if medigan can be imported correctly.

python -m tests.tests TestMediganMethods.test_medigan_imports

Test 2: test_init_generators

This test checks if the central generators class can be initialised correctly.

python -m tests.tests TestMediganMethods.test_init_generators

Test 3: test_generate_methods

This test examines whether samples can be created with any of three example generative models (1, 2, 3) in medigan.

python -m tests.tests TestMediganMethods.test_generate_methods

Test 4: test_generate_methods_with_additional_args

Additional key-value pair arguments (kwargs) can be provided to the generate() method of a generative model. This test checks if these additional arguments are passed correctly to the generate method and whether the generate method’s returned result corresponds to the passed arguments.

python -m tests.tests TestMediganMethods.test_generate_methods_with_additional_args

Test 5: test_get_generate_method

The generate() method of any of the generative models in medigan can be returned. This makes it easier to integrate the generate() function dynamically into users’ data processing and training pipelines i.e. avoiding it to reload the model weights each time it is called. This test tests if the generate() method is successfully returned and usable thereafter.

python -m tests.tests TestMediganMethods.test_get_generate_method

Test 6: test_search_for_models_method

The tested function searches for a model by matching provided key words with the information in the model’s config. This test checks whether the expected models are found accordingly.

python -m tests.tests TestMediganMethods.test_search_for_models_method

Test 7: test_find_model_and_generate_method

After searching and finding one specific model, the tested function generates samples with that model. This test checks whether the expected model is found and whether samples are generated accordingly.

python -m tests.tests TestMediganMethods.test_find_model_and_generate_method

Test 8: test_rank_models_by_performance

Provided a list of model ids, the tested function rankes these models by a performance metric. The performance metrics are stored in the models’ config. This test checks whether the ranking worked and whether the expected model is ranked the highest.

python -m tests.tests TestMediganMethods.test_rank_models_by_performance

Test 9: test_find_and_rank_models_by_performance

After searching and finding various models, the tested function ranks these models by a performance metric. This test checks whether the expected model is found, and whether it is the highest ranked one and whether it generated samples accordingly.

python -m tests.tests TestMediganMethods.test_find_and_rank_models_by_performance

Test 10: test_find_and_rank_models_then_generate_method

After searching and finding various models, the tested function ranks these models by a performance metric and generates samples with the highest ranked model. This test checks whether the expected model is found, whether it is the highest ranked one and whether it generated samples accordingly.

python -m tests.tests TestMediganMethods.test_find_and_rank_models_then_generate_method

Test 11: test_get_models_by_key_value_pair

After receiving a key value pair, the tested function returns all models that have that key-value pair in their model config. This test checks whether the expected models are found and returned correctly.

python -m tests.tests TestMediganMethods.test_get_models_by_key_value_pair