medigan.select_model package

Submodules

medigan.select_model.matched_entry module

MatchedEntry class that represents one match of a key value pair of a model’s config against a search query.

class medigan.select_model.matched_entry.MatchedEntry(key: str, value, matching_element: Optional[str] = None)[source]

Bases: object

MatchedEntry class: One target key-value pair that matches with a model’s selection config.

Parameters

key (str) – string that represents the matched key in model selection dict
value – represents the key’s matched value in the model selection dict
matching_element (str) – string that was used to match the search value

key

string that represents the matched key in model selection dict

Type: str

value: represents the key’s matched value in the model selection dict

matching_element

string that was used to match the search value

Type: str

medigan.select_model.model_match_candidate module

ModelMatchCandidate class that holds data to evaluate if a generative model matches against a search query.

class medigan.select_model.model_match_candidate.ModelMatchCandidate(model_id: str, target_values: list, target_values_operator: str = 'AND', is_case_sensitive: bool = False, are_keys_also_matched: bool = False, is_match: bool = False)[source]

Bases: object

ModelMatchCandidate class: A prospectively matching model given the target values as model search params.

Parameters

model_id (str) – The generative model’s unique id
target_values (list) – list of target values used to evaluate if a ModelMatchCandidate instance is a match
target_values_operator (str) – the operator indicating the relationship between values in the evaluation of ModelMatchCandidate instances. Should be either “AND”, “OR”, or “XOR”.
is_case_sensitive (bool) – flag indicating whether the matching of values (and) keys should be case-sensitive
are_keys_also_matched (bool) – flag indicating whether, apart from values, keys should also be matched
is_match (bool) – flag indicating whether the ModelMatchCandidate instance is a match

model_id

The generative model’s unique id

Type: str

target_values

list of target values used to evaluate if a ModelMatchCandidate instance is a match

Type: list

target_values_operator

the operator indicating the relationship between values in the evaluation of ModelMatchCandidate instances. Should be either “AND”, “OR”, or “XOR”.

Type: str

is_case_sensitive

flag indicating whether the matching of values (and) keys should be case-sensitive

Type: bool

are_keys_also_matched

flag indicating whether, apart from values, keys should also be matched

Type: bool

matched_entries

contains iteratively added MatchedEntry class instances. Each of the MatchedEntry instances indicates a match between one of the user specified values in self.target_values and the selection config keys or values of the model of this ModelMatchCandidate.

Type: list

is_match

flag indicating whether the ModelMatchCandidate instance is a match

Type: bool

add_matched_entry(matched_entry: medigan.select_model.matched_entry.MatchedEntry) → None[source]: Add a MatchedEntry instance to the matched_entries list.

check_if_is_match() → bool[source]

Evaluates whether the model represented by this instance is a match given search values and operator.

The matching element from each MatchEntry of this instance (‘self.matched_entries’) are retrieved. To be a match, this instance (‘self’) needs to fulfill the requirement of the operator, which can be of value ‘AND’, or ‘OR’, or ‘XOR’. For example, the default ‘AND’ requires that each search value (‘self.target_values’) has at least one corresponding MatchEntry, while in ‘OR’ only one of the search values needs to have been matched by a corresponding MatchedEntry.

Returns: flag that, only if True, indicates that this instance is a match given the search values and operator.
Return type: bool

get_all_matching_elements() → list[source]

Get the matching element from each of the MatchedEntry instances in the matched_entries list.

Returns: list of all matching elements (i.e. string that matched a search value) from each MatchedEntry in matched_entries
Return type: list

medigan.select_model.model_selector module

Model selection class that describes, finds, compares, and ranks generative models.

class medigan.select_model.model_selector.ModelSelector(config_manager: Optional[medigan.config_manager.ConfigManager] = None)[source]

Bases: object

ModelSelector class: Given a config dict, gets, searches, and ranks matching models.

Parameters: config_manager (ConfigManager) – Provides the config dictionary, based on which models are selected and compared.

config_manager

Provides the config dictionary, based on which models are selected and compared.

Type: ConfigManager

model_selection_dicts

Contains a dictionary for each model id that consists of the model_id and the selection config of that model

Type: list

find_matching_models_by_values(values: list, target_values_operator: str = 'AND', are_keys_also_matched: bool = False, is_case_sensitive: bool = False) → list[source]

Search for values (and keys) in model configs and return a list of each matching ModelMatchCandidate.

Uses self.recursive_search_for_values to recursively populate each ModelMatchCandidate with MatchedEntry instances. After populating, each ModelMatchCandidate is evaluated based on the provided target_values_operator and values list using ModelMatchCandidate.check_if_is_match.

Parameters

values (list) – list of values used to search and find models corresponding to these values
target_values_operator (str) – the operator indicating the relationship between values in the evaluation of model search results. Should be either “AND”, “OR”, or “XOR”.
are_keys_also_matched (bool) – flag indicating whether, apart from values, the keys in the model config should also be searchable
is_case_sensitive (bool) – flag indicating whether the search for values (and) keys in the model config should be case-sensitive.

Returns

a list of ModelMatchCandidate class instances each of which was successfully matched against the search values.

Return type

list

find_models_and_rank(values: list, target_values_operator: str = 'AND', are_keys_also_matched: bool = False, is_case_sensitive: bool = False, metric: str = 'SSIM', order: str = 'asc') → list[source]

Search for values (and keys) in model configs, rank results and return sorted list of model dicts.

Parameters

values (list) – list of values used to search and find models corresponding to these values
target_values_operator (str) – the operator indicating the relationship between values in the evaluation of model search results. Should be either “AND”, “OR”, or “XOR”.
are_keys_also_matched (bool) – flag indicating whether, apart from values, the keys in the model config should also be searchable
is_case_sensitive (bool) – flag indicating whether the search for values (and) keys in the model config should be case-sensitive.
metric (str) – The key in the selection dict that corresponds to the metric of interest
order (str) – the sorting order of the ranked results. Should be either “asc” (ascending) or “desc” (descending)

Returns

a list of the searched and matched model dictionaries containing metric and model_id, sorted by metric.

Return type

list

get_models_by_key_value_pair(key1: str, value1: str, is_case_sensitive: bool = False) → list[source]

Get and return a list of model_id dicts that contain the specified key value pair in their selection config.

The key param can contain ‘.’ (dot) separations to allow for retrieval of nested config keys such as ‘execution.generator.name’. If key1 points to a list, any value in the list that matches value1` is accepted.

Parameters

key1 (str`) – The key in the selection dict
value1 (str) – The value in the selection dict that corresponds to key1
is_case_sensitive (bool) – flag to evaluate keys and values with case sensitivity if set to True

Returns

a list of the dictionaries each containing a model’s model_id and the found key-value pair in the models config

Return type

list

get_selection_criteria_by_id(model_id: str, is_model_id_removed: bool = True) → dict[source]

Get and return the selection config dict for a specific model_id.

Parameters

model_id (str) – The generative model’s unique id
is_model_id_removed (bool) – flag to to remove the model_id from first level of each dictionary.

Returns

a dictionary corresponding to the selection config of a model

Return type

dict

get_selection_criteria_by_ids(model_ids: Optional[list] = None, are_model_ids_removed: bool = True) → list[source]

Get and return a list of selection config dicts for each of the specified model_ids.

Parameters

model_ids (list) – A list of generative models’ unique ids
are_model_ids_removed (bool) – flag to remove the model_ids from first level of dictionary.

Returns

a list of dictionaries each corresponding to the selection config of a model

Return type

list

get_selection_keys(model_id: Optional[str] = None) → list[source]

Get and return all first level keys from the selection config dict for a specific model_id.

Parameters: model_id (str) – The generative model’s unique id
Returns: a list containing the keys as strings of the selection config of the model_id.
Return type: list

get_selection_values_for_key(key: str, model_id: Optional[str] = None) → list[source]

Get and return the value of a specified key of the selection dict in the config for a specific model_id.

The key param can contain ‘.’ (dot) separations to allow for retrieval of nested config keys such as ‘execution.generator.name’

Parameters

key (str) – The key in the selection dict
model_id (str) – The generative model’s unique id

Returns

a list of the values that correspond to the key in the selection config of the model_id.

Return type

list

rank_models_by_performance(model_ids: Optional[list] = None, metric: str = 'SSIM', order: str = 'asc') → list[source]

Rank model based on a provided metric and return sorted list of model dicts.

The metric param can contain ‘.’ (dot) separations to allow for retrieval via nested metric config keys such as ‘downstream_task.CLF.accuracy’. If the value found for the metric key is of type list, then the largest value in the list is used for ranking if order is descending, while the smallest value is used if order is ascending.

Parameters

model_ids (list) – only evaluate the model_ids in this list. If none, evaluate all available model_ids
metric (str) – The key in the selection dict that corresponds to the metric of interest
order (str) – the sorting order of the ranked results. Should be either “asc” (ascending) or “desc” (descending)

Returns

a list of model dictionaries containing metric and model_id, sorted by metric.

Return type

list

recursive_search_for_values(search_dict: dict, model_match_candidate: medigan.select_model.model_match_candidate.ModelMatchCandidate)[source]

Do a recursive search to match values in the search_dict with values (and keys) in a model’s config.

The provided ModelMatchCandidate instance is recursively populated with MatchedEntry instances. A MatchedEntry instance contains a key-value pair found in the model’s config that matches with one search term of interest.

The search terms of interest are stored in ModelMatchCandidate.target_values. The model’s selection config is provided in the ‘search_dict’.

To traverse the search_dict, the value for each key in the search_dict is retrieved.

If that value is of type dictionary, the function calls itself with that nested dictionary as new search_dict.
If that value is of type list, each value in the list is compared with each search term of interest in ModelMatchCandidate.target_values.
If the value of the search_dict is of another type (i.e. str), it is compared with each search term of interest in ModelMatchCandidate.target_values.

Parameters

search_dict (dict) – contains keys and values from a model’s config that are matched against a set of search values.
model_match_candidate (ModelMatchCandidate) – a class instance representing a model to be prepared for evaluation (populated with MatchedEntry objects), as to whether it is a match given its search values (self.target_values).

Returns

a ModelMatchCandidate class instance that has been populated with MatchedEntry class instances.

Return type

list

medigan.select_model package

Submodules

medigan.select_model.matched_entry module

medigan.select_model.model_match_candidate module

medigan.select_model.model_selector module

Module contents