davisinteractive.session

DavisInteractiveSession

DavisInteractiveSession(self,
                        host='localhost',
                        user_key=None,
                        davis_root=None,
                        subset='val',
                        shuffle=False,
                        max_time=None,
                        max_nb_interactions=5,
                        metric_to_optimize='J_AND_F',
                        report_save_dir=None)

Class which allows to interface with the evaluation.

Arguments

• host: String. Host of the evuation server. Only localhost outside challenge timeframe.
• user_key: String. User identifier (e.g. email). If the session is being run in localhost, user_key does not need to be specified (username will be used).
• davis_root: String. Path to the Davis dataset root path. Necessary for evaluation when host='localhost'.
• subset: String. Subset to evaluate. If host='localhost' subset can only be train or val subsets. If the evaluation is performed against a remote server, this parameter is ignored and the subset will be given by the remote server.
• shuffle: Boolean. Shuffle the samples when evaluating.
• max_time: Integer. Maximum time to evaluate a sample (a sequence with a certain set of initial scribbles). This time should be set per object as it adapts to the number of objects internally. If max_nb_interactions is also specified, this time is defined as the time available to perform all the interactions.
• max_nb_interactions: Integer. Maximum number of interactions to evaluate per sample.
• metric_to_optimize: Enum. Metric targeting to optimize. Possible values: J, F or J_AND_F. This value will be ignored when running against a remote server.
• report_save_dir: String. Path to the directory where the report will be stored during the evaluation. By default is the current working directory. A temporal file will be storing snapshots of the results on this same directory with a suffix .tmp.

next

DavisInteractiveSession.next()

Iterate to the next iteration/sample of the evaluation process.

This function moves the iteration to the next iteration or to the next sample in case the maximum number of iterations or maximum time have been hit. This function can be used as control flow on user's code to know until which iteration the evuation is being performed.

Returns

bool: Indicates whether the evaluation is still taking place.

get_scribbles

DavisInteractiveSession.get_scribbles(only_last=False)

Ask for the next scribble

There is the possibility to ask for only the last scribble. By default, all scribbles obtained for the current sample are returned.

This method returns information about the sequence of the sample being evaluated, the scribbles and whether it is a new sample. This information might be useful for the user to perform any operation like loading a model for a new sequence.

Arguments

• only_last: Boolean.

Returns

(string, dict, bool): Returns the name of the sequence of the current sample, the scribbles of the current sample and a boolean indicating whether it is the first iteration of the given sample, respectively.

scribbles_iterator

DavisInteractiveSession.scribbles_iterator(*args, **kwargs)

Iterate over all the samples and iterations to evaluate.

Instead of running a while loop with DavisInteractiveSession.next and then call to DavisInteractiveSession.get_scribbles, you can iterate with this generator:

Example

for sequence, scribble, new_sequence in sess.scribbles_iterator():
    # Predict with model

Arguments

• *args, **kwargs: This arguments will be passed internally to DavisInteractiveSession.get_scribbles method.

Yields

(string, dict, bool): Yields the name of the sequence of the current sample, the scribbles of the current sample and a boolean indicating if it is the first iteration of the given sample, respectively.

submit_masks

DavisInteractiveSession.submit_masks(pred_masks,
                                     next_scribble_frame_candidates=None
                                     )

Submit the predicted masks.

Arguments

• pred_masks: Numpy array with the predicted mask for the current sample. The array must be of dtype=np.int and of size equal to the 480p resolution of the DAVIS dataset.
• next_scribble_frame_candidates: List of Integers. Optional value specifying the possible frames from which generate the next scribble. If values given, the next scribble will be performed in the frame where the evaluation metric scores the least on the list of given frames. Invalid frames indexes are ignored.

get_report

DavisInteractiveSession.get_report()

Gives the current report of the evaluation

Returns

pd.DataFrame: Dataframe with the current evaluation results. This DataFrame contains the same table as the store on report_save_dir.

get_global_summary

DavisInteractiveSession.get_global_summary(save_file=None)

Gives a summary from the current session.

In the case the session is running against the remote server, when calling this function, the current session will be marked as completed.

Arguments

• save_file: String or Path. Path to store the global summary of the session. By default does not save it.

Returns

Dictionary: Dictionary of parameters that summarize all the session.