Postprocessing
Non-maximum suppression
First the commonly used NMS with bounding boxes, that prioritizes either confidence score (default) or bounding box area.
non_max_suppression_fast
non_max_suppression_fast (boxes, scores, overlap_thresh:float, sort_criterion:str='score')
Possibility to sort boxes by score (default) or area
Non-max suppression can in theory be applied also on polygons, but it hasn’t been used in any publications as far as I know.
If non_max_suppression_poly is used to eliminate polygons, threshold might need to be smaller than typical value of 0.7 that is used.
non_max_suppression_poly
non_max_suppression_poly (geoms, scores, overlap_thresh:float, sort_criterion:str='score')
Possibility to sort geoms by score (default) or area
Some utils to run above functions to GeoDataFrames
do_min_rot_rectangle_nms
do_min_rot_rectangle_nms (gdf:geopandas.geodataframe.GeoDataFrame, nms_thresh=0.7, crit='score')
do_poly_nms
do_poly_nms (gdf:geopandas.geodataframe.GeoDataFrame, nms_thresh=0.1, crit='score')
do_nms
do_nms (gdf:geopandas.geodataframe.GeoDataFrame, nms_thresh=0.7, crit='score')
Weighted boxes fusion
Originally presented by Solovyev et al (2021), and available in [https://github.com/ZFTurbo/Weighted-Boxes-Fusion]. Code presented here is modified to keep track of original bounding boxes for mask fusion, and due to numpy version requirements we do not use numba here.
As WBF expects normalized coordinates, first some helpers to normalize and denormalize geocoordinates.
denormalize_bbox_coords
denormalize_bbox_coords (tot_bounds, bboxes)
normalize_bbox_coords
normalize_bbox_coords (tot_bounds, bboxes)
weighted_boxes_fusion
weighted_boxes_fusion (boxes_list, scores_list, labels_list, weights=None, iou_thr=0.55, skip_box_thr=0.0, conf_type='avg', allows_overflow=False)
:param boxes_list: list of boxes predictions from each model, each box is 4 numbers. It has 3 dimensions (models_number, model_preds, 4) Order of boxes: x1, y1, x2, y2. We expect float normalized coordinates [0; 1] :param scores_list: list of scores for each model :param labels_list: list of labels for each model :param weights: list of weights for each model. Default: None, which means weight == 1 for each model :param iou_thr: IoU value for boxes to be a match :param skip_box_thr: exclude boxes with score lower than this variable :param conf_type: how to calculate confidence in weighted boxes. ‘avg’: average value, ‘max’: maximum value, ‘box_and_model_avg’: box and model wise hybrid weighted average, ‘absent_model_aware_avg’: weighted average that takes into account the absent model. :param allows_overflow: false if we want confidence score not exceed 1.0 :return: boxes: boxes coordinates (Order of boxes: x1, y1, x2, y2). :return: scores: confidence scores :return: labels: boxes labels :return: originals: original boxes
find_matching_box_quickly
find_matching_box_quickly (boxes_list, new_box, match_iou)
Reimplementation of find_matching_box with numpy instead of loops. Gives significant speed up for larger arrays (~100x). This was previously the bottleneck since the function is called for every entry in the array.
get_weighted_box
get_weighted_box (boxes, conf_type='avg')
Create weighted box for set of boxes :param boxes: set of boxes to fuse :param conf_type: type of confidence one of ‘avg’ or ‘max’ :return: weighted box (label, score, weight, x1, y1, x2, y2)
prefilter_boxes
prefilter_boxes (boxes, scores, labels, weights, thr)
bb_intersection_over_union
bb_intersection_over_union (A, B)
do_wbf
do_wbf (gdf:geopandas.geodataframe.GeoDataFrame, iou_thr=0.55, skip_box_thr=0.5)
Run weighted_boxes_fusion and returns a gpd.GeoDataFrame where geometries are replaced by new bounding boxes. Do not use with instance segmentation data unless you want to replace your results with bounding boxes
do_wsf
do_wsf (gdf:geopandas.geodataframe.GeoDataFrame, iou_thr=0.55, skip_box_thr=0.5)
Smoothing and filling holes
Below functions are run before converting IceVision preds to COCO or shapefile format.
dilate_erode
dilate_erode (preds:list)
Run dilation followed by erosion in order to smooth masks
fill_holes
fill_holes (preds:list)
Run binary_fill_holes to predicted binary masks