Improving the Robustness of Object Detection Through a Multi-Camera–Based Fusion Algorithm Using Fuzzy Logic - Khan, Md Nazmuzzaman; Al Hasan, Mohammad; Anwar, Sohel (Frontiers in Artificial Intelligence, 2021)
A single camera creates bounding box (BB) for the detected object with certain accuracy through Convolution neural network (CNN). However, a single RGB camera may not be able to capture the actual object within the BB even if the CNN detector accuracy is high for the object. In this research, we present a solution to this limitation through the usage of multiple cameras, projective transformation, and a fuzzy-logic based fusion. The proposed algorithm generates a ‘confidence score’ for each frame to check the trustworthiness of the BB’s generated by the CNN detector. As a first step toward this solution, we created a two-camera setup to detect objects. Agricultural weed is used as objects to be detected. A CNN detector generates BB for each camera when weed is present. Then a projective transformation is used to project one camera’s image plane to another camera’s image plane. The intersect over union (IOU) overlap of the BB’s is computed when objects are detected correctly. Four different scenarios are generated based on how far the object is from the multi-camera setup and IOU overlap is calculated for each scenario (ground truth). When objects are detected correctly and bounding boxes are at correct distance, the IOU overlap value should be close to the ground truth IOU overlap value. On the other hand, IOU overlap value should differ if BB’s are at incorrect positions. Mamdani fuzzy rules are generated using this reasoning and three different confidence scores (‘High’, ‘OK’, ‘Low’) are given to each frame based on accuracy and position of BB’s. The proposed algorithm was then tested under different conditions to check it’s validity. The confidence score of the proposed fuzzy system for three different scenarios supports the hypothesis that the multi-camera based fusion algorithm improved the overall robustness of the detection system.