Feature semantic alignment and information supplement for Text-based person search - Zhou, Hang; Li , Fan; Tian , Xuening; Huang, Yuling (Frontiers in Physics, 2023)
The goal of person-text-image matching is to retrieve images of specific pedestrians using natural language. Although a lot of research results have been achieved in persona text-image matching, existing methods still face some problems.Due to the ambiguous semantic information in the features, aligning the textual features with their corresponding image features is always tricky. Additionally, the absence of semantic information in each local feature of pedestrians poses a significant challenge to the network in extracting robust features that match both modalities. To address these issues, we proposes a model for explicit semantic feature extraction and effective information supplement. On one hand, by attaching the textual and image features with consistent and clear semantic information, the course-grained alignment between the textual and corresponding image features is achieved. On the other hand, an information supplement network is proposed, which captures the relationships between local features of each modality and supplements them to obtain more complete local features with semantic information. In the end, the local features are then concatenated to a comprehensive global feature, which capable of precise alignment of the textual and described image features.We have verified the validity of our method through a large number of experiments.