Social Media Image Classification Benchmarks for Various Disaster Response Tasks

Abstract. During an ongoing disaster event, real-time image classification becomes important for crisis managers for situational awareness and crisis response tasks. Current advances in image classification methods enable the crisis informatics community to develop models for real-time image classification and facilitate humanitarian response tasks. Providing such a facility requires automatically detecting event types, filtering irrelevant image content, categorizing specific humanitarian categories, and assessing the severity level of the damage. To develop such models, there have been dispersed attempts in literature that utilize limited resources (i.e., datasets) within a constrained scope (i.e., specific tasks). In this study, we propose new datasets for disaster type and informativeness classification and relabel existing publicly available datasets for new tasks. We identify exact- and near-duplicates to form non-overlapping data splits, and finally consolidate them to create larger datasets. In our extensive experiments, we benchmark several state-ofthe-art deep learning models and achieve promising results. We will release our datasets and models publicly, aiming to provide proper baselines as well as to spur further research in the crisis informatics community.