Due to the rapid development of deep neural networks (DNNs), ensuring their quality has become increasingly important. However, the test oracle problem poses an obstacle to DNN testing because of the massive unlabeled data. Metamorphic Testing (MT) has proven effective in alleviating the test oracle problem, and many efforts have been made to improve the cost-effectiveness of MT for DNNs. Some approaches focus on selecting good metamorphic relations (MRs), while others target the selection of suspicious source test cases. Since follow-up test cases are generated by combining source test cases with MRs, selecting effective pairs of source test cases and MRs is also quite essential and beneficial for MT. In this paper, we propose CMPS, a multi-objective black-box approach for metamorphic test case pair selection. Considering both uncertainty and diversity, CMPS aims to select pairs that are more likely to trigger failures and detect unique faults in the model. It evaluates uncertainty based on model outputs and assesses diversity through clustering. Furthermore, CMPS can adaptively optimize the selection process based on feedback from the execution results of the selected pairs. We conduct extensive experiments on three datasets and five DNN models to evaluate CMPS’s performance. The experimental results demonstrate that CMPS significantly outperforms baseline approaches in both failure triggering and fault detection.