Very recently, there are many research attempts in investigating DA approaches to alleviate inter-site distributional variability, among which UDA methods demonstrated their advantages in exploiting unlabeled target samples [21]. Such UDA methods can be categorized into two groups: (1) image translation and (2) feature alignment approaches. The former one performs image appearance alignment [18, 23]. The resultant models translate images across domains using GAN-based networks [24]. However, texture similarity between the image of synthesized target and the source would be crucial for the PLDC problem. The DA process would fail with insufficient texture similarity, particularly found in the generated lesion area [23c]. Besides, lesions could be missed during the translation process due to various transferability among image regions, thus worsening the DA process [32]. Moreover, the GAN models would distort the non-lesion region's appearance, further causing unreliable lesion assessment results [25].