Accurate segmentation of rectal cancer and rectal wall based on high-resolution T2-weighted magnetic resonance imaging (MRI-HRT2) is the basis of rectal cancer staging. However, complex imaging background, highly characteristics variation and poor contrast hindered the research progress of the automatic rectal cancer segmentation. In this study, a multi-task learning network, namely mask segmentation with boundary constraints (MSBC-Net), is proposed to overcome these limitations and to obtain accurate segmentation results by locating and segmenting rectal cancer and rectal wall automatically. Specifically, at first, a region of interest (RoI)-based segmentation strategy is designed to enable end-to-end multi-task training, where a sparse object detection module is used to automatically localize and classify rectal cancer and rectal wall to mitigate the problem of background interference, and a mask and boundary segmentation block is used to finely segment the RoIs; second, a modulated deformable backbone is introduced to handle the variable features of rectal cancer, which effectively improves the detection performance of small objects and adaptability of the proposed model. Moreover, the boundary head is fused into the mask head to segment the ambiguous boundary of the target and constrain the mask head to obtain more refined segmentation results. In total, 592 annotated rectal cancer patients in MRI-HRT2 are enrolled, and the comprehensive results show that the proposed MSBC-Net outperforms state-of-the-art methods with a dice similarity coefficient (DSC) of 0.801 (95\% CI, 0.791-0.811), which can be well extended to other medical image segmentation tasks with high potential clinical applicability.