Acknowledgements
This work is partially supported by the Research Grants Council (RGC) of Hong Kong (Ref. No.: 17209021, 17207020, 17205919, 17206818, T42-409/18-R), Innovation and Technology Commission (ITC) (Project No.: MRP/029/20X), and Multi-scale Medical Robotics Center Limited.
K.W. Kwok and V. Vardhanabhuti are co-corresponding authors.
J.D., X.W., Z.L., and J.X. conceived and designed the study supervised by K.W.K., V.V., and Q.D.; V.V. was responsible for collecting data from the Hong Kong hospital local cohorts as well as conducting manual annotations; Z.L. conducted data-cleaning, weak label generation, and image processing; J.D. conducted deep learning algorithm implementations, experimental design, data analysis, and drafted the paper with the assistance and feedback of all other co-authors; Y.L. developed the open-sourced software; K.W.K., X.W., Y.L., and Y.L.N., revised the paper draft, and V.V. and Q.D. provided constructive feedback on the developed models; All authors read and approved the final manuscript.
Conflict of Interest
The authors declare no conflict of interest.
Data Availability Statement
The public dataset
I2CVB can be available online. Raw images and image-level labels of the public dataset
PROSTATEx are also accessed online. The other local cohort datasets used in this study are not publicly available at this time, as the data from the third party contain patients’ confidential information and are not authorized to be shared openly at this stage. Qualified researchers with reasonable requests for access to the data should contact the corresponding authors after permission from the local hospitals. Any data use will be restricted to non-commercial research purposes.
The
codes to train our models are based on Keras using Tensorflow (v1.15) as backend. The models were implemented using Python (v3.6.5).
References
[1] G. Lemaître, R. Martí, J. Freixenet, J. C. Vilanova, P. M. Walker, F. Meriaudeau, medicine, Computers in biology 2015, 60, 8.
[2] Z. Liu, W. Jiang, K.-H. Lee, Y.-L. Lo, Y.-L. Ng, Q. Dou, V. Vardhanabhuti, K.-W. Kwok, presented at Workshop on Artificial Intelligence in Radiation Therapy, 2019.
[3] Y. Chen, W. Wang, E. J. Schmidt, K.-W. Kwok, A. N. Viswanathan, R. Cormack, Z. T. H. Tse, IEEE/ASME Transactions on Mechatronics 2015, 21, 956.
[4] R. Alkadi, F. Taher, A. El-Baz, N. Werghi, Journal of digital imaging 2019, 32, 793.
[5] C.-L. Cheung, J. D.-L. Ho, V. Vardhanabhuti, H.-C. Chang, K.-W. Kwok, IEEE/ASME Transactions on Mechatronics 2020, 25, 1016.
[6] V. Vitiello, K.-W. Kwok, G.-Z. Yang, in Medical Robotics, Elsevier, 2012.
[7] Z. Dong, Z. Guo, K.-H. Lee, G. Fang, W. L. Tang, H.-C. Chang, D. T. M. Chan, K.-W. Kwok, IEEE Robotics Automation Letters 2019, 4, 1964.
[8] Q. Liu, Q. Dou, L. Yu, P. A. Heng, IEEE transactions on medical imaging 2020, 39, 2713.
[9] X. Wang, Y. Li, K.-W. Kwok, AI, Frontiers in Robotics 2021, 280.
[10] G. Fang, M. C. Chow, J. D. Ho, Z. He, K. Wang, T. Ng, J. K. Tsoi, P.-L. Chan, H.-C. Chang, D. T.-M. Chan, Y.-h. Liu, F. Holsinger, J. Y.-K. Chan, K.-W. Kwok, Science Robotics 2021, 6, eabg5575.
[11] Y. Chen, J. Ge, K.-W. Kwok, K. R. Nilsson, M. Fok, Z. T. Tse, Journal of cardiovascular magnetic resonance 2014, 16, 1.
[12] W. Jiang, Z. Liu, K.-H. Lee, S. Chen, Y.-L. Ng, Q. Dou, H.-C. Chang, K.-W. Kwok, arXiv preprint arXiv:.09745 2019.
[13] a)K. Wang, C. H. Mak, J. D. Ho, Z. Liu, K. Y. Sze, K. K. Wong, K. Althoefer, Y. Liu, T. Fukuda, K.-W. Kwok, Advanced Intelligent Systems 2021, 3, 2100089; b)Y.-L. Ng, M. C. Lo, K.-H. Lee, X. Xie, T. N. Kwong, M. Ip, L. Zhang, J. Yu, J. J. Sung, W. K. Wu, S. H. Wong, K.-W. Kwok, Advanced Intelligent Systems 2021, 3, 2000188.
[14] a)Z. He, Z. Dong, G. Fang, J. D.-L. Ho, C.-L. Cheung, H.-C. Chang, C. C.-N. Chong, J. Y.-K. Chan, D. T. M. Chan, K.-W. Kwok, IEEE Robotics Automation Letters 2020, 5, 2100; b)K.-W. Kwok, K.-H. Lee, Y. Chen, W. Wang, Y. Hu, G. C. Chow, H. S. Zhang, W. G. Stevenson, R. Y. Kwong, W. Luk, Circulation 2014, 130, A18568.
[15] Z. Wang, D. Acuna, H. Ling, A. Kar, S. Fidler, presented at Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[16] K. He, X. Zhang, S. Ren, J. Sun, presented at Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[17] Y. Ganin, V. Lempitsky, presented at International conference on machine learning, 2015.
[18] B. Sun, K. Saenko, presented at European conference on computer vision, 2016.
[19] L. Van der Maaten, G. Hinton, Journal of machine learning research 2008, 9.
[20] Z. Dong, X. Wang, G. Fang, Z. He, J. D.-L. Ho, C.-L. Cheung, W. L. Tang, X. Xie, L. Liang, H.-C. Chang, C. K. Ching, K.-W. Kwok, 2022, IEEE Transactions on Robotics.