Development of object detection and classification with YOLOv4 for similar and structural deformed fish
Abstract
Food scarcity is an issue of concern due to the continued growth of the human population and the threat of global warming and climate change. Increasing food production is expected to meet the challenges of food needs that will continue to increase in the future. Automation is one of the solutions to increase food productivity, including in the aquaculture industry, where fish recognition is essential to support it. This paper presents fish recognition using YOLO version 4 (YOLOv4) on the "Fish-Pak" dataset, which contains six species of identical and structurally damaged fish, both of which are characteristics of fish processed in the aquaculture industry. Data augmentation was generated to meet the validation criteria and improve the data balance between classes. For fish images on a conveyor, flip, rotation, and translation augmentation techniques are appropriate. YOLOv4 was applied to the whole fish body and then combined with several techniques to determine the impact on the accuracy of the results. These techniques include landmarking, subclassing, adding scale data, adding head data, and class elimination. Performance for each model was evaluated with a confusion matrix, and analysis of the impact of the combination of these techniques was also reviewed. From the experimental test results, the accuracy of YOLOv4 for the whole fish body is only 43.01 %. The result rose to 72.65 % with the landmarking technique, then rose to 76.64 % with the subclassing technique, and finally rose to 77.42 % by adding scale data. The accuracy did not improve to 76.47 % by adding head data, and the accuracy rose to 98.75 % with the class elimination technique. The final result was excellent and acceptable
Downloads
References
Ranney, M. A., Velautham, L. (2021). Climate change cognition and education: Given no silver bullet for denial, diverse information-hunks increase global warming acceptance. Current Opinion in Behavioral Sciences, 42, 139–146. doi: https://doi.org/10.1016/j.cobeha.2021.08.001
Bader, F., Rahimifard, S. (2018). Challenges for industrial robot applications in food manufacturing. ISCSIC '18: Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control, 1–8. doi: https://doi.org/10.1145/3284557.3284723
Goncharuk, A. (2015). Food business and food security challenges in research. Journal of Applied Management and Investments, 4 (4), 223–230. Available at: http://www.jami.org.ua/Papers/JAMI_4_4_223-230.pdf
Dos Santos, A. A., Gonçalves, W. N. (2019). Improving pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. doi: https://doi.org/10.1016/j.ecoinf.2019.100977
Alsmadi, M. K., Almarashdeh, I. (2020). A survey on fish classification techniques. Journal of King Saud University - Computer and Information Sciences. doi: https://doi.org/10.1016/j.jksuci.2020.07.005
Zhao, S., Zhang, S., Liu, J., Wang, H., Zhu, J., Li, D., Zhao, R. (2021). Application of machine learning in intelligent fish aquaculture: A review. Aquaculture, 540, 736724. doi: https://doi.org/10.1016/j.aquaculture.2021.736724
Abinaya, N. S. M., Susan, D., Rakesh Kumar, S. (2021). Naive bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries. Ecological Informatics, 61, 101248. doi: https://doi.org/10.1016/j.ecoinf.2021.101248
Ahmed, M. S., Aurpa, T. T., Azad, M. A. K. (2021). Fish disease detection using image based machine learning technique in aquaculture. Journal of King Saud University - Computer and Information Sciences. doi: https://doi.org/10.1016/j.jksuci.2021.05.003
Alshdaifat, N. F. F., Talib, A. Z., Osman, M. A. (2020). Improved deep learning framework for fish segmentation in underwater videos. Ecological Informatics, 59, 101121. doi: https://doi.org/10.1016/j.ecoinf.2020.101121
Mohamed, H. E.-D., Fadl, A., Anas, O., Wageeh, Y., ElMasry, N., Nabil, A., Atia, A. (2020). Msr-yolo: Method to enhance fish detection and tracking in fish farms. Procedia Computer Science, 170, 539–546. doi: https://doi.org/10.1016/j.procs.2020.03.123
Salman, A., Maqbool, S., Khan, A. H., Jalal, A., Shafait, F. (2019). Real-time fish detection in complex backgrounds using probabilistic background modelling. Ecological Informatics, 51, 44–51. doi: https://doi.org/10.1016/j.ecoinf.2019.02.011
Jalal, A., Salman, A., Mian, A., Shortis, M., Shafait, F. (2020). Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57, 101088. doi: https://doi.org/10.1016/j.ecoinf.2020.101088
Fouad, M. M. M., Zawbaa, H. M., El-Bendary, N., Hassanien, A. E. (2013). Automatic nile tilapia fish classification approach using machine learning techniques. 13th International Conference on Hybrid Intelligent Systems (HIS 2013). doi: https://doi.org/10.1109/HIS.2013.6920477
Kutlu, Y., Iscimen, B., Turan, C. (2017). Multi-stage fish classification system using morphometry. Fresenius Environmental Bulletin, 26 (3), 1910–1916. Available at: https://www.researchgate.net/publication/314284234_MULTI-STAGE_FISH_CLASSIFICATION_SYSTEM_USING_MORPHOMETRY
Hu, J., Li, D., Duan, Q., Han, Y., Chen, G., Si, X. (2012). Fish species classification by color, texture and multi-class support vector machine using computer vision. Computers and Electronics in Agriculture, 88, 133–140. doi: https://doi.org/10.1016/j.compag.2012.07.008
Andayani, U., Wijaya, A., Rahmat, R. F., Siregar, B., Syahputra, M. F. (2019). Fish species classification using probabilistic neural network. Journal of Physics: Conference Series, 1235, 012094. doi: https://doi.org/10.1088/1742-6596/1235/1/012094
Mohammadi Lalabadi, H., Sadeghi, M., Mireei, S. A. (2020). Fish freshness categorization from eyes and gills color features using multi-class artificial neural network and support vector machines. Aquacultural Engineering, 90, 102076. doi: https://doi.org/10.1016/j.aquaeng.2020.102076
Pornpanomchai, C., Lurstwut, B., Leerasakultham, P., Kitiyanan, W. (2013). Shape- and texture-based fish image recognition system. Kasetsart Journal - Natural Science, 47 (4), 624–634. Available at: https://www.researchgate.net/publication/289604551_Shape-_and_texture-based_fish_image_recognition_system
Miyazono, T., Saitoh, T. (2017). Fish species recognition based on cnn using annotated image. IT Convergence and Security 2017, 156–163. doi: https://doi.org/10.1007/978-981-10-6451-7_19
Rekha, B. S., Srinivasan, G. N., Reddy, S. K., Kakwani, D., Bhattad, N. (2020). Fish detection and classification using convolutional neural networks. Computational Vision and Bio-Inspired Computing, 1221–1231. doi: https://doi.org/10.1007/978-3-030-37218-7_128
Taheri-Garavand, A., Nasiri, A., Banan, A., Zhang, Y.-D. (2020). Smart deep learning-based approach for non-destructive freshness diagnosis of common carp fish. Journal of Food Engineering, 278, 109930. doi: https://doi.org/10.1016/j.jfoodeng.2020.109930
Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., Villéger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological Informatics, 48, 238–244. doi: https://doi.org/10.1016/j.ecoinf.2018.09.007
Labao, A. B., Naval, P. C. (2019). Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Ecological Informatics, 52, 103–121. doi: https://doi.org/10.1016/j.ecoinf.2019.05.004
Cai, K., Miao, X., Wang, W., Pang, H., Liu, Y., Song, J. (2020). A modified YOLOv3 model for fish detection based on MobileNetv1 as backbone. Aquacultural Engineering, 91, 102117. doi: https://doi.org/10.1016/j.aquaeng.2020.102117
Villon, S., Iovan, C., Mangeas, M., Claverie, T., Mouillot, D., Villéger, S., Vigliola, L. (2021). Automatic underwater fish species classification with limited data using few-shot learning. Ecological Informatics, 63, 101320. doi: https://doi.org/10.1016/j.ecoinf.2021.101320
Ju, Z., Xue, Y. (2020). Fish species recognition using an improved alexnet model. Optik, 223, 165499. doi: https://doi.org/10.1016/j.ijleo.2020.165499
Qin, H., Li, X., Liang, J., Peng, Y., Zhang, C. (2016). DeepFish: Accurate underwater live fish recognition with a deep architecture. Neurocomputing, 187, 49–58. doi: https://doi.org/10.1016/j.neucom.2015.10.122
Islam, M. A., Howlader, M. R., Habiba, U., Faisal, R. H., Rahman, M. M. (2019). Indigenous fish classification of bangladesh using hybrid features with svm classifier. 2019 International Conference on Computer, Communication, Chemical, Materials and Electronic Engineering (IC4ME2). doi: https://doi.org/10.1109/IC4ME247184.2019.9036679
Robotham, H., Castillo, J., Bosch, P., Perez-Kallens, J. (2011). A comparison of multi-class support vector machine and classification tree methods for hydroacoustic classification of fish-schools in Chile. Fisheries Research, 111 (3), 170–176. doi: https://doi.org/10.1016/j.fishres.2011.07.010
Kutlu, Y., Reyhaniye, A. N., Turan, C. (2014). Image analysis methods on fish recognition. 2014 22nd Signal Processing and Communications Applications Conference (SIU). doi: https://doi.org/10.1109/SIU.2014.6830503
Badawi, U., Alsmadi, M. (2014). A general fish classification methodology using meta-heuristic algorithm with back propagation classifier. Journal of Theoretical and Applied Information Technology, 66 (3), 803–812. Available at: http://www.jatit.org/volumes/Vol66No3/18Vol66No3.pdf
Liu, Z., Jia, X., Xu, X. (2019). Study of shrimp recognition methods using smart networks. Computers and Electronics in Agriculture, 165, 104926. doi: https://doi.org/10.1016/j.compag.2019.104926
Pettersen, R., Braa, H., Gaweł, B., Letnes, P., Sæther, K., Aas, L. (2019). Detection and classification of Lepeophterius salmonis (Krøyer, 1837) using underwater hyperspectral imaging. Aquacultural Engineering, 87, 102025. doi: https://doi.org/10.1016/j.aquaeng.2019.102025
Liawatimena, S., Heryadi, Y., Lukas, Trisetyarso, A., Wibowo, A., Abbas, B. S., Barlian, E. (2018). A Fish Classification on Images using Transfer Learning and Matlab. 2018 Indonesian Association for Pattern Recognition International Conference (INAPR), 108–112. doi: https://doi.org/10.1109/INAPR.2018.8627007
Shah, S. Z. H., Rauf, H. T., IkramUllah, M., Khalid, M. S., Farooq, M., Fatima, M., Bukhari, S. A. C. (2019). Fish-pak: Fish species dataset from pakistan for visual features based classification. Data in Brief, 27, 104565. doi: https://doi.org/10.1016/j.dib.2019.104565
Rauf, H. T., Lali, M. I. U., Zahoor, S., Shah, S. Z. H., Rehman, A. U., Bukhari, S. A. C. (2019). Visual features based automated identification of fish species using deep convolutional neural networks. Computers and Electronics in Agriculture, 167, 105075. doi: https://doi.org/10.1016/j.compag.2019.105075
Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y. M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv. doi: https://doi.org/10.48550/arXiv.2004.10934
Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. arXiv. doi: https://doi.org/10.48550/arXiv.1506.02640
Minh, D. H. T., Ienco, D., Gaetano, R., Lalande, N., Ndikumana, E., Osman, F., Maurel, P. (2018). Deep Recurrent Neural Networks for Winter Vegetation Quality Mapping via Multitemporal SAR Sentinel-1. IEEE Geoscience and Remote Sensing Letters, 15 (3), 464–468. doi: https://doi.org/10.1109/LGRS.2018.2794581
Copyright (c) 2022 Ari Kuswantori, Taweepol Suesut, Worapong Tangsrirat, Navaphattra Nunak
This work is licensed under a Creative Commons Attribution 4.0 International License.
Our journal abides by the Creative Commons CC BY copyright rights and permissions for open access journals.
Authors, who are published in this journal, agree to the following conditions:
1. The authors reserve the right to authorship of the work and pass the first publication right of this work to the journal under the terms of a Creative Commons CC BY, which allows others to freely distribute the published research with the obligatory reference to the authors of the original work and the first publication of the work in this journal.
2. The authors have the right to conclude separate supplement agreements that relate to non-exclusive work distribution in the form in which it has been published by the journal (for example, to upload the work to the online storage of the journal or publish it as part of a monograph), provided that the reference to the first publication of the work in this journal is included.
